Senior Data Engineer
Paystack · Abidjan, CI ·
- Work mode
- Hybrid
- Seniority
- Senior
- Category
- Data engineering
- Experience
- 2+ years
Paystack · Abidjan, CI ·
Paystack’s mission is to power African ambition. Over 300K businesses across Nigeria, South Africa, Ghana, and Kenya use Paystack’s modern payments gateway, including Qatar Airways, MTN, Burger King, UPS, Africa World Airlines, AXA Mansard Insurance, FairMoney, PiggyVest, Crocs, Under Armour, Richemont Lifestyle Group, and many others.
Over the last 10 years, we’ve built products that have helped shape online payments in Africa, from automated recurring payments to direct bank payments and automated chargebacks. Today, Paystack is part of The Stack Group (TSG), a family of technology brands building modern infrastructure across payments, banking, consumer products, and emerging technologies.
At Paystack, we hire talented people, treat them with genuine respect, and give them the space, resources, and support to do the best work of their lives. We’d love your help.
Data engineering at Paystack focuses on building and extending platforms for managing data at scale. This involves data ingestion, processing, storage and egress. Data engineers are also responsible for creating and maintaining the infrastructure our data platforms run on.
Data engineers operate across a diverse tech stack. They are expected to be adaptable and drawn to learning new skills and technologies.
The role requires a proactive individual who can work independently and collaboratively within a remote-first environment, has a strong software engineering background with good experience building and maintaining data pipelines, expertise in Python and experience in streaming technologies.
Data Pipeline Development: Design, develop, and maintain robust data pipelines using ETL and ELT methodologies to process and integrate data from various sources into a data lake, a central data warehouse, operational data stores, analytical data marts and various application interfaces.
Streaming Data Processing: Implement and manage real-time data streaming solutions utilising Kafka, Debezium, Kafka Connect.
Workflow Orchestration: Build, schedule and maintain custom workflows using Apache Airflow to ensure timely and accurate data processing and delivery.
Database Management: Work with a variety of database technologies, including relational databases (MySQL, PostgreSQL), NoSQL databases (MongoDB) and analytical/big data systems (Redshift, BigQuery, SingleStore).
Infrastructure as Code: Employ tools like Terraform, Kubernetes, and Helm to manage and provision infrastructure efficiently.
CI/CD Implementation: Develop and maintain continuous integration and deployment pipelines to streamline development processes.
Testing and Quality Assurance: Conduct unit and integration testing to ensure high code quality, data integrity and system reliability.
Collaboration: Engage with cross-functional teams, including data scientists, analysts, and business stakeholders, to understand data needs and deliver solutions.
Documentation: Maintain clear and comprehensive documentation of data processes, workflows and systems.
Monitoring and Support: Monitoring system performance and addressing faults and failures in production systems as part of an on-call rotation ****
Bachelor's degree in Computer Science, Engineering or a related field.
Proficiency in Python is essential. JavaScript and Scala development experience is advantageous.
Minimum of 3 years of experience in data engineering roles, with a focus on building and managing data pipelines.
Minimum of 2 years experience in software and/or application development roles (can be concurrent with data engineering experience)
Hands-on experience with Kafka, Debezium, and Kafka Connect.
Proficiency in a data pipeline orchestration tool or suitable workflow orchestration tool like Apache Airflow (preferred), Databricks, Dagster or Airbyte.
Strong understanding and hands-on experience working with various database technologies, including MySQL, PostgreSQL, MongoDB and Redshift (BigQuery and SingleStore advantageous)
Experience with Terraform, Kubernetes, and Helm for infrastructure management.
Solid knowledge of cloud computing concepts, with experience in AWS services being advantageous.
Ability to write complex SQL queries across different dialects.
Familiarity with unit and integration testing methodologies.
Experience in setting up and maintaining CI/CD pipelines.
Exposure to analytical systems and basic data science tooling. Familiarity with basic machine learning and analytical modelling concepts advantageous.
Exposure to self-service reporting tools like Tableau, Looker and DOMO.
Soft Skills
Good verbal and written communication skills, with the ability to convey complex technical concepts to non-technical stakeholders.
Demonstrated ability to work collaboratively within a team and across departments.
Comfortable working in a fast-paced environment with changing priorities, technologies and tooling. Life-long learners will do well here.
Strong analytical and problem-solving skills.
Competitive compensation package and benefits
13th month bonus
TSG Equity compensation
Full medical coverage
Wellbeing stipend
Generous leave and sabbatical policies
Hybrid working environment
Smart, kind colleagues who are invested in your growth.
Paystack is an equal opportunity employer and prohibits discrimination and harassment of any kind. We’re committed to providing employees with a work environment that is progressive and open-minded. Our employment philosophy is to hire the best people and empower them to do the best work of their lives. Employment decisions are based on business needs and individual merit without regard to race, color, religion, ethnicity, sexual orientation, nationality, marital status, gender, or age.