Senior Data Engineer

We’re working with an established technology consultancy, and they’re looking for a Senior Data Engineer to architect, build and lead scalable, secure data solutions as part of an agile team.

You’ll act as a technical lead — turning complex requirements into high-performing data architecture, and building the infrastructure that turns raw data into insight at scale, in batch and real time.

What you’ll do

• Architect the data-analytics framework and lead batch and real-time analytical solutions
• Translate complex functional and technical requirements into detailed architecture, design and high-performing software
• Lead user-story analysis, design, development, testing and build-automation across multiple projects
• Create data marts using Talend or a similar ETL tool, and manipulate data using Python
• Process data using the Hadoop paradigm (e.g. EMR), and create and support data feeds into AWS Cloud
• Develop for big data and business intelligence, including automated testing and deployment

What you’ll bring

• 6+ years’ data-engineering or software-engineering experience, with 2+ years in big data
• 2+ years with ETL processes and 2+ years’ AWS experience
• Strong Python, and PySpark or Spark
• The ability to architect highly scalable distributed systems using open-source tools
• Expert data modelling, and strong object-oriented design, coding and testing patterns
• A bachelor’s degree in Computer Science, Computer Engineering or equivalent, and AWS certification

Advantageous

• BigQuery, DBT, and a combination of AWS / GCP / Azure
• AI tooling (e.g. Bedrock or similar)

Working model

Hybrid in Johannesburg — three days a week in the office.

#J-18808-Ljbffr