Senior Data Engineer (3rd Party Contractor)
DVT · Melrose Arch, Gauteng, South Africa ·
- Work mode
- Hybrid
- Seniority
- Senior
- Employment
- Contract
- Category
- Data engineering
- Experience
- 8+ years
DVT · Melrose Arch, Gauteng, South Africa ·
Job Title: Senior Data Engineer
Reports to: Practice Head: Data, AI and Automation
ROLE OVERVIEW
As a Senior Data Engineer, you will play a pivotal role in designing, building, and optimizing data systems and pipelines for DVT's clients. You will be responsible for ensuring the seamless flow of data between systems, enabling efficient analytics, and supporting business intelligence efforts. Your work will involve collaborating with cross-functional teams to understand data needs, designing scalable solutions, and implementing data management best practices. Additionally, you will mentor junior engineers, contribute to the strategic vision for data architecture, and stay updated with emerging data engineering technologies.
TECHNICAL KNOWLEDGE
Strong knowledge and extensive experience in:
· Programming Languages: Python, Java, Scala, and SQL.
· Data Engineering Tools: Apache Spark, Apache Beam, Kafka, and Airflow.
· Big Data Technologies: Hadoop, Hive, and Presto.
· ETL/ELT Tools: AWS Glue, Azure Data Factory, and Talend.
· Data Warehousing: Redshift, Snowflake, BigQuery, and Synapse Analytics.
· Cloud Platforms: AWS, Azure, GCP, and Databricks.
· Version Control & CI/CD: Git, Jenkins, Docker, Kubernetes, and Terraform.
Strong knowledge and understanding of:
· Data modeling, schema design, and query optimization.
· Building and maintaining real-time data streaming pipelines.
· Implementing data governance, quality, and security frameworks.
· Data serialization formats such as Avro, Parquet, and ORC
BEHAVIOURAL COMEPTENCIES
RESPONSIBILITIES
MINIMUM EXPERIENCE NEEDED TO BE SUCCESSFUL IN THIS ROLE