Azure Data Engineer

The Azure Data Engineer will be responsible for designing, building, optimizing, and maintaining data pipelines and data solutions within the Microsoft Azure ecosystem. The role requires strong experience in cloud-based data engineering, modern data architectures, and scalable data processing frameworks.

Key Responsibilities

• Design, develop, and maintain Azure-based data pipelines (batch and streaming).
  
• Implement ETL/ELT solutions using Azure Data Factory (ADF), Azure Databricks, Synapse Analytics, and related Azure services.
  
• Build and optimize data models, including Data Lakehouse architectures (Medallion, Delta Lake).
  
• Ensure data quality, integrity, and governance across all data processes.
  
• Collaborate with data scientists, analysts, and business teams to provide clean, reliable datasets.
  
• Perform pipeline monitoring, troubleshooting, performance optimization, and cost optimization.
  
• Develop CI/CD pipelines using Azure DevOps for data-related deployments.
  
• Implement security best practices, including RBAC, encryption, and data access governance.
  

Required Technical Skills

• Strong proficiency with Azure Data Factory (ADF), Azure Databricks (PySpark), and Azure Synapse Analytics.
  
• Expert knowledge of Python, SQL, and data transformation frameworks.
  
• Experience with Delta Lake, Spark, Data Lake Storage (ADLS), and distributed compute patterns.
  
• Familiarity with Data Vault, Dimensional Modelling, or Medallion architecture.
  
• Experience with cloud DevOps tools (Azure DevOps, GitHub Actions).
  
• Knowledge of streaming technologies (Kafka, Event Hubs) is an advantage.
  

Qualifications & Certifications (Preferred)

• Azure certifications such as:
  • AZ‑900 Azure Fundamentals
    
  • DP‑203 Azure Data Engineer Associate
    
  • Databricks Data Engineer certifications
    
• Bachelor’s degree in Computer Science, Engineering, or related field.
  

Experience

• 5+ years of data engineering experience (Azure preferred).
  
• Proven experience building and supporting production-scale cloud data solutions.
  

Location & Working Model

• Location: Johannesburg (Hybrid working model)
  
• Ways of Working: Standard SAST business hours
  
• Candidate: Preferably a South African Citizen or Permanent Resident