Data and AI Engineer

Location: Bedfordview, Johannesburg | Hybrid
Employment Type: Permanent | Full-Time
Industry: Data Engineering | Applied AI | Financial Services

WatersEdge Solutions is partnering with an established South African financial services business to find a Data and AI Engineer who wants to work across both modern data engineering and practical, governed AI.

This is a genuinely combined role rather than a traditional Data Engineer position with AI added on the side. Initially, the focus will be approximately two-thirds data engineering and one-third applied AI, with the AI component expected to grow as adoption increases. You’ll build and support the data foundations that power analytics, reporting and AI while helping operate an internally governed AI platform.

About the Role

You’ll work within a Microsoft-centred environment spanning SQL Server, Azure SQL, Microsoft Fabric, Power BI, SSRS, Python and Azure DevOps.

On the data engineering side, you’ll develop reliable ELT pipelines, work with Lakehouse and Warehouse assets, support dimensional and Medallion Architecture, and ensure data quality from source through to reporting.

On the AI side, you’ll prepare trusted datasets and retrieval indexes, support the in-house AI platform and build Python automation for AI workloads. You’ll also help develop and evaluate practical AI use cases across search, summarisation, classification, extraction and workflow automation.

This is a production-first environment. Source control, peer review, gated deployment, auditability, documentation, data quality and human review of AI-generated outputs are fundamental parts of how the team works.

Key Responsibilities

• Develop and maintain batch and incremental ELT pipelines across SQL Server, Azure SQL and Microsoft Fabric.

• Write and optimise T-SQL, including transformation logic, stored procedures, views, CTEs and reconciliation queries.

• Build and maintain Microsoft Fabric pipelines, Dataflow Gen2 flows, Lakehouse and Warehouse assets.

• Support dimensional modelling and Medallion Architecture across bronze, silver and gold layers.

• Manage orchestration, scheduling, monitoring, logging and alerting for data jobs.

• Investigate and resolve data and platform incidents methodically.

• Implement data validation, reconciliation, row-count and parity checks.

• Build and maintain curated datasets, retrieval indexes and interfaces for AI workloads.

• Support the governed AI platform across content ingestion, indexing, configuration, access control and health checks.

• Develop Python automation for data preparation, embedding and indexing jobs, evaluation frameworks and approved API integrations.

• Build and evaluate AI-assisted solutions for search, summarisation, classification, extraction and workflow automation.

• Incorporate human-review checkpoints and defined success criteria into AI use cases.

• Apply strict data minimisation, anonymisation, security and confidentiality requirements.

• Maintain traceable evidence of AI evaluations and human review.

• Work with Git and Azure DevOps, including feature branches, peer-reviewed pull requests and gated deployment from development through UAT to production.

• Produce pipeline design notes, source-to-target mappings, runbooks, AI configuration notes and support documentation.

• Work with analysts, leadership and business stakeholders to translate requirements into governed, supportable data solutions.

• Grow towards end-to-end ownership of production data and AI assets: build it, document it and support it.

What You’ll Bring

• Minimum 1 year of hands-on experience in data engineering, ETL/ELT, analytics engineering, database development, applied AI or a closely related production-focused role.

• Strong hands-on SQL Server and T-SQL skills.

• Confidence working with joins, aggregations, CTEs and multi-step transformation logic.

• Ability to troubleshoot and reason about SQL query performance.

• Demonstrable Microsoft Fabric experience – you must have built, supported or substantially prototyped at least one Fabric pipeline, Lakehouse/Warehouse solution, Dataflow Gen2 flow or Fabric notebook.

• Understanding of data warehousing and pipeline design, including staging, incremental loads and dimensional modelling.

• Working Python capability, including the ability to read, write and modify scripts for data preparation, automation and integrations.

• Practical experience using modern LLM or AI-assisted tools on real work.

• Ability to critically evaluate AI outputs rather than simply accepting generated results.

• Familiarity with Git or similar source-control tooling.

• Strong commitment to data quality, security, least privilege and confidentiality.

• A willingness to take ownership from development through documentation and production support.

Nice to Have

• Stronger Python or PySpark experience for large-scale data transformation.

• Dataflow Gen2 experience.

• Mirroring and CDC exposure.

• Fabric deployment pipeline experience.

• Power BI semantic modelling and DirectLake knowledge.

• SSRS report development.

• Experience integrating APIs, file feeds or enterprise source systems.

• Knowledge of Retrieval-Augmented Generation (RAG), vector search, semantic search or embeddings.

• Azure AI Services experience.

• Prompt and AI evaluation frameworks.

• Agent tooling or MLOps exposure.

• DP-700 Fabric Data Engineer Associate or another relevant Microsoft certification.

• Relevant qualification in Computer Science, Information Systems, Engineering or a related discipline, or equivalent practical experience.

• Experience within financial services or another regulated environment.

What’s On Offer

• Permanent hybrid position based in Bedfordview.

• Hands-on exposure across both Data Engineering and Applied AI.

• Opportunity to work extensively with Microsoft Fabric, SQL Server, Azure SQL, Python and Azure DevOps.

• Exposure to a governed AI platform incorporating retrieval, indexing and agent tooling.

• Practical experience building AI solutions in an environment where responsible implementation and human review genuinely matter.

• Training and development across the broader technology stack.

• Opportunity to grow towards full ownership of production data and AI assets.

Company Culture

The team values fundamentals, curiosity, ownership and quality. There is room to learn much of the wider technology stack, but the successful candidate needs to arrive with practical SQL, Microsoft Fabric and Python capability.

This environment will suit someone who asks good questions, welcomes feedback and verifies results before trusting them – particularly when working with AI. The team values people who take responsibility from build through to production support and who see governance, security and documentation as part of good engineering rather than unnecessary process.

Please Note: If you have not been contacted within 10 working days, consider you application unsuccessful.