to
Position: Senior Data Engineer (Databricks & Cloud Data Platforms)
Reporting to: Practice Head: Data and Automation
ROLE OVERVIEW
As a Senior Data Engineer, you will play a critical consulting role in designing, building, and modernising enterprise data platforms, with a strong focus on Databricks lakehouse implementations across cloud environments.
You will work closely with solution architects, analysts, data scientists, and client stakeholders to deliver secure, scalable, and high-performing analytics platforms that drive real business impact.
This is a hands-on technical role suited to someone passionate about solving complex data challenges and delivering best-in-class cloud-native solutions.
responsibilities
RESPONSIBILITIES
- Design and implement scalable Databricks-based data platforms
- Build robust ETL / ELT pipelines for batch and streaming workloads
- Lead data migration and modernisation initiatives from legacy platforms to cloud-based lakehouse architectures
- Develop high-performance data processing solutions using PySpark, Python, and SQL
- Implement Delta Lake, Unity Catalog, and modern governance frameworks including Purview
- Design enterprise-grade lakehouse and medallion architectures
- Optimise Spark workloads for performance, scalability, and cost efficiency
- Build orchestration solutions using Databricks Workflows, ADF, Airflow, or similar tools in Azure/AWS
- Collaborate with technical and business stakeholders to define solution requirements
- Mentor junior engineers and contribute to technical best practices within the Data & AI practice
- Produce technical documentation, architecture artefacts, and reusable delivery assets
IDEALLY YOU ARE:
- A strong technical leader who can own complex delivery outcomes
- Comfortable engaging directly with clients and stakeholders
- A proactive problem-solver with excellent analytical skills
- Able to clearly communicate technical concepts to both technical and non-technical audiences
- Passionate about mentoring and uplifting engineering teams
- Delivery-focused, quality-driven, and adaptable in consulting environments
MINIMUM EXPERIENCE
- 8+ years in Data Engineering projects
- 3+ years of hands-on Databricks delivery experience
- Deep expertise in Python and PySpark
- Strong understanding of the Medallion Architecture and Lakehouse Architecture
- Hands-on experience building and maintaining live Databricks AI/BI Governance Dashboards using SQL
- Strong experience with Azure data services including Synapse, ADF, OneLake, Purview
- Practical exposure to AWS data platforms including Glue, S3, Redshift
- Proven track record delivering data migration and modernisation projects
- Experience working in consulting or client-facing environments
- Bachelor’s degree in computer science, Engineering, Information Systems, or equivalent practical experience
TECHNICAL REQUIREMENTS
- Databricks Expertise (Essential)
- Cloud Data Platform Experience (Azure Fabric, AWS) (Essential)