Senior Data Engineer

My client is looking for an experienced Senior Data Engineer to join them on a contract basis, taking end‑to‑end ownership of business‑critical data products on their Azure Databricks‑based data platform. This is a hands‑on role embedded directly within a product team, working closely with product managers and lead engineers to raise engineering quality and shape how modern data products are built for analytics and AI.

The Role

As a Senior Data Engineer, you will:

Design scalable data products within a cross‑functional product team

Act as a senior technical partner to product managers and lead data engineers

Build and maintain high‑quality data pipelines using Databricks & Spark

Convert requirements into clear technical designs and conceptual data models

Ingest large, complex datasets and automate manual processes

Improve data quality, observability, reliability, and delivery performance

Champion a build‑once‑consume‑many culture across the platform

Collaborate with stakeholders to solve data‑related engineering challenges

Contribute to engineering standards, best practices, and platform evolution

Mentor other engineers and raise the overall engineering bar

Tech Environment

My client is actively modernising their platform, moving away from legacy IaaS solutions and building a Databricks‑first Lakehouse. Spark, Delta Lake, CI/CD, and modern DevOps practices are central to how they design and scale data products.

What You’ll Need

Strong hands‑on experience with Azure Databricks & Spark in production

5+ years in data engineering or software engineering with a strong data focus

Experience building production‑grade pipelines on Azure

Strong SQL and experience with analytical data models / data warehousing

Proficiency in Python (Scala a bonus)

Experience delivering enterprise‑grade data products used by multiple teams

Ability to communicate clearly with both technical and non‑technical audiences

Experience with Azure Synapse (dedicated/serverless SQL pools)

Agile mindset; experience working in Scrum teams

Nice to Have

Hands‑on experience with Docker, Kubernetes, or container orchestration

Start: ASAP

Location: London (with remote flexibility)

Length: 6 months (likely extension)

Rate: Competitive, depending on experience