Data Engineering Lead - Databricks

Overview

You will lead a data engineering team within a UK organisation undergoing a data transformation, driving technical excellence and the growth of a Databricks Lakehouse. In this hybrid role, you shape the enterprise data platform while staying hands-on to deliver scalable data pipelines and trusted data solutions. You’ll partner with Architecture, BI, Data Science and Technology teams to enable business growth and innovation. This is a chance to influence technology decisions and mentor a high-performing team in a critical transformation effort.

Pay / Benefits

• Enhanced pension contribution
• Generous holiday allowance
• Healthcare and wellbeing benefits
• Enhanced family leave policies

Responsibilities

• Lead, mentor and develop a team of Data Engineers
• Provide technical leadership across data platforms, products and pipelines
• Drive the migration and enhancement of a Databricks-based data platform
• Design and maintain scalable data pipelines and curated datasets
• Support wider transformation initiatives including cloud and SaaS programmes
• Collaborate with Architecture, BI, Data Science and Technology teams
• Champion engineering best practice across testing, deployment, monitoring and code quality
• Ensure data platforms remain reliable, secure and high performing
• Promote Agile ways of working and continuous improvement

Key requirements

• Experience leading Data Engineering teams in a complex enterprise environment
• Strong hands-on expertise in modern cloud-based data platforms
• Excellent SQL and Python skills
• Strong experience with Databricks and Lakehouse architectures
• Deep understanding of data engineering, data modelling and platform design
• Experience delivering solutions that support reporting, analytics and data science
• Strong stakeholder management and communication skills
• Comfortable balancing technical leadership with hands-on engineering responsibilities

• Stakeholder management
• Communication skills
• Mentoring and teamwork
• Databricks
• Lakehouse architectures
• SQL