Lead Data Engineer
Lorien Resourcing · London, United Kingdom ·
- Seniority
- Lead
- Category
- Data engineering
A senior hands-on Lead Data Engineer role in London, anchoring end-to-end delivery of enterprise-scale data pipelines, platforms and services across legacy and cloud environments. Core stack is Python and Apache Spark, with strong emphasis on data quality, governance, lineage and technical leadership.
In this senior, hands-on role you will anchor data engineering work for an enterprise platform, driving end-to-end delivery of robust data pipelines and services. You’ll operate across legacy and cloud environments to improve real-world data systems, guiding teammates through design decisions and technical delivery. The role emphasizes practical problem-solving, data quality, and governance in a fast-paced, complex setting. You’ll work closely with stakeholders to translate operational challenges into scalable solutions and help raise engineering standards.
Responsibilities- Design, build, and enhance enterprise-scale data pipelines, platforms, and data services
- Lead end-to-end data engineering work from problem definition to operational use
- Collaborate with technical and non-technical stakeholders to translate problems into data solutions
- Analyse and synthesize data from multiple sources to shape structure, validation, and consumption
- Provide technical leadership through design decisions, code, and delivery support
- Improve data sharing, onboarding of new sources, and interoperability across teams
- Evaluate and adopt tools and patterns that provide genuine value while balancing pragmatism
- Develop robust, production-grade solutions and address metadata, lineage, and data quality as core concerns
- Diagnose and resolve complex data and platform issues in constrained environments
- Contribute to an inclusive engineering culture with high technical standards
- Hands-on experience as a Senior or Lead Data Engineer on complex, real-world systems
- Strong Python and Spark skills, with production data pipelines experience
- Deep understanding of the full data engineering lifecycle (ingestion, transformation, storage, serving, reuse)
- Experience designing data integrations with diverse sources and legacy systems
- Ability to design data models for analysis, operational use, and maintainability
- Knowledge of data security, compliance, and governance integrated into systems
- Experience with established data development practices (version control, testing, deployment, change management)
- Clear communication between technical and non-technical stakeholders
- Persistence tackling messy data and organizational complexity
- Clear communication with technical and non-technical stakeholders
- Persistence and resilience in challenging environments
- Collaboration and teamwork
- Python
- Apache Spark
- End-to-end data pipeline development