Lead Data Engineer

Salary: £80,000 - 115,000 per year

Requirements:

• 7+ years of experience in Data Engineering or Backend Engineering.
• Strong Python development experience.
• Experience building distributed data platforms and scalable pipelines.
• Excellent knowledge of PostgreSQL or similar relational databases and NoSQL databases.
• Experience with vector databases such as Qdrant, Milvus or pgvector.
• Familiarity with Apache Spark, Airflow, Kafka, Elasticsearch/OpenSearch, Pandas and Polars.
• Strong understanding of data modelling, storage architecture and performance optimisation.
• Experience working on AI or Machine Learning platforms.
• Knowledge of cloud platforms such as AWS, Azure or GCP.
• JavaScript/Node.js experience.
• Event-driven architecture and stream processing experience.
• Previous experience within a high-growth start-up environment.

Responsibilities:

• Design and build scalable batch and real-time data pipelines.
• Develop data ingestion, transformation and storage solutions for operational and AI workloads.
• Build and optimise vector search, retrieval systems and ML data pipelines.
• Ensure data reliability, governance, monitoring and observability across the platform.
• Work closely with AI, backend and product teams to support model training, inference and product development.
• Optimise large-scale datasets and database performance.
• Help define engineering standards and mentor future data engineers.

Technologies:

• AI
• Airflow
• AWS
• OpenSearch
• Azure
• Backend
• Cloud
• ElasticSearch
• GCP
• Support
• JavaScript
• Kafka
• Machine Learning
• NoSQL
• PostgreSQL
• Python
• Spark
• pandas
• NodeJS
• C#
• Embedded
• FPGA
• Golang
• Linux

More:

We are an exciting AI start-up developing an intelligent automation platform that is transforming B2C service. We are hiring a Lead Data Engineer in London on a hybrid basis, with a salary of £80,000-£115,000 plus equity options. As one of our first senior data hires, you will have the opportunity to shape our data architecture, work directly alongside founders and senior engineers, and build technology with high ownership, autonomy and technical influence as we grow rapidly.

last updated 39 week of 2026