Senior Data Engineer

We are seeking an experienced Senior Data Engineer to join the Customer Data team within the data and AI function of a leading UK retail organisation.

The team is responsible for building and operating the data foundations that support a wide range of customer-focused use cases, including Customer 360, analytics, AI/ML, marketing and data-driven commercial initiatives.

This is a hands-on engineering role for someone who enjoys solving complex data problems, building scalable solutions and working within modern software engineering environments.

You will play a key role in designing and delivering reliable, production-grade data ingestion and transformation pipelines, while contributing to the wider architecture and technical direction of customer data capabilities.

The successful candidate will be comfortable taking ownership of technical problems, developing solutions independently and working collaboratively with senior engineers and stakeholders to sense-check and evolve those solutions.

Key Responsibilities

• Design, build and operate scalable data ingestion and transformation pipelines for high-volume customer data.
• Develop robust, reliable and production-grade data solutions supporting Customer 360, analytics, AI/ML, marketing and commercial use cases.
• Contribute to solution design and technical architecture, taking a holistic view of systems, integrations, dependencies and potential risks.
• Build and maintain data pipelines with a strong focus on performance, reliability, scalability and data quality.
• Work across development and production environments, following established CI/CD, version control and release processes.
• Identify technical risks, integration points and opportunities for improvement across the wider data estate.
• Collaborate closely with Data Engineers, Software Engineers, Architects, Product and other technical teams to deliver complex data initiatives.
• Contribute to the design of real-time, high-throughput and low-latency data solutions where appropriate.
• Support the development of data capabilities used across customer, analytics, AI/ML and commercial data products.
• Produce clear technical documentation and contribute to maintaining a well-governed and supportable data environment.
• Proactively investigate and resolve complex technical and data engineering issues.
• Take ownership of engineering problems from initial solution through to deployment and ongoing operation.

About You

We are looking for a software-engineering-minded Data Engineer with strong foundations in data engineering and modern engineering practices.

You should be someone who can take a problem, develop a pragmatic solution with relatively little oversight and then work with senior technical colleagues to sense-check and refine the approach.

You will ideally have:

• Strong commercial experience as a Data Engineer, working on production-scale data platforms and pipelines.
• Strong software engineering fundamentals, ideally with a background in Computer Science, Software Engineering or a similar technical discipline.
• Proven experience designing and building data ingestion, transformation and integration pipelines at scale.
• Strong understanding of software engineering practices including Git/GitHub, CI/CD, development environments, testing and release management.
• Strong hands‑on experience with Python and SQL.
• Experience working with modern cloud-based data platforms across AWS, Azure and/or GCP.
• Experience with contemporary data engineering technologies and architectures.
• Strong understanding of data modelling, distributed systems and scalable architecture patterns.
• A proactive approach to problem solving, with the ability to work independently and identify issues before they become blockers.
• Experience working in engineering teams where quality, maintainability, observability and production reliability are important.
• Strong communication skills and the ability to collaborate effectively with engineers, technical leads and wider stakeholders.

Technical Environment

Experience with some of the following technologies would be beneficial:

• Python
• SQL
• Cloud platforms – AWS, Azure or GCP
• Databricks
• Kafka / event-driven architectures
• Airflow
• dbt
• Git / GitHub
• CI/CD
• Infrastructure as Code
• Modern data platforms and distributed processing technologies

Experience with Databricks or Hadoop-style data processing environments is useful, although candidates with strong transferable experience across modern data platforms will be considered.

Customer Data & Retail Context

Experience within retail or e-commerce would be advantageous, particularly exposure to customer data environments, MarTech, CRM or digital customer journeys.

Experience with any of the following would also be beneficial:

• Customer 360
• Customer Data Platforms
• Identity resolution
• Reverse ETL
• MarTech / CRM integrations
• Data monetisation
• High-throughput or low-latency data systems
• Customer data governance and privacy

These areas are not essential, provided you have strong core Data Engineering and software engineering experience.

Working Arrangement

The role is based in Waterloo, London, with hybrid working and occasional attendance at the office.

The organisation is open to flexibility around the frequency of office attendance, although candidates based in the UK are strongly preferred due to collaboration, communication and occasional face‑to‑face requirements.

This is an opportunity to join a well-established data engineering function working on large‑scale customer data challenges across retail, analytics, AI/ML and commercial use cases.