About the Company:
Our client is an AI-native technology and data engineering company focused on building modern data platforms and decision systems that help organizations turn complex data into actionable business outcomes.
The company works across data platforms, cloud modernization, and production AI solutions, with a strong emphasis on building systems that are reliable, transparent, scalable, and easy to understand. Its approach combines engineering expertise with a focus on making data and AI outputs decision-ready, measurable, and trustworthy.
Serving mid-market and enterprise organizations across industries such as financial services, retail, healthcare, and industrial sectors, the company partners closely with clients to solve complex technology challenges and deliver solutions grounded in real-world business impact.
About the Role:
We are looking for a Lead Data Engineer to join our delivery team. You will design, build, and own data platforms end to end on client engagements, working within a Medallion architecture pattern using dbt for transformations and Python for ingestion on the client's cloud of choice. You will also contribute to company's internal accelerator programme, reusable components such as our entity resolution engine, ingestion framework, and data quality layer deployed across client projects. This is a hands-on senior role, you write production code, make architectural decisions, mentor junior engineers, and engage directly with client technical teams. We are looking for someone with 5 to 7 years of hands-on data engineering experience.
What You Will Do:
- Design and build end-to-end data platforms using the Medallion architecture, Bronze ingestion via Python, Silver staging and canonical models via dbt, Gold reporting and serving layer for BI consumption.
- Build and own Python-based ingestion pipelines, extracting from SQL databases, Snowflake, REST APIs, and SaaS platforms, with watermark patterns, control tables, and schema drift handling.
- Write production-grade dbt models for Silver and Gold layers, staging models, canonical entity resolution, SCD Type 2 dimensions, and reporting models shaped for BI tools.
- Design canonical data models across multiple source systems, resolving the same entity across disparate platforms using surrogate keys, source ID preservation, and conflict resolution logic.
- Build and contribute to company's accelerator library, entity resolution engine, ingestion framework, data quality profiler, and insights dashboard scaffold.
- Set up and maintain CI/CD pipelines using GitHub Actions, including dbt test automation, data quality gates, and deployment approval workflows.
- Engage directly with client technical teams, running architecture discussions and communicating decisions clearly to both technical and non-technical audiences.
- Coach and review junior engineers through PR reviews, pair programming, and raising the quality bar across the team.
Must-Have Skills:
Data Engineering Core:
- Expert-level SQL, complex joins, window functions, CTEs, incremental merge patterns, and performance tuning.
- Strong Python, production ingestion pipelines, REST API clients, error handling, logging, and retry logic.
- dbt (any adapter), model structure, incremental materializations, tests, macros, and packages.
- Medallion architecture, deep understanding of Bronze, Silver, and Gold layer responsibilities and boundaries.
- Data modelling, canonical modelling, dimensional modelling, star schema, SCD Type 2, and the distinction between integration models and reporting models.
Cloud and Platform:
- Hands-on delivery experience on at least one major cloud, AWS (S3, Redshift, Glue, Athena), GCP (BigQuery, Dataproc, Cloud Storage), or Azure (Fabric, Synapse, ADLS), with working knowledge of the others.
- Lakehouse platforms, Delta Lake, Apache Iceberg, or similar, and how data is physically stored, partitioned, and queried.
- Infrastructure fundamentals, networking, secrets management, managed identity or IAM roles, and container-based compute.
Engineering Practices:
- Git and GitHub, trunk-based branching, pull requests, code review, and CI/CD automation.
- Data quality, dbt tests, quarantine patterns, control tables, and schema drift detection.
- 5 to 7 years of hands-on data engineering experience across ingestion, transformation, modelling, and client delivery.
Desired Skills:
- Entity resolution, probabilistic or deterministic matching across multiple source systems, surrogate key generation, and lineage preservation.
- BI and reporting, Power BI, Looker, or Tableau, and the division of logic between SQL preparation and BI-layer calculations.
- AI and LLM integration, prompt engineering, retrieval-augmented generation, or AI-assisted workflows built on a structured data layer.
- Terraform or other infrastructure-as-code for cloud resource provisioning.
- Domain experience in financial services, facility management, automotive, or professional services.
What We Look For Beyond the Tech:
- You take ownership, if something is wrong you fix it, you do not wait to be asked.
- You communicate clearly, you can explain an architecture decision to a client CTO and to a junior engineer in the same session.
- You are rigorous, you write tests, document decisions, and think about what breaks before it breaks.
- You are curious, the tools change fast and you keep up and bring that forward to the team.
How We Will Support Your Growth:
- Direct exposure to senior engineers and leadership on real client deliveries, not in a siloed unit.
- Ownership of accelerator components that go into multiple client engagements and are yours to evolve.
- Support for cloud and platform certifications, technical writing, and community participation.
- A culture where commitment matters more than compliance, and intellectual honesty is expected in every conversation.
- As the team grows, opportunity to move into a practice lead or architecture leadership role.