Lead Data Engineer

About the role

The Lead Data Engineer is responsible for designing, developing, and maintaining scalable data engineering solutions that support the migration from legacy big data platforms to modern, cloud-based data environments. The role ensures reliable data operations while enabling ongoing and new business initiatives.

Key responsibilities

• Design, build, and optimize automated data pipelines, ETL/ELT processes, and data models to ingest, process, and store large volumes of data within cloud-based platforms.

• Support large-scale data migration initiatives, ensuring data accuracy, performance efficiency, and minimal business disruption.

• Develop and maintain ETL/ELT workflows to ingest, transform, and load data from multiple internal and external sources with a focus on scalability and reliability.

• Partner with business, analytics, and product teams to translate data requirements into effective technical solutions that support strategic initiatives.

• Design and deliver data marts and customized data extractions aligned with business and reporting needs.

• Ensure compliance with enterprise data governance, security, and regulatory standards.

• Monitor data pipeline health and performance, troubleshoot data incidents, and implement preventive and corrective measures.

• Document data workflows, schemas, technical specifications, and operational runbooks to support operational stability and knowledge transfer.

• Collaborate closely with product owners, data architects, and data scientists to maintain a reliable and efficient data infrastructure.

• Drive continuous improvement of data engineering practices, tools, and automation frameworks.

About you

• At least 8 years of total Data Engineering experience, with strong exposure to large-scale data pipelines, ETL/ELT development, and enterprise or cloud-based data platforms.

• Proven experience designing, building, and optimizing scalable data solutions in modern data environments.

• Python – At least 4 out of 5 proficiency level, with strong hands-on experience in data transformation, automation, and pipeline development.

• SQL – At least 3 out of 5 proficiency level, with demonstrated capability in complex queries, data modeling, and performance tuning.

• Experience working with modern data cloud platforms, such as Databricks and/or Snowflake.

• Experience with cloud services, preferable Microsoft Azure (e.g., Azure Data Factory, Azure Synapse, Azure Storage, etc.).

• Strong verbal and written communication skills.

• Demonstrated leadership and technical influence.

• Strong analytical, critical thinking, and problem-solving abilities.

• Stakeholder and cross-functional collaboration skills.