Lead Data Engineer

The Senior Data Engineer is responsible for designing, developing, and maintaining scalable data engineering solutions that support the migration from legacy big data platforms to modern, cloud-based data environments. The role ensures reliable data operations while enabling ongoing and new business initiatives.

Duties and Responsibilities

• Design, build, and optimize automated data pipelines, ETL/ELT processes, and data models to ingest, process, and store large volumes of data within cloud-based platforms.
• Support large-scale data migration initiatives, ensuring data accuracy, performance efficiency, and minimal business disruption.
• Develop and maintain ETL/ELT workflows to ingest, transform, and load data from multiple internal and external sources with a focus on scalability and reliability.
• Partner with business, analytics, and product teams to translate data requirements into effective technical solutions that support strategic initiatives.
• Design and deliver data marts and customized data extractions aligned with business and reporting needs.
• Ensure compliance with enterprise data governance, security, and regulatory standards.
• Monitor data pipeline health and performance, troubleshoot data incidents, and implement preventive and corrective measures.
• Document data workflows, schemas, technical specifications, and operational runbooks to support operational stability and knowledge transfer.
• Collaborate closely with product owners, data architects, and data scientists to maintain a reliable and efficient data infrastructure.
• Drive continuous improvement of data engineering practices, tools, and automation frameworks.
• Provide technical guidance and mentorship to junior engineers through code reviews, best-practice sharing, and troubleshooting support.
• Manage and deliver multiple data engineering initiatives concurrently while meeting quality, scope, and timeline expectations.

Required Experience, Technical Knowledge, and Skills

• At least 8 years of total Data Engineering experience, with strong exposure to large-scale data pipelines, ETL/ELT development, and enterprise or cloud-based data platforms.
• Proven experience designing, building, and optimizing scalable data solutions in modern data environments.
• Python – At least 4 out of 5 proficiency level, with strong hands- on experience in data transformation, automation, and pipeline development.
• SQL – At least 3 out of 5 proficiency level, with demonstrated capability in complex queries, data modeling, and performance tuning.
• Experience working with modern data cloud platforms, such as Databricks and/or Snowflake.
• Platform and Cloud Exposure - Currently working with or has recent hands- on experience in enterprise-grade cloud data ecosystems. With a strong understanding of cloud-native data architecture and best practices.
• Strong verbal and written communication skills
• Demonstrated leadership and technical influence
• Strong analytical, critical thinking, and problem-solving abilities
• Process orientation with the ability to enforce standards and best practices
• Strong organizational, multitasking, and time-management skills
• Stakeholder and cross-functional collaboration skills