Senior Azure Data Engineer
Agensi Pekerjaan RecruitFirst Sdn Bhd · Kuala Lumpur, Malaysia ·
- Seniority
- Senior
- Category
- Data engineering
- Experience
- 5+ years
Agensi Pekerjaan RecruitFirst Sdn Bhd · Kuala Lumpur, Malaysia ·
RecruitFirst Pte Ltd · Malaysia
RecruitFirst Pte Ltd · Malaysia
Health-E Commerce · United States
weekday-1 · Mumbai, Maharashtra, India
A senior data engineer role building and maintaining scalable data pipelines, ETL, and lakehouse (Bronze/Silver/Gold) solutions on Azure Synapse and Databricks using SQL, Python, and PySpark. The person also designs data models, monitors platform performance, maintains documentation, and drives data governance and quality in collaboration with stakeholders and vendor engineers.
Design, develop, and maintain scalable data pipelines, ETL processes, and data integration solutions across multiple sources.
Translate business requirements into effective data and analytical solutions in collaboration with analysts, data scientists, stakeholders, and vendors.
Design and optimize data models, including star schemas, aggregation tables, and curated datasets for reporting and analytics.
Build and manage data infrastructure using Azure Synapse Analytics, Databricks, and other Azure services.
Develop and automate data ingestion and transformation using SQL, Python, and PySpark.
Ensure data quality, consistency, security, and reliability across Bronze, Silver, and Gold lakehouse layers.
Monitor and optimize data platform performance, scalability, and reliability.
Contribute to data architecture, engineering standards, and best practices as a subject matter expert.
Maintain documentation for data pipelines, architecture, data sources, data dictionaries, and the Business Glossary.
Collaborate with vendor data engineers to deliver data infrastructure and integration solutions aligned with business objectives.
Improve data maturity, governance, and visibility to enable trusted insights and informed business decisions.
Bachelor’s degree in Computer Science, Data Science, Statistics, Mathematics, or a related field.
Minimum 5 years’ experience in data engineering or a related role.
Strong experience with Azure Synapse, Databricks, Delta Lake, and Azure cloud technologies.
Proficient in Python, PySpark, SQL/T-SQL, and ETL/data pipeline development.
Experience with data modelling, data management, ETL, BI, dashboards, and data visualization.
Strong understanding of data governance, security, quality, and BI best practices.
Strong analytical and problem-solving skills with the ability to translate business needs into technical solutions.
Excellent communication and stakeholder management skills, with the ability to work across teams and cultures.
Able to manage multiple priorities in a fast-paced environment and travel internationally when required.
Fluent in written and spoken English.
InTalent Asia