Sr. Data Engineer
iLink Digital · Pune City, India ·
- Seniority
- Senior
- Category
- Data engineering
- Experience
- 7+ years
iLink Digital · Pune City, India ·
Sandisk · Bengaluru, KA, in
Encora · Kuala Lumpur
pwc · Cairo - ETIC
Designs and maintains scalable data pipelines and ETL processes using PySpark, SQL, and Python, with a focus on Azure Databricks for large-scale data processing and ensuring data quality and performance.
Key Responsibilities:
Design, build, and maintain scalable data pipelines and ETL processes.
Work with large datasets to ensure high data quality, accuracy, and performance.
Develop data solutions leveraging Azure Databricks, PySpark, and SQL.
Collaborate with cross-functional teams including Data Scientists, Analysts, and Product teams to support data-driven initiatives.
Troubleshoot and optimize data workflows to improve reliability and efficiency.
Ensure compliance with best practices in data governance, security, and performance.
Contribute to agile ceremonies and work in a collaborative, fast-paced environment.
Required Skills & Qualifications:
7–8 years of experience in Data Engineering.
Strong programming skills in Python and SQL.
Hands-on experience with PySpark for large-scale data processing.
Good working knowledge of Azure Databricks.
Strong problem-solving and debugging skills.
Experience in designing and optimizing ETL pipelines.
Knowledge of Agile methodologies (Scrum/Kanban) is a plus.
Nice to Have:
Familiarity with other Azure services such as Data Lake, Synapse, or Data Factory.
Exposure to CI/CD practices and version control systems (e.g., Git).
Experience working in a cloud-first data ecosystem.