Data Engineer

Job Summary

We are looking for a hands-on, self-driven Data Engineer with 4-8 years of experience to join our team. In this role, you will be the driving force behind our cloud data infrastructure, responsible for onboarding multiple data sources and building robust, scalable data pipelines using the Medallion Architecture on Microsoft Azure. You will collaborate closely with data scientists, analysts, and cross-functional squads to power downstream reporting, analytics, and machine learning initiatives.

Key Responsibilities

Data Ingestion & Pipeline Development: Design, build, and maintain scalable batch/ real-time data pipelines to onboard, clean, transform, and integrate complex data sources into the Azure Cloud. Implement and manage the Medallion Architecture (Bronze, Silver, Gold layers) using Azure Databricks and Delta tables.

Data Modeling & Aggregation: Combine multiple data sources to create optimized aggregate tables, views, and data models for downstream reporting and analytics applications. Automate data workflows and pipelines on weekly/monthly/ad-hoc schedules to ensure seamless data delivery.

Collaboration & Optimization: Partner closely with data engineers and data scientists across squads to feature-engineer tables and views that support advanced analytics use cases. Continuously monitor, troubleshoot, and optimize data pipelines for performance, scalability, and cost-efficiency.

Best Practices & Governance: Act as an advocate for engineering best practices, including code modularity, documentation, error handling, and data quality checks.

Required Qualifications & Skills

Experience: Minimum 4 years of hands-on professional experience in data warehousing, ETL/ ELT design, and building enterprise data pipelines.

Cloud & Big Data Stack: Strong expertise working with large volumes of data within Microsoft Azure. Deep hands-on experience with Azure Databricks, Delta Tables, and modern Azure components.

Programming & Querying: Advanced working knowledge of Python and PySpark for data transformation and distributed computing. Expert-level SQL skills with strong understanding of data warehousing concepts, database schemas, and data modeling.

Soft Skills & Independence: A self-starter with a proven track record of working independently to design, tune, and deliver high-performance tables and views.

Preferred / Good-to-Have Skills

Experience working with automotive sales and marketing data domains.

Practical understanding of Agile/Scrum methodologies and cross-functional team collaboration.

Familiarity with CI/CD tools or data orchestration frameworks