Senior Data Engineer

Job Summary\:

We are looking for a hands-on, self-driven Data Engineer with 5+ years of experience to join our team. In this role, you will be the driving force behind our cloud data infrastructure, responsible for onboarding multiple data sources and building robust, scalable data pipelines using the Medallion Architecture on Microsoft Azure. You will collaborate closely with data scientists, analysts, and cross-functional squads to power downstream reporting, analytics, and machine learning initiatives.

Key Responsibilities\:

Data Ingestion & Pipeline Development\: Design, build, and maintain scalable batch/ real-time data pipelines to onboard, clean, transform, and integrate complex data sources into the Azure Cloud. Implement and manage the Medallion Architecture (Bronze, Silver, Gold layers) using Azure Databricks and Delta tables.

Data Modeling & Aggregation\: Combine multiple data sources to create optimized aggregate tables, views, and data models for downstream reporting and analytics applications. Automate data workflows and pipelines on weekly/monthly/ad-hoc schedules to ensure seamless data delivery. Collaboration & Optimization\: Partner closely with data engineers and data scientists across squads to feature-engineer tables and views that support advanced analytics use cases. Continuously monitor, troubleshoot, and optimize data pipelines for performance, scalability, and cost-efficiency.

Best Practices & Governance\: Act as an advocate for engineering best practices, including code modularity, documentation, error handling, and data quality checks.