Azure Data Engineer

Additional Details

Exp: 3 - 6 years

Shift time: 02.00 P.M. to 11.00 P.M

Location: Bangalore, Mumbai, Pune & Gurgaon

Notice Period: Immediate to 30 Days only

Shift timing : 2 PM to 11 PM - IST time zone

Job Description

Looking for a hands‑on Senior Data Engineer – Azure & Databricks with up to 6 years of total experience to build, optimize, and maintain scalable cloud data platforms.

This is an individual contributor role, focused on developing reliable data pipelines and analytics‑ready datasets using Microsoft Azure and Databricks. The role requires strong hands‑on expertise in SQL and Python, along with experience building batch (and basic streaming) data pipelines in a cloud environment.

You will collaborate closely with architects, analytics teams, QA, DevOps, and business stakeholders in a global delivery model.

Must have skills

Hands‑on experience delivering Azure‑based data engineering solutions

Cloud & Data Engineering (Azure)

Strong hands‑on experience with Azure data services, including:

Azure Data Lake Storage Gen2 (ADLS)

Azure Data Factory (ADF)

Azure Databricks

Azure Synapse Analytics

Experience designing cloud‑native data lakes and analytical data stores

Solid understanding of batch data pipelines and basic streaming concepts

SQL & Python (Mandatory)

Strong SQL skills (mandatory)

Writing complex queries, joins, aggregations, and transformations

Hands‑on experience working with large datasets in Synapse / Databricks

Strong Python skills (mandatory)

Python for ETL / ELT and data engineering use cases

Hands‑on experience with PySpark in Databricks

Strong understanding of data modeling, transformations, and query performance tuning

Data Processing & Engineering

Hands‑on experience with Spark / PySpark on Databricks

Experience handling structured and semi‑structured data

Understanding of partitioning, schema evolution, and data validation concepts

DevOps & Platform Basics

Working knowledge of Infrastructure as Code (ARM Templates and/or Terraform)

Basic experience with CI/CD pipelines (Azure DevOps or GitHub Actions)

Understanding of logging and monitoring using Azure Monitor / Log Analytics

Collaboration

Ability to work effectively with architects, QA, DevOps, and business stakeholders

Good communication skills for explaining technical concepts clearly

Good to have skills

Exposure to streaming technologies such as Azure Event Hubs or Kafka

Familiarity with Lakehouse architecture and Delta Lake concepts

Experience integrating Azure data platforms with BI tools (Power BI preferred)

Basic knowledge of data governance and data quality frameworks

Awareness of Azure cost‑optimization best practices

Experience working in Agile delivery models, especially with global clients

Key Responsibilities

Data Engineering & Development

Build scalable ETL / ELT pipelines using Azure and Databricks

Develop SQL‑based transformations and Python / PySpark pipelines

Ingest and process data using ADF, Databricks, and ADLS

Build analytics‑ready data models optimized for performance and cost

Platform & Operations

Support deployment and execution of data pipelines across environments

Monitor data pipeline health, performance, and data quality

Troubleshoot pipeline failures and perform root‑cause analysis

Follow Azure security, reliability, and scalability best practices

Collaboration & Delivery

Work closely with architects and product teams to understand requirements

Translate business and analytics needs into working Azure data solutions

Contribute to documentation, code reviews, and engineering standards