Senior Data Engineer (Databricks)

Job Title: Sr Databricks Engineer

Location: Remote (But need to work onsite at Bengaluru office for 1-2 week in November and March/April)

Hire Type: Fulltime

Summary:

We are looking for a seasoned Senior Data Engineer with strong Databricks expertise to contribute to the expansion of our enterprise Databricks data platform. The selected candidate will focus on integrating additional datasets, engineering scalable data pipelines, and building new business dimensions and measures within the existing Databricks ecosystem.

Key Responsibilities:

Data Engineering & Development

• Architect and build scalable data pipelines leveraging Databricks and PySpark
• Connect and incorporate additional data sources into the existing Databricks Lakehouse environment
• Create and refine dimensional models, business dimensions, measures, and curated datasets for analytical use
• Prepare, transform, and optimize data to support reporting, analytics, and downstream applications
• Contribute to data quality initiatives, validation activities, monitoring, and performance tuning
• Investigate and resolve platform-related issues throughout the project.

Platform & Data Management

• Apply Databricks development, deployment, security, and governance best practices across solutions
• Use Unity Catalog to address data governance and access control requirements
• Build, package, and deploy solutions through Databricks Asset Bundles
• Work closely with the project team to convert business and technical requirements into scalable data solutions.

DevOps & Automation

• Establish and manage CI/CD workflows for data engineering components
• Use GitHub and/or GitLab to manage source code, branching approaches, and peer code reviews
• Contribute to automated deployment workflows and release management activities.

Required Skills:

• 8+ years of professional experience in Data Engineering
• 4+ years of practical, hands-on experience with Databricks (required)
• Strong proficiency with:

-Databricks

-PySpark

-Python

-SQL

-Delta Lake

-Unity Catalog

-Databricks Asset Bundles

-Databricks Security & Access Management

• Proven experience designing and supporting enterprise-level ETL/ELT solutions
• Extensive experience developing dimensional models, business dimensions, measures, KPIs, and analytical datasets
• Hands-on experience developing and maintaining CI/CD pipelines
• Strong background in source code management using GitHub and/or GitLab
• Experience delivering solutions within cloud-based data environments
• Strong verbal and written communication abilities.

Preferred Skills:

• Exposure to Fivetran or comparable cloud-based data ingestion solutions
• Working knowledge of REST APIs and integrations with enterprise applications
• Experience with Power BI or comparable enterprise business intelligence and reporting platforms
• Background supporting AI, machine learning, or advanced analytics projects
• Experience working with HR, Finance, or other enterprise business data domains.