Senior Data Engineer
R Systems · Maharashtra, India ·
- Seniority
- Senior
- Category
- Data engineering
- Experience
- 5+ years
R Systems · Maharashtra, India ·
thgrp · Indianapolis, IN
Computomic · Bengaluru, India
Software International Corporation (M) Sdn Bhd · Kuala Lumpur, Malaysia
Steampunk · McLean, VA, US
Senior Data Engineer designing, building, and optimizing scalable data pipelines on the Databricks Lakehouse Platform. Day to day: batch/streaming ETL with PySpark, Delta Lake, and Delta Live Tables, cluster administration and Unity Catalog governance, CI/CD deployment pipelines, performance tuning, and mentoring while partnering with data science and analytics teams.
We are looking for a Senior Data Engineer to join our team. In this role, you will design, build, and optimize scalable data pipelines on the Databricks Lakehouse Platform. You will partner with data science, analytics, and business teams to deliver reliable, high-performance data solutions in a collaborative environment.
Your responsibilities will include building and maintaining batch or streaming ETL pipelines using PySpark, Delta Lake, and Delta Live Tables. You will manage Databricks clusters and implement Unity Catalog for governance. You will also design dimensional models, optimize Delta tables, and troubleshoot performance issues. A key part of the role involves building deployment pipelines using CI/CD tools and supporting ML productionization while mentoring fellow team members.
Required Skills:
Preferred Skills:
Profitroom · Poland