Senior Data Engineer (Databricks Migration)
Sigma Software · Lviv, Lviv Oblast, Ukraine ·
- Work mode
- Remote
- Seniority
- Senior
- Category
- Data engineering
Senior Data Engineer to migrate and modernize a data platform from BigQuery to Databricks, building PySpark pipelines and lakehouse architecture with Delta Lake and Unity Catalog.
- Analyze ETL ELT workloads and define migration approach
- Build PySpark data processing solutions
- Design lakehouse architecture with Databricks and Delta Lake
- Develop and optimize data pipelines for large volume datasets
- Implement data governance and security with Unity Catalog
- Implement incremental processing strategies
- Implement medallion architecture bronze/silver/gold
- Migrate data platform from BigQuery to Databricks
- Optimize SQL and Spark workloads
- Support production deployments and platform modernization