Databricks Data Engineer: Real-Time Pipelines & Lakehouse
OpenTalent · Singapore, Singapore ·
- Category
- Data engineering
Design, build, and optimize high-throughput batch and real-time data pipelines on the Databricks Lakehouse platform in Singapore, working with Spark, PySpark, Spark SQL, Delta Lake, Unity Catalog, and CI/CD to deliver scalable, secure data infrastructure.
OpenTalent is seeking a skilled Databricks Data Engineer to design, build, and optimize high-throughput batch and real-time data pipelines on the Databricks Lakehouse platform in Singapore. You will implement Delta Lake, Unity Catalog, and advanced CI/CD practices to ensure scalable, secure data infrastructure.
Responsibilities include architecting robust data pipelines with Spark, PySpark, and Spark SQL, ingesting data from APIs, databases and streams, and implementing the Medallion