Databricks Data Engineer - PySpark, Delta Lake, Remote
INGEPSY · Valle del Cauca, Colombia ·
- Category
- Data engineering
Data Engineer building batch and streaming pipelines on Databricks with PySpark and Delta Lake, migrating legacy workloads to a governed Lakehouse. Day to day: modeling bronze/silver/gold medallion architecture for analytics and ML, optimizing Spark jobs, and implementing data quality and governance controls in an Agile, remote/hybrid-friendly team.
AgileEngine is seeking a Data Engineer to build batch and streaming pipelines on Databricks using PySpark and Delta Lake. You will migrate legacy workloads to a governed Lakehouse and implement robust data quality and governance controls.
You will model a bronze/silver/gold medallion architecture for analytics and ML use cases, optimize Spark jobs, and collaborate with cross-functional teams in an Agile environment. Flextime with remote/hybrid options is available.