Senior Databricks Data Engineer - Lakehouse & ML Pipelines
INGEPSY · Santander, Colombia ·
- Seniority
- Senior
- Category
- Data engineering
Senior Data Engineer designing, building, and operating batch and streaming data pipelines on Databricks, using PySpark and Delta Lake to migrate legacy workloads into a governed Lakehouse with a medallion architecture for analytics and AI. Core stack: Databricks, PySpark, Delta Lake, SQL, Python, Unity Catalog.
AgileEngine is seeking a Senior Data Engineer to design, build, and operate batch and streaming data pipelines on Databricks, leveraging PySpark and Delta Lake to migrate legacy workloads to a governed Lakehouse. You will model a medallion architecture powering analytics, dashboards, and AI use cases.
The role requires strong SQL, Python, and experience with Unity Catalog governance, plus collaboration with DevOps and analytics teams to ensure performance, reliability, and data quality.