Senior Databricks Data Engineer — Flexible Work & Growth
INGEPSY · Atlantico, Colombia ·
- Seniority
- Senior
- Category
- Data engineering
A senior data engineer builds and operates batch and streaming data pipelines on Databricks using PySpark and Delta Lake, migrating legacy ETL to a governed Lakehouse with a medallion architecture for analytics and AI, while optimizing Spark jobs and working with DevOps and analytics teams.
AgileEngine is seeking a Senior Data Engineer to build and run batch and streaming data pipelines on Databricks using PySpark and Delta Lake. You will migrate legacy ETL workloads to a governed Lakehouse and model a medallion architecture powering analytics and AI use cases.
Strong SQL, Python, and Unity Catalog governance are essential as you optimize Spark jobs, ensure data quality, and collaborate with DevOps, platform, and analytics teams to meet security and compliance standards.