Databricks Data Engineer: Lakehouse & AI Analytics
INGEPSY · Atlantico, Colombia ·
- Category
- Data engineering
A data engineer designs and runs batch and streaming pipelines on Databricks using PySpark and Delta Lake, migrating legacy data warehouse/ETL workloads into a governed medallion-architecture Lakehouse that powers analytics and AI use cases. Core tech includes Databricks, Spark, SQL, Python, Unity Catalog, and DevOps collaboration.
AgileEngine is seeking a Data Engineer to build batch and streaming pipelines on Databricks using PySpark and Delta Lake. You will migrate legacy data warehouse and ETL workloads onto a governed Lakehouse, modeling a medallion architecture that powers analytics and AI use cases.
Strong SQL, Python, and experience with Unity Catalog governance round out the role. The role involves designing and operating pipelines, optimizing Spark and Delta tables for performance, and collaborating with DevOps