Lead Databricks Data Engineer
Namasys Analytics · India ·
- Seniority
- Lead
- Employment
- Full time
- Category
- Data engineering
- Experience
- 6+ years
Namasys Analytics · India ·
gsbsolutions1 · Jalisco, Mexico
ncs3 · Singapore, sg
spgi · Hyderabad, Telangana
Guidehouse · US - Remote (Any location)
Lead Databricks Data Engineer to own data engineering workstream of a large-scale predictive analytics programme, architecting and leading production-grade data pipelines on Databricks, integrating SAP and Oracle systems into a governed lakehouse, and leading a small engineering team.
Lead Databricks Data Engineer
Type: Full-time
Location- Remote
Experience- 6+ years
About the role
We are hiring a Lead Databricks Data Engineer to own the data engineering workstream of a large-scale predictive analytics programme. You will architect and lead the build of production-grade data pipelines on Databricks, integrating complex enterprise source systems (SAP and Oracle) into a governed lakehouse that feeds ML models, scoring engines, and executive dashboards. You will lead a small engineering team and set the standards the whole programme builds on.
What you'll do
1-Architect end-to-end ingestion and transformation pipelines on Databricks (Delta Lake, Workflows, Unity Catalog) from SAP and Oracle source systems
2-Design the integration approach for SAP data extraction (PM, MM, S/4HANA) in read-only patterns, working within enterprise security and data governance constraints
3-Implement data quality validation, anomaly detection, and lineage tracking as first-class pipeline components with contractual quality thresholds
4-Lead and review the work of data engineers; own pipeline reliability through SIT, UAT, go-live, and hypercare
5-Establish CI/CD-based deployment practices for data assets in coordination with the client platform team
Contribute to solution architecture, estimation, and technical documentation
What you'll need
1-6+ years in data engineering, with production Databricks delivery (Delta Lake, Spark, Workflows)
2-Demonstrated experience integrating SAP systems (ideally PM/MM or S/4HANA) into cloud data platforms
3-Strong pipeline orchestration, data modelling, and data quality engineering skills
4-Experience leading small engineering teams in fixed-deadline delivery
5-Clear technical communication with architects, security teams, and business stakeholders
Nice to have
1-Databricks Certified Data Engineer Associate or Professional (certification support provided)
2-Experience in asset-intensive or regulated industries
3-Familiarity with Oracle P6 / EAM or finance data sources
4-Prior delivery experience in the GCC region
Interested candidates share their resume at [email protected]
JobCubby · London, United Kingdom