Senior Data Engineer [T500-29419]

About Guidant Global:

Guidant Global delivers dynamic, tailored workforce solutions that empower businesses to thrive in ever-evolving markets. Through our MSP, RPO, Direct Sourcing, Services Procurement and Consulting services, we help organisations find, engage, and manage the best permanent and contingent talent across the globe.

Part of Impellam Group, we champion A Better Way, a people-first approach that integrates deep industry expertise with advanced technology. Our technology turns workforce complexity into clarity, with real-time insights that support smarter talent decisions at scale.

With over 1.5 million STEM and professional experts in our network across 80+ countries, we don't just solve today's challenges. We're helping to shape the workforce of tomorrow.

About Guidant Global India:

At Guidant Global India, we play our part in shaping the future of work by powering recruitment and support services for Guidant teams across APAC and wider global markets. As a strategic delivery hub and global capability centre, we partner with stakeholders worldwide to raise quality, strengthen compliance, and deliver consistently at scale.

We're not standing still. We're building deeper capability in-house, taking ownership of the work that matters most, and designing an agile, future-ready delivery model built for where the business is heading next.

Join us, and you'll be part of a people-first team that's expert, ambitious, and collaborative, delivering A Better Way, every day.

Role Overview:

HFG is building its Hyderabad Global Capability Centre (GCC) as the long-term engineering backbone of its global data platform. This is a permanent, full-time role requiring 5+ years' total experience with 3+ years in production Databricks, working a minimum four-hour overlap with either CET (approx. 09:00–13:00 IST) or US Eastern/Central hours (approx. 18:00–22:00 IST) across a 40 hour week. We are looking for Senior Data Engineers who will be founding members of this team — engineers who own the full depth of the platform, from Lakeflow ingestion through medallion pipelines to AI-ready data products, and who will help set the engineering standards the GCC will run on for years to come. Working closely with the Global Head of Data & AI and senior engineers in the UK and Netherlands, the Senior Data Engineer helps set the technical standards and mentoring culture the India GCC engineering team will run on as it scales.

About HFG & the Data Platform:

Headfirst Global (HFG) is a global staffing and workforce solutions company operating across the EU, UK, North America, and APAC. The Global Data & AI team is building the Headless Data Architecture (HDA) — a hub-and-spoke federated lakehouse platform on Azure Databricks, enabling analytics, AI, and operational intelligence across the group's global business units. The platform runs on Azure Databricks with Unity Catalog, Lakeflow, Mosaic AI, and AgentBricks as core components, and integrates with SnapLogic for enterprise connectivity. All AI workloads route through the Unity AI Gateway. The India GCC team is being built as the long-term engineering backbone of this platform, working in close collaboration with senior engineers in the UK, Netherlands, and the US.

Key Responsibilities:

Design, build, and own end-to-end Databricks Lakehouse pipelines — from Lake flow ingestion through Bronze/Silver/Gold medallion layers to Gold-layer data products consumed by analytics and AI workloads.

• Implement and maintain Unity Catalog governance — catalog and schema design, RBAC, column-level security, lineage, and data contracts.
• Deploy and manage pipelines using Databricks Asset Bundles (DABs) and Azure DevOps/GitHub Actions CI/CD across dev, test, and production environments.
• Design reusable, idempotent, and testable pipelines adhering to HFG FITT standards (Functional, Idempotent, Tested, Two-stage).
• Lead technical design discussions and contribute architectural decisions to the India GCC platform standards.
• Mentor and support medior data engineers on the team, reviewing code and establishing engineering best practices.
• Integrate enterprise source systems through SnapLogic, ADF, and Databricks-native connectors including Salesforce, Dynamics 365, and SAP.
• Contribute to AI-enabled data engineering — building AI-ready Delta Lake datasets, vector search pipelines, and LLM-adjacent data products.
• Own production reliability — monitoring, alerting, Delta table maintenance (OPTIMIZE, VACUUM, liquid clustering), and incident response.
• Collaborate daily with UK and NL senior engineers in code reviews, architecture reviews, and sprint planning.

Qualifications & Experience:

Must-have:

• 3+ years of production Azure Databricks experience — PySpark, Delta Lake, Databricks Workflows.
• Proven medallion architecture design across Bronze, Silver, and Gold layers with explicit quality gates.
• Unity Catalog governance — RBAC, fine-grained access control, lineage.
• Databricks Asset Bundles (DABs) or equivalent CI/CD deployment of Databricks workloads.
• Strong Python and Spark SQL for complex transformation logic.
• Azure cloud familiarity — ADLS Gen2, Entra ID, Azure Key Vault.
• Data contracts and schema versioning practices in a multi-consumer Lakehouse environment.
• Databricks Certified Data Engineer Associate or Professional certification (or active pursuit).
• SnapLogic or comparable iPaaS tool (MuleSoft, Boomi) — building and maintaining enterprise integration flows.
• Azure Synapse Analytics — familiarity with Dedicated SQL Pool and Synapse pipelines for maintaining existing flows during the Databricks migration.
• Excellent written and spoken English — comfortable in daily international collaboration.

Nice to have:

• Delta Live Tables (DLT) or Lake flow Connect for streaming ingestion.
• Snap Logic or comparable iPaaS integration experience.
• AI-enabled DE work: RAG pipelines, Databricks Vector Search, MLflow, LangChain/LangGraph.
• dbt on Azure Databricks.
• Experience working in staffing, HR-tech, or workforce intelligence platforms.
• Databricks Certified Data Engineer Professional or GenAI Engineer Associate.

Working Model & Time zone:

This role is based in Hyderabad and works in a hybrid model. Daily collaboration with HFG teams requires a minimum four-hour overlap with either CET (approx. 09:00–13:00 IST) or US Eastern/Central hours (approx. 18:00–22:00 IST) across a 40 hour week. Candidates should be comfortable with structured asynchronous communication for the remaining hours. The India GCC team meets daily with the global Data & AI team for standups, code reviews, and delivery planning. Strong written and spoken English communication is essential.

Our Platform Stack:

Azure Databricks — Delta Lake, Unity Catalog, DLT, DABs, Lake flow, Mosaic AI

• SnapLogic — enterprise integration and data orchestration
• Azure (ADLS Gen2, Entra ID, Key Vault, Azure DevOps, GitHub Actions)
• Python, PySpark, Spark SQL
• Unity AI Gateway — governed LLM routing and observability
• MLflow, Databricks Vector Search, AgentBricks.

What We Offer:

• Be a founding member of the India GCC — shape how the team is built and the engineering standards it runs on
• Work on a globally scaled platform serving EU, UK, US, and APAC business units simultaneously
• Direct exposure to AI-enabled data engineering on one of the most modern Databricks deployments in the staffing industry
• Collaborate with senior engineers across the UK and Netherlands who will actively invest in your growth
• Hyderabad-based with a global engineering mandate — not an offshore support function.