Databricks Data Engineer

• We are seeking a highly skilled Databricks Data Engineer to join our Data & AI practice.
• The successful candidate will have deep expertise in building scalable data pipelines, optimizing Lakehouse architectures and enabling advanced analytics and AI use cases on the Databricks platform.
• This role is critical in building and optimizing modern data ecosystems that enable data-driven decision making, advanced analytics, and AI capabilities for our clients
• As a trusted practitioner, you will design and implement robust ETL/ELT workflows, integrate real-time and batch data sources, and enable secure, well-governed data products and pipelines.
• You will thrive in a collaborative, client-facing environment, with a passion for solving complex data challenges, driving innovation and ensuring the seamless delivery of data solutions

Primary Responsibilities;

• Client Engagement & Delivery
• Data Pipeline Development (Batch and Streaming)
• Databricks & Lakehouse Architectures
• Data Modelling & Optimisation (Delta Lake, Medallion architecture)
• Collaboration & Best Practices
• Quality, Governance & Security

Business Relationships;

• Solution Architects
• Data Engineers, Developers, ML Engineers, and Analysts
• Client stakeholders up to Head of Data Engineering, Chief Data Architect, and Analytics leadership

Measures of Success;

• Delivery of high-performing, scalable, and secure data pipelines aligned to client requirements
• High client satisfaction and successful adoption of Databricks-based solutions
• Demonstrated ability to innovate and improve data engineering practices
• Contribution to the growth of the practice through reusable assets, accelerators, and technical leadership

Strong consulting values with ability to collaborate effectively in client-facing environmentsStrong problem-solving, analytical, and communication skillsHands-on expertise across the data lifecycle: ingestion, transformation, modelling, governance, and consumptionExperience leading or mentoring teams of engineers to deliver high-quality scalable data solutionsProven experience in data engineering and pipeline development on Databricks and cloud-native platformsUnderstanding of data governance, security, and compliance frameworksProficiency in ETL/ELT tools such as DBT, Matillion, Talend, or equivalentHands-on experience with cloud platforms (AWS, Azure, GCP)Strong SQL and Python (or equivalent language) skills for data manipulation and automationFamiliarity with Databricks Workflows and other orchestration toolsDeep expertise with the Databricks platform (Spark/PySpark/Scala, Delta Lake, Unity Catalog, MLflow)Experience with version control tools (GitHub, Bitbucket) and CI/CD pipelinesExposure to AI/ML workloads desirableFamiliarity with medallion architectures, data lakehouse principles and distributed data processingKnowledge of data modelling methodologies (star schemas, Data Vault, Kimball, Inmon)Experience: Minimum 5–8 years in data engineering, data warehousing, or data architecture roles, with at least 3+ years working with DatabricksPreferred: BSc/MSc in Computer Science, Data Engineering, or related fieldDatabricks certifications (Data Engineer Professional) highly desirableEducation: University degree required