Must have Skills : Databricks, ML Engineering & MLOps ” Critical, Azure + Kubernetes ” Critical, GenAI / LLM Engineering
Job Purpose
To design, prototype, and build next-generation analytic engines and services by applying strong expertise in Artificial Intelligence (AI)
Purpose
Build, deploy, scale and support machine learning, AI and GenAI solutions in production. The role focuses on operationalising models, developing AI applications and agents, and creating the platforms and services required to deliver business value at scale.
Key Responsibilities
- Productionise, deploy and monitor machine learning models and data science pipelines on Databricks.
- Build, deploy and support AI Agents, GenAI applications and RAG solutions on Databricks.
- Develop and maintain reusable ML pipelines using MLOps principles, including CI/CD, automated testing, monitoring and governance.
- Deploy, optimise and manage open source AI and machine learning models on Azure Kubernetes Service (AKS).
- Design, develop and support custom APIs and microservices on AKS to expose AI and machine learning capabilities to business applications.
- Implement containerised solutions using Docker and Kubernetes to ensure scalable, secure and resilient deployments.
- Monitor model performance, drift, reliability and operational health in production environments.
- Partner with Data Scientists to productionise prototypes and enable business-ready solutions.
- Collaborate with platform, security, cloud and infrastructure teams to ensure compliance with enterprise standards.
- Troubleshoot and resolve production issues related to models, pipelines, APIs and AI applications.
- Optimise AI and ML solutions for performance, scalability, cost and reliability.
- Contribute to engineering standards, reusable frameworks and best practices across the AI and ML ecosystem.
- Mentor junior engineers and promote knowledge sharing across the team.
- Stay current with advancements in AI, GenAI, MLOps, Databricks, Kubernetes and cloud technologies.
Core Deliverables
- Production-ready ML models and pipelines running on Databricks.
- AI Agents and business applications deployed on Databricks.
- Open source LLMs and AI services deployed on AKS.
- Secure and scalable APIs exposing AI capabilities to consuming systems.
- Automated deployment, monitoring and governance processes.
- Reliable, scalable and compliant AI platforms supporting business outcomes.
Key Skills
- Databricks Workflows, Model Serving, MLflow and Mosaic AI
- Azure Kubernetes Service (AKS)
- Python, SQL and REST APIs
- Docker and Kubernetes
- CI/CD and MLOps practices
- Machine Learning and Generative AI
- LLM deployment and optimisation
- Cloud engineering and infrastructure automation
- Monitoring, observability and troubleshooting
Computer Science, Engineering, Econometrics, Mathematical Statistics, Actuary Science. Masters or Doctorate will be an added advantage.
Preferred Certifications
- Microsoft Azure certifications (AZ-104, AZ-305, AI-102 or equivalent)
- Databricks certifications (Data Engineer, Machine Learning Engineer, Generative AI Engineer)
- Kubernetes and containerisation certifications (CKA, CKAD or equivalent)
- DevOps, MLOps or Platform Engineering certifications
- AWS or Google Cloud certifications will be advantageous
- Machine Learning, Artificial Intelligence or Data Science certifications from recognised providers such as Microsoft, Databricks, SAS, Coursera or DeepLearning.AI will be an added advantage
Technical / Professional Knowledge
- Strong understanding of MLOps, DevOps and software engineering practices for machine learning platforms.
- Experience building, deploying and supporting machine learning solutions in production environments.
- Proficiency in Python and experience with SQL and API development.
- Experience with Databricks, MLflow, Model Serving and cloud-native AI/ML platforms.
- Hands-on experience with Kubernetes, Docker and containerised application deployment.
- Experience deploying and supporting machine learning and Generative AI solutions on Azure Kubernetes Service (AKS).
- Knowledge of CI/CD pipelines, infrastructure automation and platform monitoring.
- Experience with distributed computing technologies such as Spark and large-scale data processing frameworks.
- Understanding of machine learning, large language models (LLMs), retrieval-augmented generation (RAG) and AI agents.
- Ability to productionise data science solutions and collaborate effectively with Data Scientists.
- Experience delivering end-to-end AI and machine learning use cases from development to production.
- Ability to translate technical concepts into business outcomes and communicate effectively with stakeholders.
- Strong written and verbal communication skills with the ability to work across cross-functional teams.
- Self-driven, adaptable and capable of thriving in a fast-paced, technology-driven environment.