Principal AI Evaluations Platform Engineer

As Microsoft continues to push the boundaries of AI, we are looking for passionate individuals to work with us on some of the most interesting and challenging AI questions of our time. Our vision is bold and broad: to build systems with true artificial intelligence across agents, applications, services, and infrastructure. It is also inclusive: we aim to make AI accessible to consumers, businesses, and developers so that everyone can realize its benefits.

Microsoft AI (MS AI) is seeking an experienced engineer to help build and operate the evaluation platform that supports large-scale model training and development. We’re looking for someone who combines strong systems engineering fundamentals with operational excellence and who can build reliable evaluation infrastructure at scale. This role will work closely with researchers and training teams across our European offices to ensure evaluations are reliable, efficient, and available when teams need them.
We seek a versatile engineer who can build solutions that stand the test of time and who brings positive energy, empathy, and kindness to the team while remaining highly effective in a fast-paced environment.

Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees, we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day, we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.
This role is based in London, U.K., or Zurich, Switzerland. Candidates are expected to be local to the applicable office and work in the office four days per week.

Responsibilities

• Serve as a primary engineer for the evaluation platform during European hours, providing dedicated coverage for the evaluation stack supporting large-scale model training.
• Develop and extend core evaluation platform capabilities, including benchmark configurations, problem sets, graders, and evaluation runners.
• Improve evaluation throughput and scheduling efficiency.
• Build and maintain monitoring, alerting, and dashboards to proactively detect evaluation degradation.
• Partner closely with researchers and training teams across European offices to define evaluation requirements, coordinate launches, and unblock experiments.
• Contribute to technical design and code reviews across the evaluation platform stack, and maintain high-quality operational documentation and on-call runbooks.
• Embody our culture and values.

Required Qualifications

• Bachelor's Degree in Computer Science, Engineering, Mathematics, or a related field AND 4+ years of technical engineering experience with coding in languages including, but not limited to, Python, C++, Go, or Rust;
• OR Master's Degree in Computer Science, Engineering, Mathematics, or a related field AND 2+ years of technical engineering experience with coding in languages including, but not limited to, Python, C++, Go, or Rust;
• OR equivalent experience.
• Demonstrated experience operating production distributed systems, including participation in a formal on-call rotation.
• Strong proficiency in Python, including experience working in a large, shared, typed codebase.
• Experience debugging distributed system failures across the stack, including scheduling, resource exhaustion, networking, and process-level faults.
• Willingness and availability to participate in a scheduled on-call rotation covering European business hours.

Preferred Qualifications

• Experience with GPU-based workloads, distributed training, or large-scale inference infrastructure.
• Experience with distributed compute frameworks and schedulers, such as Ray, Kubernetes, Slurm, or comparable systems.
• Experience with observability and alerting tooling, such as Datadog, Prometheus, or Grafana.
• Familiarity with LLM evaluation, benchmarking, or reinforcement learning training infrastructure.
• Track record of reducing operational burden through systematic reliability improvements, automation, or tooling.
• Ability to work effectively in a fast-paced environment, manage multiple priorities, and adapt to changing requirements and deadlines.

The typical base pay range for this role across Switzerland is CHF 183,800.00 - CHF 309,700.00 per year. Certain roles may be eligible for benefits and other compensation.

Find additional benefits and pay information here: https://careers.microsoft.com/v2/global/en/corporate-pay/switzerland-corporate-pay

Find Jobs in United Kingdom on Arbeitnow