AI Engineer Model Quality and Performance
Cerebras Systems, Inc. · Sunnyvale, CA ·
- Work mode
- Onsite
- Category
- AI engineering
Owns model quality and performance evaluation for Cerebras's AI inference offerings: designs evaluation suites powered by AI agents, automates end-to-end benchmarking pipelines, builds customer-specific benchmarks from workload trajectories, and creates tooling that presents combined quality/performance data. Stack includes Claude-based agents, Docker, Git, and automation tooling.
You will own model quality and performance evaluation for inference offerings. You will design evaluation suites, use AI agents to generate and validate test cases, automate evaluation pipelines, build customer-specific benchmarks, and create tooling that combines quality and performance data.
Responsibilities
- Design model evaluation suites using AI agents
- Build customer-specific evaluations from workload trajectories
- Automate end-to-end evaluation execution using AI-driven pipelines
- Build automations to forecast and benchmark model performance
- Build tooling that presents quality and performance data
Requirements
- Experience building AI agents using Claude or an equivalent tool
- Strong mathematics and statistics background
- Comfort with Docker, Git, and automation tooling
- Experience designing tools for non-engineering users