Software Engineer Evals

You will own and improve a research platform that supports evaluation work. You will design, build, and ship reliable systems, create evaluation instruments, support dataset cleaning and experiment feedback, track training progress, diagnose agent failures, and help design datasets and grading schemes.

Responsibilities

• Own and improve the research platform
• Create and execute instruments for measuring model capabilities and behaviors
• Clean datasets and provide feedback for experiments
• Track growth throughout training
• Diagnose agent failures and identify improvements
• Manage execution infrastructure
• Design datasets and grading schemes
• Design, build, and ship reliable systems

Requirements

• Software engineering proficiency
• 3+ years actively writing code
• Track record of shipping high-quality projects
• Product judgment for translating quality, helpfulness, truthfulness, and safety into measurable criteria
• Attention to detail

Benefits

• Equity
• Medical coverage
• Vision coverage
• Dental coverage
• 401(k) retirement plan
• Short-term disability insurance
• Long-term disability insurance
• Life insurance
• Discounts and perks