Reflection AI is hiring a Data Quality Engineer for its post-training team. Day to day, this person defines measurable data-quality standards for LLM training and evaluation, builds automated QA pipelines in Python over large datasets, gives feedback to external data vendors, and monitors quality trends to improve acceptance criteria.
You will define and operationalize measurable data-quality standards for LLM post-training and evaluation. You will analyze datasets, build and validate automated QA methods, create reusable quality pipelines, provide feedback to vendors, and monitor quality trends to improve acceptance criteria.
Responsibilities
Own upstream data quality for LLM post-training and evaluation
Translate requirements into measurable quality signals
Provide actionable feedback to external data vendors
Design, validate, and scale automated QA methods
Build reusable QA pipelines for model training and evaluation
Monitor and report data quality over time
Improve quality standards, processes, and acceptance criteria
Requirements
Experience building data pipelines, QA systems, or evaluation workflows for post-training data and agentic environments
Understanding of SFT, RL, training, and evaluation
Experience designing automated quality checks
Proficiency in Python
Experience building ML or LLM workflows
Experience with large datasets and automated evaluation or quality-checking systems
Familiarity with LLM training and evaluation
Communication skills
Benefits
Stock options
Medical, dental, vision, and life insurance
Annual wellness allowance
Daily office lunch and dinner
22 weeks of paid parental leave
Unlimited paid time off in the U.S.
30 vacation days in the U.K.
Visa sponsorship support
Regular off-sites, happy hours, and team celebrations