Reflection is hiring a Data Quality Engineer for LLM pre-training to define measurable data-quality standards, build automated QA pipelines and human-in-the-loop checks, and monitor quality trends using Python and ML/LLM workflows. On-site roles in San Francisco, New York, or London.
You will define and operationalize measurable data-quality standards for LLM pre-training. You will analyze data, build and validate automated QA methods alongside human-in-the-loop processes, create reusable quality pipelines, provide vendor feedback, and monitor quality trends to improve acceptance criteria.
Responsibilities
Own upstream data quality for LLM pre-training
Translate requirements into measurable quality signals
Provide actionable feedback to external data vendors
Design, validate, and scale automated QA methods
Build reusable QA pipelines for model training
Monitor and report data quality over time
Improve quality standards, processes, and acceptance criteria
Requirements
Experience building data pipelines, QA systems, or evaluation workflows for pre-training data
Understanding of data quality impacts on pre-training
Experience designing automated quality checks
Proficiency in Python
Experience building ML or LLM workflows
Experience with large datasets and automated evaluation or quality-checking systems
Familiarity with LLM training and evaluation
Communication skills
Benefits
Stock options
Medical, dental, vision, and life insurance
Annual wellness allowance
Daily office lunch and dinner
22 weeks of paid parental leave
Unlimited paid time off in the U.S.
30 vacation days in the U.K.
Visa sponsorship support
Regular off-sites, happy hours, and team celebrations