Member of Technical Staff - Synthetic Data & Data Scaling
prometheus · San Francisco ·
- Work mode
- Hybrid
- Employment
- Full time
- Category
- Software engineering
Owns what Prometheus's AI models learn from: designing and running synthetic data generation, filtering, and weighting pipelines for pretraining and midtraining, and running scaling and ablation experiments to measure whether data interventions actually improve the model and the next training recipe.
About the Role
This role owns the question of what our models learn from: how we generate, filter, weight, and scale training data across pretraining and midtraining. You will design and run synthetic data pipelines at scale, build rigorous methods to measure whether a data intervention actually improves the model, and run the scaling and ablation experiments that decide what goes into the next training run.
The work is end to end: from a hypothesis about data, to a generation or curation pipeline, to a controlled training experiment, to a verdict that changes the recipe.
What You'll Do
Design and run synthetic data generation pipelines at scale, spanning pretraining and midtraining data mixes
Build filtering, weighting, and curation methods that shape what data the model actually sees
Develop rigorous evaluation methods to determine whether a given data intervention measurably improves the model, not just correlates with improvement
Design and execute scaling law and ablation experiments that inform decisions on the next training run's data recipe
Own the full loop: hypothesis, pipeline, controlled experiment, verdict, recipe change
Partner closely with pretraining, evals, and infra teams to translate data decisions into training outcomes
What We're Looking For
Strong track record in large-scale data work for LLM training: synthetic data generation, data curation, filtering, or mixing at pretraining or midtraining scale
Experience designing and interpreting scaling law or ablation experiments, with the statistical rigor to separate signal from noise
Comfort owning a problem end to end, from experimental design through to a recommendation that changes the training recipe
Strong software engineering fundamentals for building and operating data pipelines at scale
Prior experience at a frontier lab or similar large-scale training environment preferred
Why This Role Matters
Data is one of the highest-leverage levers on model quality, and this role sits at the center of deciding what that lever does. The decisions made here directly shape what the next generation of models learns from.