Senior Machine Learning Engineer Applied AI Quality
Block, Inc. · Remote Bay Area, CA, US ·
- Work mode
- Remote
- Seniority
- Senior
- Category
- ML ai
- Experience
- 5+ years
You will lead the technical direction for AI-driven quality and evaluation systems. You will develop scalable systems using LLMs, agents, and behavioral signals to evaluate quality, detect regressions, and generate product insights. You will define evaluation and measurement approaches, translate organizational needs into technical roadmaps, influence engineering standards, lead cross-functional initiatives, and mentor engineers.
Responsibilities
- Lead the technical strategy and architecture for AI-driven quality and evaluation systems.
- Develop scalable systems that use LLMs, agents, and behavioral signals to evaluate quality, detect regressions, and generate product insights.
- Define long-term approaches for evaluation, measurement, and quality intelligence.
- Translate organizational needs into technical direction, roadmap priorities, and platform capabilities.
- Influence engineering standards and best practices for reliable, measurable, and trustworthy AI systems.
- Lead cross-functional initiatives spanning product, infrastructure, data, and applied AI.
- Mentor engineers through technical leadership, design reviews, and systems thinking.
- Identify opportunities for AI systems to improve product understanding, debugging, and behavior.
Requirements
- 5+ years of experience in software engineering, machine learning engineering, or applied AI.
- Experience designing and shipping large-scale AI and machine learning systems in production.
- Expertise with LLMs, agents, evaluation systems, retrieval architectures, and AI infrastructure.
- Ability to lead ambiguous, high-impact technical initiatives from concept through adoption across multiple teams.
- Experience defining technical strategy and influencing roadmaps.
- Communication and cross-functional leadership skills.
- Experience creating platforms, frameworks, and systems that enable other engineers and teams.
Benefits
- Remote work
- Medical insurance
- Flexible time off
- Retirement savings plans
- Family planning