Senior Product Manager, Experimentation Tooling

What We're Looking For

We’re looking for a Senior Product Manager to own Lightning AI’s experimentation and post-training product end to end—from product strategy and roadmap through launch, adoption, pricing, and go-to-market.

This is a role focused on how AI researchers and engineers turn an idea into a high-quality, validated model. You’ll define the workflow for running and comparing experiments, managing fine-tuning and reinforcement-learning workloads, evaluating model quality, understanding failures, and selecting the right model or checkpoint for production.

You’ll work at the intersection of developer tooling, AI infrastructure, and model quality. The right candidate understands how modern AI teams work today: notebooks, training jobs, experiment trackers, checkpoints, evaluation suites, spreadsheets, and custom internal tooling. You can identify where those workflows break down and turn them into a cohesive, minimal product experience.

You should be able to move fluidly between designing an intuitive developer workflow, discussing distributed training and artifact lineage with engineers, evaluating model outputs, and explaining the product’s value to customers and sales teams.

This role requires unusually high ownership. You will work directly with engineering, customers, and the executive team; develop strong product opinions; create your own product artifacts; and drive work forward without waiting for another function to define the next step.

You will join the Product Team, report to our VP of Product, and work directly with our executive team as we grow this business.

This is a hybrid role based in our New York City or San Francisco office, with an in-office expectation of two days per week.

What You’ll Do

• Own the product vision and roadmap for post-training and experimentation — what we build, what we integrate with, what we don't build, and in what order
• Understand how ML engineers and AI researchers actually work today: the jobs they run, the comparisons they make, the failures they debug, and the handoffs that break down between research and production — then build the product that makes that workflow coherent
• Develop a strong point of view on where Lightning should build differentiated experiences versus integrate with the existing ecosystem of experiment trackers, evaluation frameworks, data tools, and model registries
• Work directly with engineers from problem definition through architecture, implementation, and launch — understand the constraints, help shape the solutions, don't hand off requirements and wait
• Use the product yourself; inspect failed workflows, read logs, identify friction, and remove it without waiting for someone to surface it
• Own model evaluation as a product function — write evals, assess outputs, and let quality signals drive roadmap decisions
• Design pricing and packaging in partnership with Growth and Finance — model unit economics, run experiments, and make calls that affect both adoption and margin
• Build workflows that help teams collaborate: share results, compare models, move work from research into production, and maintain enough lineage that decisions can be explained and reproduced
• Be the product voice in GTM — sales positioning, technical objection handling, and developer-facing content that builds credibility with ML engineers and platform teams
• Define and instrument the metrics that matter across activation, iteration speed, compute consumption, retention, and expansion

What You’ll Need

• 7+ years of product management experience, including at least 3 years building infrastructure, platform, developer-tooling, or machine-learning products.
• Hands-on experience building products for ML engineers, AI researchers, or data scientists.
• A detailed understanding of experimentation and post-training workflows, including training jobs, checkpoints, metrics, artifacts, experiment comparison, reproducibility, and model evaluation.
• Experience with one or more modern post-training techniques, such as supervised fine-tuning, preference optimization, reinforcement learning, distributed training, or hyperparameter optimization.
• Experience designing or working closely with model evaluations. You understand that model quality is multidimensional and know how qualitative and quantitative signals should inform product decisions.
• Strong product and interaction judgment. You can simplify technically complex workflows, develop a clear information architecture, and create usable product experiences without depending heavily on a dedicated design function.
• Enough technical depth to work directly with infrastructure and ML engineers on APIs, SDKs, execution systems, distributed workloads, observability, data and artifact management, and failure handling.
• A record of end-to-end ownership. You do not stop at roadmap definition; you investigate problems personally, create the necessary artifacts, drive decisions, support implementation, validate the result, and help bring the product to market.
• Strong prioritization and willingness to say no. You can identify the narrowest valuable product wedge and resist building a broad collection of loosely connected features.
• Experience owning pricing, packaging, or consumption-based products, including working with unit economics and making decisions that affect adoption and margin.
• Strong written and verbal communication. You are equally comfortable writing a technical product specification, reviewing a developer workflow, presenting to executives, and explaining the product to a customer.
• High bias for action and comfort operating in ambiguous, fast-moving environments with limited process and support.
• BS in Computer Science, Engineering, or equivalent practical experience.

Bonus Points

• Experience at an AI infrastructure company, neocloud, hyperscaler ML platform, experiment-tracking company, or developer-tooling startup.
• Familiarity with PyTorch, PyTorch Lightning, distributed training, GPU infrastructure, or large-scale fine-tuning.
• Experience with tools such as Weights & Biases, MLflow, Hugging Face, Ray, Slurm, Kubernetes, or comparable internal ML platforms.
• Experience building collaborative workflows for teams moving models from research through evaluation and production.

Compensation