UX Data Scientist
sitetracker · New Delhi ·
- Work mode
- Hybrid
- Category
- Data science
The Opportunity
The Skills You'll Have
- Thrive in fast-moving environments and deliver high-impact outcomes at pace
- Independently identify ambiguous challenges and take ownership of driving them to resolution
- Champion new ideas, influence stakeholders, and rally support around strategic initiatives
- Identify workflows where AI tools and emerging technologies accelerate delivery, and make
building the default way you work
Measurement, Experimentation, and Evaluation
- Define what "good" means for an agentic experience and turn it into metrics, evals, and datasets that hold up under scrutiny
- Design and analyze experiments — A/B tests, holdouts, shadow runs — with correct sizing, clean reads, and honest conclusions
- Build offline and online evaluation systems for LLM behavior across correctness, grounding, safety, tone, and latency
- Apply statistical and ML fluency to real, messy production behavior — distinguishing signal from noise and root cause from symptom
Agentic UX and Behavior Design
- Turn complex enterprise workflows into agent experiences people can actually use without training
- Design context architecture, decision paths, and communication patterns, and reason about how they shape behavior across the full input distribution
- Work directly with customers — running pilots, mining usage, and iterating based on how they actually behave, including customers with complex needs or in delicate situations
- Make tradeoff decisions across user needs, agent behavior, technical constraints, and business goals, and connect those decisions to commercial outcomes
Competitive & Market Awareness
- Maintain running awareness of the competitive landscape across multiple verticals
- Identify gaps and opportunities and produce analysis that connects to product and behavior decisions
- Track market trends, adjacent products, and emerging tools, connecting what's happening in the market to what Sitetracker should be doing
- Regularly contribute to competition and industry Slack channels with substantive observations
Within 90 Days, You'll:
-
Collaborate regularly with product management, design, and engineering peers, and have met key stakeholders across the departments you work with
-
Be on the phone with customers, learning their workflows, pain points, and how they use Sitetracker and Scout. Know the personas and subverticals
-
Deliver work to your scrum team — pushing code, building instrumentation and evals, and demonstrating the builder standard
-
Be able to confidently demo Sitetracker's core product in your area and give a solid cross-product demo
-
Stand up baseline evals and quality tracking for your area in Watch, and establish what you don't yet measure
-
Begin mentoring others and providing substantive feedback in reviews
Within 180 Days, You'll:
-
Lead the successful delivery of high-impact agent experiences across complex enterprise workflows
-
Own day-to-day decisions for your experience area independently, establishing leading indicators and making tradeoff decisions confidently
-
Architect Scout graphs and design sophisticated AI interactions, owning a comprehensive eval suite and tracking quality over time
-
Design and run experiments on real agent behavior, and run customer betas and previews independently — including with customers who have more complex needs or stronger opinions
-
Actively mentor others and raise the bar through consistent, high-quality reviews of evals, experiments, and agent designs
-
Contribute meaningfully to shared evaluation infrastructure and establish higher standards for quality measurement in your domain
Within 365 Days, You'll:
-
Point to product successes you have driven that have directly contributed to commercial outcomes
-
Be recognized as an expert in your domain — people across the org come to you when they need to understand your experience area or how its quality is measured
-
Ship code confidently to production, build the instrumentation and evals that keep quality honest, and operate as a true peer with engineering
-
Be trusted with the most complex customer situations and maintain relationships with key customers that you built and maintain on your own
-
Shape direction across multiple projects with well-reasoned, evidence-backed positions that influence the roadmap
-
Set the pace for delivery in your domain and make others faster by reducing ambiguity early