Senior AI Engineer

VAST Data is looking for a Senior AI Engineer to design and build production-grade, LLM-powered systems. You'll work at the intersection of software engineering and applied AI — shipping agents, RAG pipelines, and tool-using systems that solve real problems at scale. This is a hands-on, high-ownership role for someone who thrives at the frontier of what's possible with modern LLMs and isn't afraid to write the glue, the infrastructure, and the prompts that make it all work.

This is a **cross-functional, company-wide role**. You won't be embedded in a single product team — instead, you'll partner with every department to identify high-leverage opportunities and build AI-powered tools and workflows that boost productivity and efficiency across the entire organization.

This is a great opportunity to be part of one of the fastest-growing infrastructure companies in history, an organization that is in the center of the hurricane being created by the revolution in artificial intelligence.

"VAST's data management vision is the future of the market."- Forbes

VAST Data is the data platform company for the AI era. We are building the enterprise software infrastructure to capture, catalog, refine, enrich, and protect massive datasets and make them available for real-time data analysis and AI training and inference. Designed from the ground up to make AI simple to deploy and manage, VAST takes the cost and complexity out of deploying enterprise and AI infrastructure across data center, edge, and cloud.

Our success has been built through intense innovation, a customer-first mentality and a team of fearless VASTronauts who leverage their skills & experiences to make real market impact. This is an opportunity to be a key contributor at a pivotal time in our company’s growth and at a pivotal point in computing history.

What You'll Do:

- Design, build, and operate LLM-powered applications, agents, and workflows end-to-end — from prototype to production.

- Architect retrieval, context engineering, and tool-use strategies that make models reliable, accurate, and cost-efficient.

- Integrate LLMs with internal services, third-party APIs, and data stores to automate complex business and engineering workflows.

- Build, evaluate, and continuously improve evaluation harnesses for non-deterministic systems.

- Collaborate closely with product, research, and platform teams to translate ambiguous problems into shipped capabilities.

- Stay ahead of the rapidly evolving LLM ecosystem (models, frameworks, agentic patterns) and bring the best ideas into our stack.

Requirements

Engineering Foundations:

- Strong Python skills- you write clean, idiomatic, well-tested code and understand the language deeply.

- Hands-on experience using coding agents(Cursor, Claude Code, GitHub Copilot, or similar) to build complex software systems. You know how to delegate effectively to AI assistants and review their output critically.

- Experience with multiple database paradigms- both SQL (PostgreSQL, MySQL) and NoSQL (MongoDB, Redis, DynamoDB, or similar). You can choose the right tool for the job.

- Experience designing and integrating with third-party APIs- REST and gRPC. Comfortable building robust clients, handling auth, retries, rate limits, and schema evolution.

- Production experience with Docker and Kubernetes- containerizing services, writing manifests, and debugging deployments.

- Strong Linux fundamentals- confident in bash and the terminal; you can navigate, script, and troubleshoot a server without reaching for a GUI.

- Experience building cloud-native tools on AWS, GCP, or Azure (compute, storage, queues, serverless, IAM).

AI / LLM Expertise:

- Solid understanding of what an LLM is and how it works- tokenization, attention, context windows, sampling, and the practical implications of each for system design.

- Strong grasp of modern patterns for integrating LLMs into real workflows, including RAG, MCP (Model Context Protocol), vector databases, agents, tool use, and context engineering- with hands-on experience building with several of them.

- Production experience implementing LLM-powered systems end-to-end, using relevant tools and frameworks (e.g. LangChain, LlamaIndex, LangGraph, Haystack, Pydantic AI, vector stores like Pinecone/Weaviate/pgvector, observability tools like LangSmith or Langfuse).

- Solid foundation in core ML concepts; embeddings, evaluation, overfitting, generalization, and how classical ML relates to and differs from modern LLM-based approaches.

Nice to Have:

- Experience fine-tuning or distilling open-source models.

- Contributions to open-source AI/ML projects.

- Experience with streaming, real-time systems, or low-latency inference.

- Familiarity with prompt evaluation frameworks and LLM-as-judge methodologies.

- Background in distributed systems or high-scale backend engineering.

What We Look For:

- A builder's mindset; you ship, measure, iterate, and don't get stuck in analysis.

- Comfort with ambiguity; LLM systems are non-deterministic and the field moves weekly. You navigate that with curiosity rather than frustration.

- Strong communication; you can explain trade-offs to engineers, product managers, and executives alike.

- Pragmatism; you know when to reach for a 200-line script and when to invest in proper infrastructure.