We are looking for a Senior LLM Engineer / LLM Architect to join a project focused on building a production-grade LLM-powered document extraction pipeline.
The system analyzes complex documents and transforms their content into predefined structured data. The key challenge is ensuring that every extraction is accurate and traceable to the source, schema-valid, and free of duplicates.
This is a hands-on architecture role for someone who has already built and shipped complex LLM systems and knows how to make their quality measurable, testable, and reliable in production.
What you’ll work on
Design and evolve a multi-stage LLM extraction pipeline: extract → validate → deduplicate/link → verify → iterate.
Build structured document extraction with exact source references and evidence.
Separate LLM-driven extraction from deterministic code-based validation.
Design schema validation, record linkage, identity-based deduplication, and rule-conformance mechanisms.
Build and maintain LLM evaluation frameworks using golden datasets, measurable quality metrics, regression testing, and automated quality gates.
Design idempotent and resumable agent workflows with caching and token/cost optimization.
Work with hybrid exact + semantic search and incremental indexing.
Define executable acceptance criteria for LLM system quality.
What we’re looking for
3+ LLM-powered systems shipped to production, with measurable quality/evaluation frameworks you personally built or owned.
Hands-on experience with LLM evaluation: golden sets, precision/recall or similar metrics, regression testing, CI quality gates.
Strong experience with structured data/document extraction using LLMs.
Proven ability to combine model-driven extraction with deterministic validation in code.
Strong knowledge of RAG, information retrieval, hybrid search, embeddings/vector search.
Experience with deduplication, record linkage/entity resolution, and data modeling.
Experience designing multi-step / agentic LLM pipelines, including validation and verification loops.
Strong software engineering and system architecture background.
English — Upper-Intermediate+.
Nice to have
Experience with insurance, finance, legal, or other regulated-domain documents.
Knowledge graphs and entity resolution.
Event-sourced or lakehouse-style versioned data stores.
Adversarial verification patterns for LLM outputs.
Experience building custom LLM evaluation tooling.
Important
This role is not focused on prompt engineering or standard RAG/chatbot development. We are looking for someone with deep production experience in building LLM systems where quality is validated through measurable evaluation and executable checks, not prompts alone.
Format: Full-time, remote.