Lead Data Engineer – AI/Machine Learning

We are looking for a Lead AI Engineer to help shape and drive AI/ML enablement and readiness across the organization. This role requires strong data engineering fundamentals: you will start hands-on, contributing directly to our data platform and pipelines, while progressively taking on a leading role in defining how the organization builds, deploys, and governs AI/ML capabilities.

Reporting directly to the VP, Head of Data, you will work autonomously to identify gaps, propose solutions, and bring innovative thinking to how our data and AI/ML ecosystem should evolve. You will partner closely with Data Governance, Data Engineering, and Product stakeholders to define our AI/ML frameworks and MLOps strategy, and to ensure the organization is well-positioned to adopt AI/ML responsibly and at scale.

Technical Knowledge and Understanding:

Data Engineering

• Strong data engineering fundamentals: deep expertise in data pipeline design, optimization, and distributed data processing (e.g., Spark, dbt, Airflow, Kafka, or equivalent).

• Platforms: hands-on experience with Snowflake, Databricks, and/or Azure Synapse Analytics, with the ability to architect and optimize workloads on one or more of these platforms.

• Strong knowledge of cloud platforms (AWS, Azure, or GCP) and modern data warehouse/lakehouse architectures.

AI/ML Engineering

Programming & software engineering fundamentals

• Strong Python (the de facto language for AI/ML tooling); solid software engineering practices (testing, version control, code review) since AI engineers ship production systems, not just notebooks

• API design and integration — most AI engineering work today is building systems around models (orchestration, tool-calling, retrieval), not training them from scratch

LLM & foundation model fluency

• Practical experience with LLM APIs (ie. OpenAI) and open-weight models

• Prompt engineering and prompt evaluation as a discipline, not just trial-and-error

• Understanding of context windows, tokenization, embeddings, and model limitations (hallucination, latency, cost tradeoffs)

RAG (Retrieval-Augmented Generation) & data retrieval

• Vector databases (Pinecone, Weaviate, pgvector, etc.) and embedding models

• Chunking strategies, hybrid search, reranking

Agentic systems & orchestration

• Frameworks like LangChain, LangGraph, LlamaIndex, or custom orchestration

• Tool-use / function-calling design, multi-step reasoning chains, agent memory and state management

Fine-tuning & model adaptation

• When to fine-tune vs. prompt vs. RAG

• Familiarity with parameter-efficient methods (LoRA, etc.) MLOps / LLMOps

• Model evaluation frameworks, A/B testing for model outputs, observability (tracing, logging model calls)

• Deployment patterns: latency/cost optimization, caching, streaming responses, fallback handling

• Versioning prompts and models, not just code

• Safety, evaluation & governance awareness

• Bias/safety evaluation, guardrails, handling PII appropriately

Experience:

• Minimum 7+years of experience in data engineering, with experience working on large-scale, mature data platforms.

• 3+ years of experience developing ML or AI deliverables – includes deployment to production

• Bachelor's degree in related field or demonstrated equivalent experience in a related field required.

• Working knowledge of Agentic Workflows for engineering and architecture

• Demonstrated experience taking autonomous technical ownership of complex projects from design through delivery, with minimal oversight.

• Experience contributing to or shaping AI/ML enablement efforts, such as defining frameworks, evaluating MLOps tooling, or building infrastructure that supports model training and deployment.

• Experience partnering with Data Governance, Data Science, or Compliance teams to align technical practices with governance and regulatory requirements.

• A track record of proposing and driving innovative technical solutions rather than simply executing predefined plans.

• Experience designing or implementing Agentic workflows for data engineering preferred.

• Experience working with Property & Casualty insurance carriers preferred.

• Experience with Data Vault 2.0 or Ensemble data modeling techniques preferred.

Applicants must be authorized to work for any employer in the U.S. We are unable to sponsor or take over work authorization sponsorship now or in the future for this position.


#LI-Hybrid