Senior AI Data Engineer (Consultant)

We are looking for a Senior Data Engineer Consultant to join an AI consulting engagement focused on LLM architecture, RAG, data strategy, and AI platform advisory.

This is an advisory role rather than a full-time implementation position. The consultant will work with AI architects, ML engineers, MLOps specialists, and the client’s engineering team to assess the current data landscape, review proposed solutions, and provide practical technical recommendations.

Responsibilities

Data Strategy

Review the existing enterprise data landscape and available data sources.

Assess data readiness for foundation model training, fine-tuning, evaluation, and RAG.

Recommend data acquisition, prioritization, and preparation approaches.

Advise on multilingual data preparation strategies.

Data Pipelines

Review the current ETL/ELT architecture and data processing workflows.

Recommend pipeline designs for model training, inference, and evaluation.

Evaluate orchestration, dataset versioning, and data traceability approaches.

Assess pipeline scalability, reliability, maintainability, and performance.

RAG Data Engineering

Review document ingestion architecture for enterprise knowledge bases.

Recommend parsing, chunking, metadata enrichment, embedding, and indexing approaches.

Evaluate vector database and search integration strategies.

Advise on incremental updates, synchronization, access-aware retrieval, and retrieval optimization.

Data Governance and Quality

Review data lifecycle management and data governance practices.

Assess PII handling, data masking, retention policies, access control, lineage, and auditability.

Recommend data validation and quality-control frameworks.

Review training and evaluation datasets.

Advise on monitoring data freshness, consistency, completeness, and data drift.

Documentation and Consulting

Prepare technical recommendations, architecture assessments, and implementation guidelines.

Review technical documentation, diagrams, and proposed designs.

Participate in architecture reviews, design discussions, and technical workshops.

Conduct knowledge-transfer sessions with the client’s engineering team.

Technology Stack

Python, SQL, Apache Airflow, Apache Kafka, Hadoop, Apache Superset, n8n, ETL/ELT, RAG Pipelines, Vector Databases, Data Governance, Cloud (AWS, Azure or GCP).

Preferred Domain Experience

Experience in banking, fintech, financial services, or another highly regulated domain is highly desirable.

The ideal candidate should understand domain-specific requirements related to:

sensitive and confidential data;

PII protection and data masking;

role-based access control;

data retention and auditability;

regulatory and compliance requirements;

secure use of enterprise data in AI and LLM solutions.

Engagement Details

Engagement model: Part-time consulting.

Workload: Flexible and non-uniform, depending on the current project phase.

Expected activities: Architecture reviews, workshops, document reviews, technical assessments, and advisory sessions.

Duration: Approximately 6 months.

Project type: AI/LLM consulting engagement.

Client interaction: Direct collaboration with the client’s engineering and architecture teams.

Availability: The consultant should be able to join scheduled sessions and provide asynchronous reviews between meetings.