Role Overview
We are looking for an experienced Senior AI Engineer – Cloud Data Platform to help enhance our enterprise cloud data platform with AI, Generative AI, and intelligent automation capabilities.
The ideal candidate will bring a strong foundation in data engineering and modern cloud data platforms, combined with hands-on experience building and productionizing AI/ML and Generative AI solutions.
This role requires an engineer who can work across data pipelines, cloud services, APIs, LLMs, vector databases, and enterprise applications to build scalable AI capabilities on top of an established data platform.
Experience
Overall Experience: 6+ years in Data Engineering / Cloud Data Platforms
Relevant AI Experience: Minimum 1+ year of hands-on experience in AI, Generative AI, LLM-based applications, or ML engineering
Key Responsibilities
AI & Generative AI Engineering
- Design and develop AI and Generative AI capabilities integrated with the existing cloud data platform.
- Build enterprise applications using Large Language Models (LLMs) and foundation models.
- Develop Retrieval-Augmented Generation (RAG) solutions using enterprise structured and unstructured data.
- Design prompting, context management, grounding, and retrieval strategies for enterprise AI applications.
- Build AI agents and agentic workflows for data discovery, analytics, operational automation, and knowledge retrieval.
- Implement embeddings, semantic search, vector indexing, and vector database solutions.
- Integrate enterprise data with LLM platforms and AI services through secure APIs.
- Implement mechanisms for evaluating AI responses for accuracy, relevance, hallucination, and overall quality.
- Develop appropriate guardrails, observability, monitoring, and responsible-AI controls for production AI applications.
Cloud Data Platform Engineering
- Enhance and extend existing enterprise cloud data platform capabilities.
- Design and develop scalable data ingestion, transformation, and processing pipelines.
- Work with structured, semi-structured, and unstructured datasets.
- Build reusable data services and APIs that can be consumed by AI applications.
- Optimize data pipelines and storage for performance, scalability, reliability, and cost.
- Support data quality, metadata management, lineage, governance, and security requirements.
- Work with batch and real-time/streaming data processing patterns.
- Ensure AI solutions integrate effectively with existing data architecture and enterprise security standards.
Solution Engineering
- Translate business requirements and use cases into scalable AI/data engineering solutions.
- Develop reusable frameworks and components for AI-enabled data platform capabilities.
- Conduct technical POCs and rapidly evaluate new AI technologies and frameworks.
- Productionize successful prototypes following enterprise engineering standards.
- Collaborate with Data Architects, Data Engineers, Cloud Engineers, Data Scientists, Product Owners, and business stakeholders.
- Participate in architecture/design discussions, code reviews, troubleshooting, and performance optimization.
Required Technical Skills
Data Engineering
Strong hands-on experience with:
- Python
- SQL
- Data engineering and ETL/ELT frameworks
- Data modeling and data processing
- REST APIs and microservices
- Distributed data processing technologies such as Apache Spark
- Modern cloud data platforms
Experience with one or more cloud ecosystems:
- AWS
- Microsoft Azure
- Google Cloud Platform (GCP)
Experience with modern data platforms/technologies such as:
- Databricks
- Snowflake
- Cloud-native data lakes/lakehouses
- Delta Lake / Iceberg or similar technologies
- Airflow or equivalent orchestration frameworks
AI / Generative AI
Hands-on experience with:
- Large Language Models (LLMs)
- Generative AI application development
- Prompt engineering
- Retrieval-Augmented Generation (RAG)
- Embeddings and semantic search
- Vector databases
- LLM APIs and model integration
- AI agents / agentic workflows
- LLM evaluation and monitoring
Experience with frameworks/platforms such as:
- OpenAI / Azure OpenAI
- Anthropic Claude
- Google Gemini
- Hugging Face
- LangChain
- LangGraph
- LlamaIndex
- Semantic Kernel or similar AI orchestration frameworks
Experience with vector technologies such as:
- Pinecone
- Weaviate
- Milvus
- FAISS
- pgvector
- Azure AI Search
- OpenSearch or equivalent
Preferred Skills
- Experience building enterprise-grade GenAI applications rather than only prototypes or demos.
- Experience implementing RAG over enterprise data sources.
- Understanding of AI agents, tool calling, MCP, and multi-agent architectures.
- Experience integrating AI applications with databases, APIs, enterprise applications, and document repositories.
- Knowledge of MLOps / LLMOps concepts.
- Experience with Docker and Kubernetes.
- CI/CD and DevOps experience.
- Experience with infrastructure-as-code tools such as Terraform.
- Understanding of cloud security, IAM, encryption, secrets management, and data privacy.
- Knowledge of data governance and responsible AI principles.
- Experience implementing observability and cost monitoring for AI applications.
Education
Bachelor's or Master's degree in:
- Computer Science
- Information Technology
- Data Science
- Artificial Intelligence
- Engineering
or a related technical discipline.