We are seeking a Senior AI Engineer with Spark and AWS Services expertise to join the RBQM Production Pod within the program. You will build and maintain data pipelines that power AI/GenAI applications for Risk-Based Quality Management in clinical trials. This role focuses on RAG document ingestion, vector indexing, and building data APIs for AI applications.
Responsibilities
- Design and build RAG document ingestion pipelines (chunking, embedding, vector indexing) for clinical trial quality data
- Build and manage vector databases (AWS OpenSearch) for RAG-powered AI workflows
- Develop batch and streaming ETL/ELT pipelines from scratch for unstructured clinical data (PDF, DOCX, clinical reports)
- Build and expose data APIs for AI application consumption
- Optimize chunking strategies, embedding generation, and retrieval performance for RAG architectures
- Manage data quality, lineage, and governance for AI/ML data pipelines
- Deploy and maintain AWS data infrastructure (S3, Lambda, Glue, Athena, Step Functions, DynamoDB)
- Collaborate with Data Scientists and Backend Developers in an integrated pod team
Requirements
- 5+ years of hands-on data engineering experience at scale
- Expertise in RAG document ingestion pipelines (chunking, embedding, vector indexing)
- Proficiency in AWS OpenSearch as a vector database for RAG workflows
- Advanced proficiency in Python, including SQL and Spark SQL
- Skills in unstructured data transformation (PDF, DOCX) for RAG/LLM applications
- Familiarity with AWS Services: S3, Lambda, Glue, Athena, Bedrock, Step Functions, API Gateway, CloudWatch, DynamoDB
- Knowledge of containerization with Docker
- Capability to build custom pipelines from scratch, beyond configuring out-of-the-box services
- Proficiency in English at a B2+ level
Nice to have
- Background in pharmaceutical or life sciences domain
- Familiarity with Snowflake, Pinecone (vector DB alternative)
- Knowledge of SageMaker processing jobs
- Skills in CI/CD tools (Jenkins, Git/Bitbucket) and infrastructure tools (CDK or Terraform)
- Understanding of clinical data standards (CDISC, ADaM, SDTM)
Benefits
We connect like-minded people
- Delivering innovative solutions to industry leaders, making a global impact
- Enjoyable working environment, whether it is the vibrant office or the comfort of your own home
- Opportunity to work abroad for up to two months per year
- Relocation opportunities within our offices in 55+ countries
- Corporate and social events
We invest in your growth
- Leadership development, career advising, soft skills and well-being programs
- Certifications, including GCP, Azure and AWS
- Unlimited access to EPAM's internal learning database
- Free English classes with certified teachers
We cover it all
- Participation in the Employee Stock Purchase Plan
- Monetary bonuses for engaging in the referral program
- Comprehensive medical & family care package
- Five trust days per year (sick leave without a medical certificate)
- Benefits package (sports activities, a variety of stores and services)
We connect like-minded people
- Delivering innovative solutions to industry leaders, making a global impact
- Enjoyable working environment, whether it is the vibrant office or the comfort of your home
- Opportunity to work abroad for up to two months per year
- Relocation opportunities within our offices in 55+ countries
- Corporate and social events
We invest in your growth
- Leadership development, career advising, soft skills and well-being programs
- Certifications, including GCP, Azure and AWS
- Unlimited access to EPAM's internal learning database
- Free English classes with certified teachers
We cover it all
- Participation in the Employee Stock Purchase Plan
- Monetary bonuses for engaging in the referral program
- Comprehensive medical & family care package
- Four trust days per year for personal needs
- Discounts for fitness clubs
- Benefits package (hotels, restaurants, stores and services)
We connect like-minded people:
- Delivering innovative solutions to industry leaders, making a global impact
- Enjoyable working environment, whether it is the vibrant office or the comfort of your own home
- Opportunity to work abroad for up to two months per year
- Relocation opportunities within our offices in 55+ countries
- Corporate and social events
We invest in your growth:
- Leadership development, career advising, soft skills and well-being programs
- Certifications, including GCP, Azure and AWS
- Unlimited access to EPAM's internal learning database
- Free English classes with certified teachers
- Discounts in local language schools, including online courses for the Kazakh language
We cover it all:
- Participation in the Employee Stock Purchase Plan
- Monetary bonuses for engaging in the referral program
- Medical & family care package
- Six trust days per year (sick leave without a medical certificate)
- Coverage of psychology sessions of your choice
- Benefits package (sports activities, a variety of stores and services)
- Housing support program: preferential mortgage access via Otbasy Bank partnership
We connect like-minded people:
- Delivering innovative solutions to industry leaders, making a global impact
- Enjoyable working environment, whether it is the vibrant office or the comfort of your own home
- Opportunity to work abroad for up to two months per year
- Relocation opportunities within our offices in 55+ countries
- Corporate and social events
We invest in your growth:
- Leadership development, career advising, soft skills and well-being programs
- Certifications, including GCP, Azure and AWS
- Unlimited access to EPAM's internal learning database
- Free English classes with certified teachers
We cover it all:
- Monetary bonuses for engaging in the referral program
- Medical & family care package
- Six trust days per year (sick leave without a medical certificate)
- Coverage of psychology sessions of your choice
- Discounts for fitness clubs and sports programs
- Benefits package (sports activities, a variety of stores and services)
We connect like-minded people:
- Delivering innovative solutions to industry leaders, making a global impact
- Enjoyable working environment, whether it is the vibrant office or the comfort of your own home
- Opportunity to work abroad for up to two months per year
- Relocation opportunities within our offices in 55+ countries
- Corporate and social events
We invest in your growth:
We cover it all:
- Monetary bonuses for engaging in the referral program
- Medical & family care package
- Four trust days per year (sick leave without a medical certificate)
- Discounts for fitness clubs, dance schools and sports programs
- Benefits package (sports activities, a variety of stores and services)