AWS Data Engineer

(10+ years of experience required)

Role Overview

We are building a next-generation data platform and are looking for an experienced Senior Data Engineer to help design, develop, and optimize large-scale data solutions. This role involves end-to-end data engineering, modern cloud-based development, and close collaboration with cross-functional stakeholders to deliver reliable, scalable, and high-quality data products.

Key Responsibilities

• Design, develop, and maintain scalable, testable, and high-performance data pipelines using Python and Apache Spark.
  
• Orchestrate data workflows using cloud-native services such as AWS Glue, EMR Serverless, Lambda, and S3.
  
• Apply modern engineering practices including modular design, version control, CI/CD automation, and comprehensive testing.
  
• Support the design and implementation of lakehouse architectures leveraging table formats such as Apache Iceberg.
  
• Collaborate with business stakeholders to translate requirements into robust data engineering solutions.
  
• Build observability and monitoring into data workflows; implement data quality checks and validations.
  
• Participate in code reviews, pair programming, and architecture discussions to promote engineering excellence.
  
• Continuously expand domain knowledge and contribute insights relevant to data operations and analytics.
  

What You’ll Bring

• Strong ability to write clean, maintainable Python code using best practices such as type hints, linting, and automated testing frameworks (e.g., pytest).
  
• Deep understanding of core data engineering concepts including ETL/ELT pipeline design, batch processing, schema evolution, and data modeling.
  
• Hands-on experience with Apache Spark or willingness and capability to learn large-scale distributed data processing.
  
• Familiarity with AWS data services such as S3, Glue, Lambda, and EMR.
  
• Ability to work closely with business and technical stakeholders and translate needs into actionable engineering tasks.
  
• Strong team collaboration skills, especially within Agile environments, emphasizing shared ownership and high transparency.
  

Nice-to-Have Skills

• Experience with Apache Iceberg or similar lakehouse table formats (Delta Lake, Hudi).
  
• Practical exposure to CI/CD tools such as GitLab CI, GitHub Actions, or Jenkins.
  
• Familiarity with data quality frameworks such as Great Expectations or Deequ.
  
• Interest or background in financial markets, analytical datasets, or related business domains.