Data Engineer

• Competitve Salary
• Opportunity to work alongside a large team with cutting edge technology

About Our Client

A company in the insurance industry based in Kuala Lumpur.

Job Description

Data Engineering & Data Platform Development

• Design, develop, and maintain scalable, reliable, and secure data pipelines to support enterprise analytics and reporting initiatives.
• Build and optimize ETL/ELT processes to ingest, transform, and deliver data from various source systems into Data Lake and Data Warehouse environments.
• Design, implement, and maintain cloud-native data solutions on AWS.
• Develop scalable batch and near real-time data processing solutions.
• Develop and maintain Apache Iceberg datasets to support enterprise analytical workloads.
• Ensure data from source systems is successfully loaded into the enterprise Data Lake daily with accuracy, completeness, and timeliness.
• Monitor and troubleshoot data ingestion processes and resolve data-related issues proactively.
• Conduct root cause analysis and implement sustainable solutions for recurring incidents.
• Optimize data processing performance, scalability, and operational efficiency.
• Support enterprise reporting, analytics, and regulatory data requirements.

AWS Cloud Platform & Infrastructure

• Design, build, and support AWS-based data platforms using:
  • AWS Glue
  • Amazon Redshift
  • Amazon S3
  • Amazon RDS
  • Amazon SNS
  • AWS Step Functions
  • Amazon CloudWatch
  • AWS IAM
  • Amazon EventBridge
  • AWS DataZone
• Develop and maintain Infrastructure as Code (IaC) solutions using Terraform.
• Automate environment provisioning, deployment, and operational processes.
• Implement monitoring, alerting, and operational support frameworks to ensure platform stability and high availability.
• Optimize cloud infrastructure for performance, security, reliability, and cost efficiency.
• Support and enhance CI/CD deployment pipelines and DevOps practices.

Disaster Recovery & Business Continuity

• Participate in Disaster Recovery (DR) planning, validation, and execution activities for critical data platforms and services.
• Perform Disaster Recovery (DR) drills, failover, failback, and recovery procedures within established Recovery Time Objective (RTO) and Recovery Point Objective (RPO) requirements.
• Ensure data pipelines, Data Lake, and Data Warehouse platforms can be recovered and restored during disaster scenarios.
• Maintain and regularly update Disaster Recovery documentation, runbooks, and recovery procedures.
• Collaborate with infrastructure, security, and application teams to ensure business continuity readiness.

Data Governance & Quality Management

• Establish and maintain data quality validation, reconciliation, and monitoring controls.
• Ensure data accuracy, consistency, completeness, and reliability across the data platform.
• Support enterprise data governance initiatives through AWS DataZone, including metadata management, data ownership, and data discoverability.
• Collaborate with stakeholders to define and implement data standards, governance policies, and best practices.
• Ensure compliance with enterprise security, privacy, regulatory, and audit requirements.

Stakeholder Collaboration

• Collaborate closely with Business Analysts, Data Analysts, BI Developers, Architects, Product Owners, and business stakeholders to understand and deliver data requirements.
• Translate business requirements into scalable technical designs and data models.
• Provide technical leadership and guidance on data engineering best practices.
• Partner with cross-functional teams to continuously improve data platform capabilities.

Documentation & Knowledge Transfer

• Create and maintain comprehensive technical documentation, including:
  • Solution Design Documents
  • Technical Specifications
  • Data Flow Diagrams
  • Runbooks
  • Support Guides
  • Operational Procedures
  • Disaster Recovery Procedures
• Conduct Knowledge Transfer (KT) sessions to ensure team members understand implemented solutions, processes, and support procedures.
• Promote knowledge sharing, engineering standards, and best practices across the team.
• Mentor junior engineers and support team capability development.

The Successful Applicant

A successful Senior Data Engineer should have:

• Bachelor's Degree in Computer Science, Information Technology, Data Engineering, Engineering, Information Systems, or a related discipline.
• Minimum 3+ years of experience in Data Engineering, Data Warehousing, Data Platform Engineering, or related roles.
• Strong hands-on experience in:
  • SQL
  • Python
  • PySpark
  • Terraform
• Strong experience with AWS services including:
  • AWS Glue
  • Amazon Redshift
  • Amazon S3
  • Amazon RDS
  • Amazon SNS
  • AWS Step Functions
  • Amazon CloudWatch
  • AWS IAM
  • Amazon EventBridge
  • AWS DataZone
• Experience designing and supporting enterprise Data Lake and Data Warehouse solutions.
• Experience working with Apache Iceberg tables and large-scale data processing platforms.
• Strong understanding of ETL/ELT methodologies, data modeling, and data integration techniques.
• Experience implementing Infrastructure as Code (IaC) using Terraform.
• Strong understanding of cloud security, IAM permissions, and access control principles.
• Experience planning, executing, and supporting Disaster Recovery (DR) activities, including failover testing, recovery validation, and business continuity processes.
• Knowledge of Agentic AI, Generative AI, and AI-assisted software development practices.
• Experience utilizing AI-powered coding assistants and modern AI development workflows to improve engineering productivity, accelerate delivery, and enhance code quality.
• Strong analytical, problem-solving, troubleshooting, and communication skills.

What's on Offer

• Contract: 1 year (extendable)
• Medical benefits
• Opportunity to work with cutting edge technology
• Competitive salary