Senior Data Engineer at Credit Agricole CIB's Singapore Data & Analytics team: design, build, and run production ETL/ELT pipelines and on-premise lakehouse architecture. Core stack: Python, PySpark, SQL Server, Kubernetes, Docker, Airflow, and Iceberg. 12-month renewable contract, Singaporeans/PRs only.
Crédit Agricole Corporate and Investment Banking (Crédit Agricole
CIB) is the corporate and investment banking arm of Crédit Agricole Group, world's 10th largest bank by total assets.
Our Singapore center is the
2nd largest IT setup (after Paris Head Office) for Crédit Agricole CIB's worldwide
business. We work daily with international branches located in 30 markets by:
Envisioning and preparing the Bank's futures information systems
Partnering and supporting core banking flagships and transverse areas in their large-scale development projects
Providing premium In-house Banking applications
This unique positioning empowers us to bring our core banking business a sustainable competitive advantage on the market. We seek innovative and agile people sharing our mindset to support ambitious and forthcoming technological challenges.
Position
We are seeking a
Senior Data Engineer
to join our Data & Analytics team. You will design, build, and maintain robust on-premise data pipelines, architect flexible lakehouse solutions, and lead data infrastructure initiatives to enable data-driven decision-making across the organization.
This is an exciting opportunity for a talented technical leader to be part of a high-performing engineering team responsible for building scalable, production-grade data platforms that support critical business operations across multiple domains.
Main responsibilities
Main responsibilities of the role:
Design and develop autonomous, production-grade ETL/ELT data pipelines using Python and PySpark that ingest, transform, and deliver high-quality data while maintaining integrity and performance standards.
Implement and manage flexible lakehouse architecture across raw, curated, and consumption layers, including data partitioning, cataloging, and metadata management.
Deploy and manage data pipelines using Kubernetes and Docker to ensure scalability, reliability, and efficient resource utilization in on-premise environments.
Leverage strong SQL Server expertise to design optimal data models, write complex queries, and perform query optimization across the data platform.
Establish and maintain robust CI/CD practices for data pipeline deployment, including automated testing, version control, and continuous monitoring.
Enforce security, governance, and role-based access controls across all data layers while ensuring compliance and auditability.
Mentor junior engineers, conduct code reviews, and establish best practices across the team.
Collaborate with Data Scientists, Business Analysts, and stakeholders to deliver datasets aligned with operational and analytical needs.
Provide L3 support and expert consultation for complex data challenges; evaluate and recommend new tools and practices to improve agility and performance.
Qualifications and Profile
Must-have qualifications
8+ years
IT experience;
5+ years
hands-on data engineering or data pipeline development
Expert-level SQL proficiency
with strong expertise in SQL Server, including query optimization, indexing, and performance tuning
Advanced Python programming skills
for data processing, automation, and production-grade pipeline development
Kubernetes expertise
– Design, deploy, and manage containerized data pipelines in on-premise environments
Strong data modeling expertise
– Both relational and non-relational concepts
Proven experience with flexible lakehouse/data lake architecture
– Multi-layer data lakes, partitioning strategies, and metadata management,
Iceberg
tables and optimization
CI/CD and DevOps practices
– Setting up CI/CD pipelines, Git, automated testing, and infrastructure-as-code tools
ETL/ELT orchestration experience
– Apache Airflow or similar tools for scheduling and monitoring batch and real-time jobs
Hands-on experience with at least one NoSQL database
(MongoDB, Cassandra, etc.)
Hands-on experience with Apache Spark and PySpark
for distributed data processing and performance optimization
Data security and governance
– Role-based access control, data masking, and compliance sframeworks
Proven ability to work autonomously
on complex projects while maintaining high code quality standards
Excellent problem-solving, communication, and cross-functional collaboration skills
Bachelor's degree in Computer Science, IT, Engineering, or related field
with demonstrated continuous learning ethos.
Preferred qualifications
Experience with on-premise data virtualization or logical data warehouse concepts
Understanding of data mesh or data fabric architecture patterns
Real-time streaming technologies (Kafka, Apache Flink)
Metadata management and data lineage tools
Experience mentoring junior engineers or leading technical initiatives
Agile delivery methodologies and product-oriented data architecture
Other Professional Skills and Mind-set
Autonomous Work Ethic – Work independently on complex problems while proactively seeking collaboration
Continuous Learning – Committed to staying current with data engineering trends and best practices
Note - This is 12 months contract role. Renewable subject to performance.
We offer a competitive remuneration package, consistent with your qualifications and experience. For fair employment practices, we are keen on
Singaporeans/SPR ONLY.
Interested applicants, please click on 'APPLY'
Visit us on:
http://www.ca-cib.com