Manpower Singapore (staffing agency) is hiring a Data Engineer to build and maintain large-scale data pipelines on the Quantexa platform supporting AML solutions, deployed on OpenShift. Day to day: Hadoop/Spark data engineering, containerized deployments, and DevOps work (CI/CD, monitoring, troubleshooting).
We are seeking an experienced Data Engineer with excellent expertise in big data platforms, containerized environments, and DevOps practices. In this role, you will design, build, and maintain scalable data pipelines that process large-scale datasets supporting advanced analytics and machine learning solutions.
You will primarily work on the
Quantexa platform within the AML domain , ensuring high availability, performance, and reliability of data infrastructure deployed on OpenShift Container Platform (OCP).
Key Responsibilities
Data Engineering & Analytics
Design, implement, and maintain data transformation, aggregation, and enrichment pipelines to support analytics and machine learning initiatives.
Process and manage large-scale datasets using Hadoop, Spark, and related technologies.
Ensure data quality, integrity, and consistency across the entire data lifecycle.
Implement data governance practices, including data lineage, metadata management, and compliance controls.
Platform & Containerization
Design and deploy data engineering solutions on OpenShift Container Platform (OCP) using containerization and orchestration techniques.
Optimize data workflows for containerized deployment and efficient resource utilization.
Monitor, tune, and enhance data pipeline performance to ensure scalability and reliability.
DevOps & Operations
Collaborate with DevOps teams to streamline deployments and implement CI/CD pipelines.
Implement monitoring and logging solutions to ensure health, availability, and performance of the data infrastructure.
Troubleshoot production issues and perform root cause analysis with a proactive approach.
Collaboration & Leadership
Work closely with cross-functional teams to translate data requirements into effective engineering solutions.
Document data pipelines, workflows, and infrastructure configurations for knowledge sharing.
Stay current with emerging technologies, industry trends, and best practices in data engineering and DevOps.
Provide technical guidance and mentorship to junior team members, contributing to continuous improvement of analytics capabilities.
Technical & Professional Requirements
Core Experience
Bachelor's degree in Computer Science, Information Technology, or a related discipline.
Minimum
6 years of experience
as a Data Engineer in large-scale data environments.
Hands-on experience developing or maintaining
Quantexa solutions
(AML domain experience).
Big Data & Processing
Solid expertise in Hadoop ecosystem tools (HDFS, Hive, Pig, Cloudera).
Advanced experience with Apache Spark for large-scale data processing.
Proficiency in SQL and data modeling concepts.
Programming & Scripting
Proficiency in one or more of the following:
Scala, Python, Java .
Working knowledge of Shell scripting.
Containerization, DevOps & Tooling
Hands-on experience with OpenShift Container Platform (OCP) and Kubernetes.
Good understanding of DevOps practices, CI/CD pipelines, and automation tools such as Docker, Jenkins, Ansible, and Bitbucket.
Experience with enterprise job schedulers such as
Control-M .
Monitoring & Observability
Experience with monitoring and logging tools such as
Grafana, Prometheus, and Splunk .
Cloud & Platform Exposure
Exposure to cloud platforms such as
AWS, Azure, or GCP , including data services.
Quantexa exposure and/or certification.
Ramakrishna Sarma Srikanth EA License No.: 02C3423 Personnel Registration No.: R22108699
Your Safety and Data Security Matter to Us
ManpowerGroup is committed to a safe and transparent hiring process. We will never request payment, banking details, or sensitive personal information as part of our recruitment. If you receive suspicious outreach claiming to be from us, please contact
(HIDDEN TEXT) .
Please note that your response to this advertisement and subsequent communications with us will constitute informed consent to the collection, use, and disclosure of personal data by Manpower Singapore for recruitment and employment-related purposes, in compliance with the Personal Data Protection Act 2012. To learn more about ManpowerGroup's Global Privacy Policy, please visit:
https://www.manpower.com.sg/privacy-policy .