We are seeking an experienced Data Engineer with excellent expertise in big data platforms, containerized environments, and DevOps practices. In this role, you will design, build, and maintain scalable data pipelines that process large-scale datasets supporting advanced analytics and machine learning solutions.
You will primarily work on the Quantexa platform within the AML domain, ensuring high availability, performance, and reliability of data infrastructure deployed on OpenShift Container Platform (OCP).
Key Responsibilities
Data Engineering & Analytics
- Design, implement, and maintain data transformation, aggregation, and enrichment pipelines to support analytics and machine learning initiatives.
- Process and manage large-scale datasets using Hadoop, Spark, and related technologies.
- Ensure data quality, integrity, and consistency across the entire data lifecycle.
- Implement data governance practices, including data lineage, metadata management, and compliance controls.
Platform & Containerization
- Design and deploy data engineering solutions on OpenShift Container Platform (OCP) using containerization and orchestration techniques.
- Optimize data workflows for containerized deployment and efficient resource utilization.
- Monitor, tune, and enhance data pipeline performance to ensure scalability and reliability.
DevOps & Operations
- Collaborate with DevOps teams to streamline deployments and implement CI/CD pipelines.
- Implement monitoring and logging solutions to ensure health, availability, and performance of the data infrastructure.
- Troubleshoot production issues and perform root cause analysis with a proactive approach.
Collaboration & Leadership
- Work closely with cross-functional teams to translate data requirements into effective engineering solutions.
- Document data pipelines, workflows, and infrastructure configurations for knowledge sharing.
- Stay current with emerging technologies, industry trends, and best practices in data engineering and DevOps.
- Provide technical guidance and mentorship to junior team members, contributing to continuous improvement of analytics capabilities.
Technical & Professional Requirements
Core Experience
- Bachelor's degree in Computer Science, Information Technology, or a related discipline.
- Minimum 6 years of experience as a Data Engineer in large-scale data environments.
- Hands‑on experience developing or maintaining Quantexa solutions (AML domain experience).
Big Data & Processing
- Solid expertise in Hadoop ecosystem tools (HDFS, Hive, Pig, Cloudera).
- Advanced experience with Apache Spark for large-scale data processing.
- Proficiency in SQL and data modeling concepts.
Programming & ing
- Proficiency in one or more of the following: Scala, Python, Java.
- Working knowledge of Shell ing.
Containerization, DevOps & Tooling
- Hands‑on experience with OpenShift Container Platform (OCP) and Kubernetes.
- Good understanding of DevOps practices, CI/CD pipelines, and automation tools such as Docker, Jenkins, Ansible, and Bitbucket.
- Experience with enterprise job schedulers such as Control‑M.
Monitoring & Observability
- Experience with monitoring and logging tools such as Grafana, Prometheus, and Splunk.
Cloud & Platform Exposure
- Exposure to cloud platforms such as AWS, Azure, or GCP, including data services.
- Quantexa exposure and/or certification.
Ramakrishna Sarma Srikanth EA License No.: 02C3423 Personnel Registration No.: R22108699
Your Safety and Data Security Matter to Us
ManpowerGroup is committed to a safe and transparent hiring process. We will never request payment, banking details, or sensitive personal information as part of our recruitment. If you receive suspicious outreach claiming to be from us, please contact (HIDDEN TEXT).
Please note that your response to this advertisement and subsequent communications with us will constitute informed consent to the collection, use, and disclosure of personal data by Manpower Singapore for recruitment and employment-related purposes, in compliance with the Personal Data Protection Act 2012. To learn more about ManpowerGroup's Global Privacy Policy, please visit: https://www.manpower.com.sg/privacy-policy.