Company Overview
OPTIMUM SOLUTIONS (SINGAPORE) PTE LTD is a Singapore-headquartered IT outsourcing firm specializing in software engineering, application development, and IT infrastructure management. Since 1997, it has grown into a leading provider of digital transformation services.
Job Summary
We are seeking a Senior Data Engineer with expertise in enterprise data platforms and banking domains to lead the design and development of scalable data pipelines across Teradata, Hadoop/Cloudera, and cloud ecosystems while mentoring engineering talent and driving technical excellence.
Responsibilities
- Lead the architecture and implementation of scalable batch, streaming, and API-based data pipelines to support large-scale data delivery and historical data processing frameworks.
- Design and implement Slowly Changing Dimension (SCD) and Change Data Capture (CDC) mechanisms to maintain accurate and up-to-date data products.
- Develop semantic-layer data products that enhance data accessibility and usability for business stakeholders.
- Optimise complex SQL queries, Spark workloads, partition strategies, and storage frameworks to improve performance for large-volume data processing in regulated enterprise banking environments.
- Implement infrastructure-as-code using Terraform to automate provisioning and management of cloud and on-premises resources.
- Establish automated Continuous Integration/Continuous Deployment (CI/CD) pipelines and containerized deployment workflows to ensure reliable and repeatable software delivery.
- Enforce rigorous testing standards to maintain high-quality, production-ready data engineering solutions.
- Lead code reviews and enforce engineering best practices to uphold technical excellence across teams.
- Mentor junior engineers to develop their technical skills and promote disciplined delivery in regulated settings.
- Collaborate closely with architects, product teams, DevOps, and business stakeholders to align data engineering efforts with organizational goals.
Required competencies and certifications
- Bachelor’s degree in Computer Science, Engineering, Information Technology, or a related discipline (required education).
- At least 10 years of data engineering experience with technical leadership in Teradata and Hadoop/big-data delivery.
Preferred competencies and qualifications
- Certifications in Teradata, Cloudera, Databricks, Cloud Data Engineering, or Terraform.
- Proficiency with core technologies including Teradata, Hadoop/Cloudera, Spark, Hive, Impala, HBase, SQL, Python, Scala/Java, Kafka, Airflow, and Denodo.
- Experience with infrastructure and DevOps tools such as Terraform, Kubernetes/OpenShift, Git, and CI/CD automation tools.
- Domain expertise in enterprise banking data including Customer, Payments, Risk, Finance, AML, and Regulatory Reporting, with a focus on data quality, reconciliation, lineage, and historical reconstruction.
- Demonstrated technical leadership, disciplined delivery in regulated environments, strong communication skills, and accountability for production-ready outcomes.