Role
This role oversees and manages the ingestion framework to ingest data from various data sources for data analytics and AI purposes. Detailed activities include:
- Analyzing the data requirements across various entities, develop, implement and maintain data ingestion pipelines from source to data lake pipelines.
- Automating data validation steps and report generation.
- Automating codes/ scripts that can be repeatedly used across similar analytics and reporting requirements.
- Managing and ensuring accuracy and timeliness and automation solution in end to end data extraction and integration with analytics and management reporting system.
- Designing the jobs schedule in scheduling tools and developing the configuration files for job schedulers.
- Performing production L2 batch support after production deployment.
- Perform code review functions for applications / programs developed by team members.
- To be part of initiatives that brings data into the data lake and delivers insights.
- Monitor and measure performance to assure ongoing data ingestion is meeting the SLA and optimization of the ingestion process to manage the performance and the SLAs.
- Work effectively with other stakeholders such as data engineering team, IT team, etc.
- Troubleshoot MapReduce/Spark Jobs and do performance tuning in production environments.
- Independently develop and sustain technical knowledge, certifications, and skills.
- Effectively handling day-to-day assignments given moderate directions and supervision.
- Bachelor or Master Degree in Computer Science, Engineering, or similar relevant field.
- Working experience in data ingestion or data engineering with Hadoop tech stack for 8+ years.
- Proficient with Scala and PySpark.
- Hands-on experience on Spark framework and other distributed data processing frameworks like Hadoop Map-Reduce, Hive etc. Proficient in ETL tools like Talend.
- Proficient in RDBMS databases such as Oracle, MySQL, MSSqlServer.
- Strong scripting skills in Linux environment and SQL.
- Expertise in Hadoop ecosystems.
- Hands-on Experience in Sqoop, Hive, Spark, Python, Scala is a must.
- Hands-on Experience in Job orchestration / Job schedulers like Autosys, Control-M
- Good to have working experience with one of the cloud platforms like AWS (Amazon Web Services), Microsoft Azure or Google Cloud Platform.
- Ability to plan and organise technical work and deliverables.
- Ability to follow guidelines and adhere to the established software development standards and conventions.
- S
elf-motivated and independent.
- Able to work with minimum supervision and to work well with stakeholders and project staff.
- Ability to prioritize and multi-task across numerous work streams.
- S
strong interpersonal skills; ability to work on cross-functional teams.
- Strong verbal and written communication skills.
- Deep knowledge of best practices through relevant experience across data-related disciplines and technologies particularly for enterprise wide data architectures and data warehousing/BI.
- Demonstrated problem-solving skills. Ability to learn effectively and meet deadline.
Role
This role oversees and manages the ingestion framework to ingest data from various data sources for data analytics and AI purposes. Detailed activities include:
- Analyzing the data requirements across various entities, develop, implement and maintain data ingestion pipelines from source to data lake pipelines.
- Automating data validation steps and report generation.
- Automating codes/ scripts that can be repeatedly used across similar analytics and reporting requirements.
- Managing and ensuring accuracy and timeliness and automation solution in end to end data extraction and integration with analytics and management reporting system.
- Designing the jobs schedule in scheduling tools and developing the configuration files for job schedulers.
- Performing production L2 batch support after production deployment.
- Perform code review functions for applications / programs developed by team members.
- To be part of initiatives that brings data into the data lake and delivers insights.
- Monitor and measure performance to assure ongoing data ingestion is meeting the SLA and optimization of the ingestion process to manage the performance and the SLAs.
- Work effectively with other stakeholders such as data engineering team, IT team, etc.
- Troubleshoot MapReduce/Spark Jobs and do performance tuning in production environments.
- Independently develop and sustain technical knowledge, certifications, and skills.
- Effectively handling day-to-day assignments given moderate directions and supervision.
- Bachelor or Master Degree in Computer Science, Engineering, or similar relevant field.
- Working experience in data ingestion or data engineering with Hadoop tech stack for 8+ years.
- Proficient with Scala and PySpark.
- Hands-on experience on Spark framework and other distributed data processing frameworks like Hadoop Map-Reduce, Hive etc. Proficient in ETL tools like Talend.
- Proficient in RDBMS databases such as Oracle, MySQL, MSSqlServer.
- Strong scripting skills in Linux environment and SQL.
- Expertise in Hadoop ecosystems.
- Hands-on Experience in Sqoop, Hive, Spark, Python, Scala is a must.
- Hands-on Experience in Job orchestration / Job schedulers like Autosys, Control-M
- Good to have working experience with one of the cloud platforms like AWS (Amazon Web Services), Microsoft Azure or Google Cloud Platform.
- Ability to plan and organise technical work and deliverables.
- Ability to follow guidelines and adhere to the established software development standards and conventions.
- S
elf-motivated and independent.
- Able to work with minimum supervision and to work well with stakeholders and project staff.
- Ability to prioritize and multi-task across numerous work streams.
- S
strong interpersonal skills; ability to work on cross-functional teams.
- Strong verbal and written communication skills.
- Deep knowledge of best practices through relevant experience across data-related disciplines and technologies particularly for enterprise wide data architectures and data warehousing/BI.
- Demonstrated problem‑solving skills. Ability to learn effectively and meet deadline.
How You Succeed
- Champion and embody our Core Values in everyday tasks and interactions.
- Demonstrate high level of integrity and accountability.
- Take initiative to drive improvements and embrace change.
- Take accountability of business and regulatory compliance risks, implementing measures to mitigate them effectively.
- Keep abreast with industry trends, regulatory compliance, and emerging threats and technologies to understand and highlight potential concerns/ risks to safeguard our company proactively.
Who We Are
Founded in 1908, Great Eastern is a well-established market leader and trusted brand in Singapore and Malaysia. With over S$100 billion in assets and more than 16 million policyholders, including 12.5 million from government schemes, it provides insurance solutions to customers through three successful distribution channels – a tied agency force, bancassurance, and financial advisory firm Great Eastern Financial Advisers. The Group also operates in Indonesia and Brunei. The Great Eastern Life Assurance Company Limited and Great Eastern General Insurance Limited have been assigned the financial strength and counterparty credit ratings of "AA-" by S&P Global Ratings since 2010, one of the highest among Asian life insurance companies. Great Eastern's asset management subsidiary, Lion Global Investors Limited, is one of the leading asset management companies in Southeast Asia. Great Eastern is a subsidiary of OCBC, the longest established Singapore bank, formed in 1932. It is the second largest financial services group in Southeast Asia by assets and one of the world’s most highly-rated banks, with an Aa1 rating from Moody’s and AA- by both Fitch and S&P. Recognised for its financial strength and stability, OCBC is consistently ranked among the World’s Top 50 Safest Banks by Global Finance and has been named Best Managed Bank in Singapore by The Asian Banker.
To All Recruitment Agencies
Great Eastern does not accept unsolicited agency resumes. Please do not forward resumes to our email or our employees. We will not be responsible for any fees related to unsolicited resumes.
Work Locations
SG-GE Centre
Job
Data and AI