Recruiting firm PERSOL is hiring a Big Data Engineer to work with its client Huawei in Singapore, developing and maintaining Huawei's big data stack. Day-to-day work centers on Hadoop, Spark, and Hive — extracting and modeling data, running ETL pipelines, optimizing cost/compute/storage, and building reports and visualizations for search and recommendation systems.
About the company:
We have partnered with a renowned global leader in information and communications technology (ICT) infrastructure and smart devices. They are providing full-stack, all-scenario solution for products and services carriers, enterprises, governments, and individual consumers worldwide.
Our client is looking for a
Big Data Engineer
to join the team.
Job Description:
Development and maintenance of Huawei Big Data Stack. Extraction of structured and unstructured data from different sources and create a structured Data Model. Daily work will be on Hadoop, Spark, Hive and other related tools and technologies.
In this role, you will be designing and developing data and features for Huawei's search and recommendation systems.
Develop reports from the data based on the business requirements. Develop and Maintain systems for Self-Analytics and other Visualization requirements.
Maintain and Manage the daily running of the ETL jobs and fix the data pipeline issues, focus on 'cost, compute, storage' optimization tasks on day to day basis.
Ensure the data quality, privacy, security as mandated by the company and take necessary actions.
Skills / Qualifications:
Master Degree/Bachelors in Computer Science or other related fields with coding and development background in the data base queries and related products.
You should be strong in coding algorithms and data structures. Preferred programming languages (Python, Java, Scala.).
Strong Working Experience with Hive, ETL, SQL, data modeling, very strong experience in writing complex SQL/Hive queries in optimized way so that it can run on huge data set with optimized resource usage.
Working experience in large-scale data pipeline development for recommendation and search is preferred .
Strong working experience in big data technologies such as Apache Spark, Hive, Hadoop and Linux.
Strong capability to coalesce and present complex data sets from different sources to different data consumers.
Strong working experience in data visualization and report generation, through tools like Tableau/PowerBI/FineBI etc.
Working experience with large-scale data processing frameworks such as Spark, Flink, and Kafka.
Solve data integration problems, utilizing optimal ETL patterns, frameworks, query techniques, sourcing from structured and unstructured data sources
Assist in owning existing processes running in production, optimizing complex code through advanced algorithmic concepts
Maintain the data pipeline, ensure if the data is valid, avoid data issues before exposing.
Interested candidates, who wish to apply for the advertised position, please click on 'Apply Now'. We regret that only shortlisted candidates will be notified.
EA License No.: 01C4394
By sending us your personal data and curriculum vitae (CV), you are deemed to consent to PERSOL Singapore Pte Ltd and its affiliates to collect, use and disclose your personal data for the purposes set out in the Privacy Policy available at https://www.persolsingapore.com/policies. You acknowledge that you have read, understood, and agree with the Privacy Policy