Data Engineer (Python Backend, IT)47k

$42300 - $47k p.m. + Medical+10%Gratuity

Description

• Design, develop, and maintain scalable data pipelines and data products within a cloud-based data platform.
• Build, enhance, and troubleshoot Apache Spark workloads on AWS EMR for both batch and real-time data processing.
• Enable self-service analytics and data science capabilities through well-structured, governed, and high-quality datasets.
• Contribute to MLOps processes, including support for model deployment, monitoring, and data preparation activities for production ML solutions.
• Assist with AI and Generative AI initiatives by delivering the necessary data foundations and platform integrations.
• Maintain data quality, governance, lineage, security, and consistency across the end-to-end data lifecycle.
• Collaborate with business users, data specialists, and platform engineering teams to gather requirements and implement data-driven solutions.
• Adhere to established architecture guidelines, engineering best practices, and development standards.

Profile

• 3-5 years of hands-on experience in data engineering.
• Strong practical experience with Apache Spark running on AWS EMR.
• Experience working with AWS data services, including S3, Glue, Athena, Redshift, Step Functions, and Lambda; exposure to Lake Formation and SageMaker is advantageous.
• Proficiency in Python and advanced SQL, with experience in ETL/ELT development and data modelling.
• Familiarity with big data and streaming technologies such as Hive, Presto, Kafka, and Spark Streaming.
• Knowledge of MLOps principles and experience supporting machine learning models in production environments is beneficial.
• Exposure to AI and Generative AI-related projects is an advantage.
• Strong collaboration skills and effective communication abilities.
• Fluent in Cantonese and English; Mandarin proficiency is a plus.
• Prior experience gained within an IT consulting environment or technology services organisation is preferred.

Job Offer

• Competitive monthly salary.
• Access to the usual benefits provided by the employer.