Data Engineer

Key Responsibilities:

• Develop and maintain data pipelines and ETL/ELT/analytics engineering workflows to support advanced search and retrieval capabilities.


• Collaborate with data teams to understand requirements and automate deployment and monitoring.


• Optimize data storage and troubleshoot issues to enhance performance.


• Work closely with data engineers and relevant parties to ensure sprints are planned out properly and completed timely.


• Perform BAU monitoring, investigations and resolutions.


• Assist with data governance and data management initiatives.


• Production and application support from a day-to-day basis.

Skillset Requirements:

• Minimum 5 years of hands-on experience in software or data engineering with proficiency in Python.


• Strong experience in unit and integration testing.


• Familiarity with DevOps practices and Agile methodologies.


• Experience with AWS and Kubernetes (K8s).


• Familiarity with data platforms such as Snowflake, Databricks, Apache Spark, Apache Hive, open table formats (Delta Lake, Apache Iceberg), and vector databases.


• Experience with orchestration tools such as Apache Airflow, Dagster, Prefect, and Temporal.


• Familiarity with GitHub workflows and Datadog.


• Good written and verbal communication skills.


• Agile, fast learner and able to adapt to changes.