Senior Data Engineer: Spark & Cloud Data Pipelines
UNISONEDGE CONSULTING PTE. LTD. · Singapore, Singapore ·
- Seniority
- Senior
- Category
- Data engineering
Senior Data Engineer designing and maintaining scalable pipelines for large-volume datasets on-site in Singapore. Day to day involves building Spark/PySpark/Scala/Python transformations on Azure Databricks, Data Factory, and AWS EMR, working with Hadoop, Hive, Snowflake, and data lakes, and delivering batch and real-time processing with Spark Streaming and Kafka.
UNISONEDGE CONSULTING PTE. LTD. is seeking an experienced Data Engineer to design and maintain scalable data pipelines for large-volume datasets.
You will develop transformations with Spark, PySpark, Scala and Python while optimizing pipelines on Azure Databricks, Data Factory, and AWS EMR across diverse cloud environments. You will work with Hadoop, HDFS, Hive, Snowflake and Data Lake technologies, delivering batch and real-time processing using Spark Streaming and Kafka.