Data Engineer at Shopee in Singapore building and operating the Marketplace data warehouse end to end — batch and real-time ingestion, semantic layer design, and large-scale pipeline optimization. Works with Spark, Flink, Hadoop/HBase and Java/Scala/Python/SQL to support BI, self-serve analytics, and algorithmic applications.
Build and operate the Marketplace data warehouse end to end, covering batch and real-time ingestion, semantic layer design, and application-layer modeling for downstream consumers.
Collaborate with product, business, and engineering teams to translate requirements into scalable data architectures that support BI reporting, self-serve analytics, and algorithmic applications.
Design and optimize large-scale data processing pipelines, improving throughput, latency, and cost efficiency across offline and streaming platforms.
Lead data quality and governance efforts, including schema standardization, lineage tracking, SLA monitoring, and best practices that increase reliability and user trust.
Evaluate and adopt emerging data technologies such as streaming engines, lakehouse architectures, and AI/ML data stacks to address key Marketplace challenges and evolve the platform roadmap.
Job Description
Build and operate the Marketplace data warehouse end to end, covering batch and real-time ingestion, semantic layer design, and application-layer modeling for downstream consumers.
Collaborate with product, business, and engineering teams to translate requirements into scalable data architectures that support BI reporting, self-serve analytics, and algorithmic applications.
Design and optimize large-scale data processing pipelines, improving throughput, latency, and cost efficiency across offline and streaming platforms.
Lead data quality and governance efforts, including schema standardization, lineage tracking, SLA monitoring, and best practices that increase reliability and user trust.
Evaluate and adopt emerging data technologies such as streaming engines, lakehouse architectures, and AI/ML data stacks to address key Marketplace challenges and evolve the platform roadmap.
Requirements
Bachelor's Degree or above in Computer Science or related fields
At least 3 years of data engineering experience
Prior experience or knowledge working with Traffic Mart would be advantageous
Experience with big data processing technologies such as Spark, Flink, Hadoop or HBase
Proficiency in one or more programming languages, such as Java, Scala, Python, SQL, etc.
Familiar with data warehouse architecture and principles