Build and maintain large-scale data pipelines on Google Cloud Platform using PySpark, BigQuery, and Airflow to process and analyze high-volume datasets.
Big Query background with data processing for large volumes
Another NO Sql DB is big plus - Spanner DB & Graph DB
Qualifications:
Degree in Computer Science, Applied Mathematics, Engineering, or any other technology related field (or equivalent work experience)
6+ years of experience working as a big data engineer (Required) must be able to articulate use cases supported and outcomes driven.
knowledge of the following (expectation is to demonstrate these skills live during the interview): PySpark, Spark, Python, Scala, Hive, Pig, and MapReduce.
Experience in GCP.
Experience in SQL.
Large scale data engineering and business intelligence delivery experience
Design of large-scale enterprise level big data platforms
Experience working with and performing analysis using large data sets
Proven and Demonstrated experience working with or on a mature, self-organized agile team