Data Engineer-GCP
Xpt Software Australia Pty · New South Wales, Australia ·
- Category
- Data engineering
Xpt Software Australia Pty · New South Wales, Australia ·
Ingrity · Sydney, Australia
ncsaustralia · Brisbane, QLD, au
SmartRecruiters, Inc. · Australia
SmartRecruiters, Inc. · Sydney, Australia
Data Engineer designing and building scalable batch and streaming data pipelines on Google Cloud Platform. Day to day involves ETL/ELT development and data warehouse work with BigQuery, Dataflow, Pub/Sub, and Cloud Storage, using Python, SQL, and Apache Beam or Spark.
Job Details:
Must Have Skills:
ï· Strong
GCP Data Services Expertise - Hands-on experience
with BigQuery, Dataflow, Pub/Sub, Cloud Storage, Ability to design scalable
data solutions using native GCP tools.
ï· Experience in Data Pipeline Development (Batch & Streaming) - Building and maintaining ETL/ELT pipelines, Experience with real-time
streaming and batch processing.
ï· Strong Programming & Query Skills - Proficiency
in Python and SQL, Experience with data processing frameworks like Apache
Beam or Spark
ï· Understanding of Data Modeling & Warehousing - Strong
understanding of data modeling (star/snowflake schemas), Experience designing
and optimizing data warehouses (BigQuery)
Nice to Have Skills:
ï· Experience in Multi-Cloud Exposure : Understanding of cross-cloud data integration and migration, Experience
working with AWS or Azure alongside GCP
ï· Knowledge of Streaming & Messaging Frameworks : Experience with Kafka or advanced Pub/Sub patterns, Knowledge of
real-time data architectures
ï· Basic Understanding of Containerization & Orchestration : Hands-on with Docker and Kubernetes (GKE),
Ability to deploy and manage data workloads in containerized environments
ï· Data
Governance & Security : Familiarity with data
lineage, cataloging, and governance tools, Understanding of IAM roles,
data security, and compliance in GCP.
Kagool · Mumbai, India