Data engineer building batch and streaming data pipelines on Google Cloud Platform (BigQuery, Dataflow, Composer, DBT) for a banking-sector data platform project. Day to day involves Python/SQL development, automating deployments with Terraform and GitHub, and testing/monitoring solutions, with office visits 1-2 times per week in Warsaw on a B2B hourly contract.
The company provides IT services and consulting, with this project focused on banking and the development of innovative data platforms in Google Cloud Platform.
Задачи
Develop and maintain data processing solutions in the Google Cloud Platform (GCP) environment;
Create batch and streaming data pipelines using BigQuery, DBT, Dataflow, Composer, and Cloud Storage;
Program in Python and SQL;
Build and optimize data structures supporting analytics and reporting;
Automate deployments using Terraform and GitHub repositories;
Test, monitor, and maintain developed solutions;
Collaborate with analysts, architects, and project teams to deliver business requirements.
Требования
Experience designing and building cloud solutions on Google Cloud Platform (GCP);
Very good knowledge of Python and SQL;
Knowledge of Dataproc, Composer (Airflow), Cloud Storage, and BigQuery;
Experience working with GitHub code repositories;
Experience managing application artifacts using Google Artifact Registry;
Nice to have: knowledge of DBT, knowledge of Terraform.
Условия
Rate: do approx. 158 zł/h net + VAT, B2B; Office visits 1-2 times per week in Warsaw.