Cloud Data Engineer
SIX · Warszawa, Poland ·
- Category
- Data engineering
- Experience
- 4+ years
SIX · Warszawa, Poland ·
SIX Group
HAVI
One51 | Data & Analytics Consultancy · New South Wales, Australia
ShyftLabs · Toronto, Ontario
Data platform engineer at SIX building and maintaining a company-wide cloud-native data platform serving internal analytics teams. Hands-on work building ETL and ML pipelines, infrastructure-as-code components, and enabling 600+ users. Core stack: Databricks/Spark (Delta/Iceberg), Python, SQL, Terraform/IaC on Azure (plus AWS/GCP).
SIX is recruiting a data platform engineer to work on the company wide data platform. In this role, you will be part of a cross-functional team responsible for both designing and implementing a reliable platform for analytics and data science, as well as consulting internal customers who use the platform across the organization.
What You Will Do
Be part of an international team that designs and maintains a mature cloud native data science platform, currently serving more than 14 internal analytics teams and 600+ users.
Closely collaborate with key stakeholders from business data science teams and IT data engineering teams to enable them on the platform. Identify new requirements, analyze platform suitability, propose best practices and existing solutions through demos...
Act as a hands-on data engineer to onboard new use cases and optimize existing ones, build production ready ETL pipelines, ML/AI use cases and reporting solutions.
Implement new infrastructure components with Infrastructure as code (IaC), as well as creating templates to ensure best practices and standardization among analytics teams.
What You Bring
4+ years of practical experience building production data pipelines, data modeling and BI tools in the cloud.
Experience with Lakehouse architecture, preferably Databricks, with a strong knowledge of Spark and Open Table formats (Delta and/or Iceberg).
Proficiency with Python (PySpark) and SQL with a solid understanding of software development and automation concepts like CI/CD to build production data and ML pipelines (DataOps/MLOps).
Basic knowledge of cloud foundation services (IAM, networking, storage...), cloud costs management (FinOps) as well as tools to manage cloud infrastructure with IaC (Terraform/OpenTofu, Pulumi, bash scripting...) on Microsoft Azure (our primary cloud service provider) and/or Amazon Web Services, Google Cloud Platform
Growth mindset and a very proactive attitude for continuous learning and development.
Good presentation and communication skills. Fluency in written and spoken English.
TJX · Hyderabad, TS 500081