data engineer for cloud-native data platforms

Описание

Sigma Software provides IT services and consulting. The data engineering team builds cloud-native data platforms, migrates legacy systems to the cloud, and develops AI-ready data infrastructure.

Задачи

• Design and build scalable, cloud-native data platforms from greenfield to production
• Implement near-real-time ingestion pipelines using event-driven patterns
• Define and enforce platform standards, including Data Lake / Lakehouse principles, medallion architecture, and data contracts
• Refactor and optimise existing Spark and PySpark scripts for performance and maintainability
• Introduce best practices for code quality, testing, and CI/CD across data pipelines
• Drive adoption of AI tooling and agentic workflows within the data engineering team
• Ensure data quality, observability, and reliability across all pipelines and platforms
• Develop self-service tooling and microservices to simplify platform usage for other teams
• Collaborate with Machine Learning, Data Science, and Product teams as a key technical contributor and thought leader
• Drive R&D efforts around agentic AI architectures, event-driven systems, and LLM-ready data pipelines

Требования

• 5+ Years of professional experience in Data Engineering
• Strong Python and SQL development skills for pipeline development and optimisation
• Proficiency in Apache Spark / PySpark, including query optimisation and performance tuning
• Hands-on experience with Databricks or Snowflake
• Experience with at least one major cloud provider: Azure, AWS, or GCP
• Experience with stream processing technologies such as Kafka and Spark Structured Streaming
• Solid understanding of ETL/ELT patterns, data modelling, including dimensional and Data Vault models, and data warehousing
• Experience with orchestration tools such as Apache Airflow, Azure Data Factory, or equivalent
• Knowledge of Infrastructure as Code, such as Terraform or equivalent
• Understanding of production-grade system requirements: reliability, scalability, observability, and performance
• Upper-Intermediate English level
• Self-driven and proactive in identifying improvements
• Comfortable working in a fast-paced, innovative environment
• Strong problem-solving mindset with attention to detail
• Open to experimenting with emerging technologies and approaches
• Nice to have: familiarity with RAG pipeline design and LLM integration patterns, knowledge of data governance frameworks and tools such as Unity Catalog and Apache Atlas, experience with dbt for data transformation and modelling, familiarity with MLflow, Feature Stores, or ML platform integration

Условия

Office location: Warsaw, Mazowieckie, Poland.