DATA ENGINEER
Kraków (hybrid: 6 days per month in the office)
B2B Contract: 195 - 215 PLN / hour
We are looking for a Data Engineer to join an international project delivered for a large organization in the financial services sector. The team is responsible for building and maintaining scalable data pipelines, supporting analytical and business teams, and ensuring high-quality data for strategic decision-making.
This role involves working with modern big data technologies and distributed systems, contributing to data engineering best practices, and collaborating closely with analysts and engineers. The scope of responsibilities will be adjusted based on the candidate’s experience and technical profile.
Key Responsibilities:
- Collaborate with Engineers, Data Analysts, and Business Analysts to design, develop, and operate data pipelines and systems.
- Implement tools and processes to handle performance, scale, availability, accuracy, and monitoring of data platforms.
- Build and maintain ETL processes, using Hadoop, Spark, Hive, and other big data components.
- Ensure requirements are correctly interpreted and implemented by liaising with BAs.
- Promote development standards through code reviews, mentoring, and knowledge sharing.
- Participate in Agile ceremonies, including sprint planning, reviews, and retrospectives.
- Provide production support, troubleshoot issues, and contribute to improving system reliability.
- Contribute to system architecture and design decisions.
Requirements (Must Have):
- 5+ years of experience in software design and development, preferably in Pyspark or Scala.
- Hands-on experience building data pipelines using Hadoop ecosystem components (Hive, Spark, Spark SQL).
- Experience with scheduling tools such as Airflow.
- Strong knowledge of Unix/Linux platforms.
- Experience with big data frameworks: Apache Hadoop, Apache Spark, YARN, Hive, Python, ETL frameworks, MapReduce, SQL, RESTful services.
- Familiarity with version control (Git/GitHub), automated deployment tools (Ansible, Jenkins), and requirement management (Jira).
- Strong analytical and debugging skills.
Nice to Have:
- Experience with Elasticsearch and Java API development.
- Knowledge of cloud platforms and cloud design patterns.
- Experience with data ingestion processes.
- Exposure to DevOps and Agile methodologies (Scrum, Kanban).