Senior Data Engineer Apache Hadoop, Apache Spark, SQL, Kafka Gdańsk

As a recruitment company, DCG understands that every business is powered by experienced professionals. Our management style and partnership approach enable us to meet your needs and provide continuous support. Due to our ongoing growth and the large number of recruitment projects we undertake for our partners, we are currently looking for:

Responsibilities

• Develop and maintain data pipelines supporting regulatory reporting for the initiative
• Design and implement distributed data processing solutions using Hadoop and Spark
• Translate complex business requirements into efficient and executable SQL transformations
• Build and maintain SQL‑based data transformations for regulatory data processing
• Manage and monitor workflow orchestration using Oozie and Airflow
• Work with event‑driven architectures, including Kafka
• Provide L3 support for production systems, including incident analysis and resolution
• Collaborate with Production Assurance teams to ensure system stability
• Communicate technical issues and SQL logic clearly to business stakeholders
• Contribute to CI/CD processes, including Git workflows, pull requests, and Jenkins pipelines
• Ensure high data quality, reliability, and compliance with regulatory requirements

Requirements

• Strong experience in data engineering and distributed data processing
• Strong knowledge of SQL‑based data transformation
• Hands‑on experience with Apache Hadoop, Apache Spark, Oozie, Apache Airflow, Kafka / event‑driven architectures and CI/CD tools (Git, PR workflows, Jenkins)
• Ability to translate business requirements into SQL logic
• Experience providing L3 production support
• Ability to collaborate with Production Assurance teams
• Strong analytical and problem‑solving skills
• Strong communication skills, including ability to explain SQL‑related issues to business stakeholders
• Very good command of English, both spoken and written

Nice to Have

• Experience with containerisation and orchestration (Kubernetes)
• Experience with Apache Flink

Offer

• Private medical care
• Co‑financing for the sports card
• Constant support of dedicated consultant