Data Engineer

Position Description:
As a Data Engineer, you will play a key role in designing and maintaining scalable data pipelines and platforms.
You’ll work closely with data scientists, analysts, and product teams to ensure high-quality, reliable, and scalable data is always available. This role requires deep technical expertise, strong analytical skills, and exposure to emerging AI/LLM technologies
Location(s): Germany & EU
Working Model: remote

Role Overview
As a Senior Data Engineer, you will play a key role in designing and maintaining scalable data pipelines and platforms. You’ll work closely with
data scientists, analysts, and product teams to ensure high-quality, reliable, and scalable data is always available. This role requires deep
technical expertise, strong analytical skills, and exposure to emerging AI/LLM technologies.

Key Responsibilities
Design, develop, and optimize scalable data pipelines using Python and PySpark.
Build, manage, and optimize big data solutions leveraging Google Cloud Platform (BigQuery, DataProc, Dataflow).
Collaborate with cross-functional teams to translate business requirements into efficient data models and workflows.
Ensure data accuracy, consistency, and availability across multiple systems and applications.
Implement best practices for data governance, quality, and security.
Stay updated with advancements in big data, cloud, and AI technologies, recommending innovative solutions.
Mentor junior engineers and establish best practices within the team.

Requirements for "Professional Skills"
All professional skills required to fulfill the following general task description:
Works as a developer and is responsible for the development and programming of software, tools, systems, etc., as well as the associated troubleshooting and error analysis. This role requires comprehensive knowledge of various programming languages as well as appropriate development tools, methodologies, and development processes.

Works on the integration of systems, applications, or databases with one another or into other systems/software. This role requires exceptional knowledge of scripting and various programming languages, along with relevant integration tools, methods, and processes. It also requires a strong understanding of the technologies or systems being integrated.

Also works as an engineer and documents operational requirements in the form of system performance parameters and system configurations. He/she is additionally responsible for ensuring the seamless integration of architecture and development.

This also includes system integration or network integration activities, for example, SDM.

Projects using Agile development methodology:
If the contractor works within a Scrum team, they possess extensive knowledge of Scrum methodology and have experience in developing software within such a framework.

Requirements for "Social Skills"
All social skills required to fulfill the previously mentioned general task description as well as the following requirements:
The employee has intermediate-level experience, typically between 2 and 5 years, in their role or in the tasks performed by the group. Alternatively, the role and responsibilities require solid, but not exceptional, expertise.
The activities are associated with a moderate level of complexity and responsibility and require the ability to work independently.
The project may be carried out at either a national or international level.

Other Requirements & Relativizations
Skill Sets
Core Technical Skills
Python (expert-level coding, modular programming, debugging, performance optimization)
PySpark (distributed processing, tuning, handling large-scale data, performance optimization)
Google Cloud Platform (GCP):
BigQuery – advanced SQL, optimization, partitioning, clustering, cost management
DataProc – Spark/Hadoop cluster management, job orchestration
Dataflow – streaming & batch pipelines, real-time processing

Big Data & Cloud Skills
Strong understanding of big data technologies (distributed computing, Hadoop/Spark ecosystem, data lakes, Kafka, batch & stream processing)
Experience with data modeling and designing scalable data pipelines
Familiarity with cloud-native architectures and serverless processing

Analytical & Problem-Solving Skills:
Ability to analyze complex data sets and derive insights
Strong logical and critical thinking
Experience optimizing query performance and pipeline efficiency
Comfortable troubleshooting large-scale data workflows
Preferred / Nice-to-Have Skills
Experience with CI/CD pipelines for data engineering workflows
Workflow orchestration: Airflow, Cloud Composer, Dagster
Data governance, data quality frameworks, and compliance standards
AI & Advanced Analytics Exposure
Familiarity with Large Language Models (LLMs) and their applications in data workflows
General understanding of AI/ML concepts and how they integrate with data engineering pipelines
Exposure to MLOps tools or frameworks for deploying and monitoring AI models

Soft Skills
Strong communication and collaboration with cross-functional teams
Ability to work in Agile environments
Mentoring and guiding junior engineers
Self-driven, proactive learner, and adaptable to emerging technologies