Data Engineer (Cloud Data Pipelines & Data Quality)

Willing to start a new freelance challange? Get in touch.

What You'll Do

• Design, build and optimise cloud-based data pipelines using SQL, Python/PySpark and Airflow or a similar orchestration platform.
• Implement automated data-quality checks covering completeness, thresholds, referential integrity and business rules.
• Develop reliable ingestion patterns, including idempotent and merge-based processing.
• Investigate failed pipelines, data-quality issues and production incidents, identifying and implementing effective fixes.
• Improve pipeline performance, schemas and processing logic to reduce unnecessary compute and improve maintainability.
• Define and implement data contracts where appropriate to improve reliability between data producers and consumers.
• Collaborate with Data Engineers and business/technical stakeholders to understand requirements and deliver robust data solutions.
• Produce clear technical documentation and contribute to knowledge-transfer sessions to ensure solutions remain maintainable after handover.

What you bring

• 3+ years of professional experience in Data Engineering, with hands-on experience building cloud data pipelines.
• Strong SQL skills and professional experience with Python and/or PySpark.
• Practical experience with Apache Airflow or a comparable workflow orchestration tool.
• Experience implementing idempotent, incremental or merge-based data ingestion patterns.
• Understanding of automated data-quality testing, ideally with tools such as Great Expectations, Soda or equivalent.
• Experience troubleshooting pipeline failures and improving data-processing performance.
• A strong engineering mindset around reliability, documentation, testing and maintainability.
• Comfortable collaborating with other Data Engineers and sharing knowledge.

Interested? Apply today!

myNEBIRU: Not into this role, but interested in what NEBIRU does? That’s totally fine.

Visit https://nebiru.be/mynebiru to see how we can support you - even outside our client missions. Let’s build the bridge to your next step, together.