Data Engineer Consultant

Working as a Data Engineer Consultant, you will:

Design and implement data ingestion pipelines connecting external APIs and business systems with the Databricks data platform.

Develop reliable processes for retrieving and processing historical data, including mechanisms for rate limiting, checkpointing, retries and safe reprocessing.

Build and maintain the Bronze layer of the data platform, covering data standardisation, typing, historical tracking and document storage.

Develop processes for linking digital documents with corresponding business records while ensuring data consistency and idempotent processing.

Use Databricks capabilities to classify documents and extract structured information from PDFs and images, including confidence scoring and workflows for manual review.

Contribute to the development of the Silver layer using appropriate entity modelling, CDC and incremental processing patterns.

Implement deterministic entity resolution and source-priority logic across multiple data sources.

Apply data quality controls within the existing framework and ensure that quality results are properly stored, monitored and available for governance purposes.

Support the development of analytical and machine-learning data products, including feature and metric stores and batch scoring workflows.

Work with MLflow and related processes for model registration and promotion where required.

Improve automation, monitoring and reliability across data pipelines and contribute to CI/CD processes.

Collaborate with other technical stakeholders to deliver agreed project milestones within a defined consulting scope.

Chabre IT Services is a global professional IT services provider, building long-lasting relationships with Enterprises. We specialize in the delivery of tailor‑made solutions, smart outsourcing, try&hire, and success fee services. We are a smart IT boutique with unique knowledge, which will deliver your ideas into reality.

About our Client

Our client is a global technology services organisation delivering data and digital transformation projects for enterprise customers. The project focuses on building and enhancing a modern data platform based on Databricks, with a strong emphasis on data engineering, governance, automation and analytics.

The successful candidate will join a project involving multiple data sources, API integrations, document processing and layered data architecture. The environment combines traditional data engineering with modern Databricks AI capabilities and machine‑learning workflows, providing an opportunity to work across several stages of the data lifecycle.

Qualifications

• Recent, hands‑on experience delivering production solutions with Databricks.
• Strong knowledge of PySpark, Delta Lake and Unity Catalog.
• Practical experience with the Databricks medallion architecture and production data pipelines.
• Experience with Spark Declarative Pipelines / Delta Live Tables and Databricks Workflows.
• Very good Python and advanced SQL skills.
• Experience integrating REST APIs into data engineering solutions, including authentication, pagination, rate limiting and incremental or checkpointed data retrieval.
• Strong understanding of data modelling, including SCD Type 2 and current‑state / CDC patterns.
• Experience designing data quality controls, reconciliation processes and mechanisms for handling invalid or quarantined data.
• Knowledge of Git‑based development and CI/CD for data platforms, preferably with Azure DevOps or GitHub Actions.
• Experience implementing monitoring and observability for production data pipelines.
• Ability to work within a fixed project scope, estimate work, meet milestones and take ownership of agreed deliverables.
• Strong communication skills and the ability to work independently in a consulting environment.
• Excellent English and Polish skills (min. B2 level).

We offer

• Rate up to 240,00 PLN/h + VAT
• Subsidy for peripherals in the amount of 500,00zł
• Working tool (MacBook Pro or Lenovo Legion 5)
• Co‑financing of courses related to the position