Senior Data Engineer

We are partnering with a growing technology organisation to appoint a Senior Data Engineer to help build and scale its data platform.

This is a hands-on role with ownership across data ingestion, processing, storage, modelling and governance. You will build reliable data infrastructure used across multiple technical and business applications, while helping shape engineering standards as the platform evolves.

Key Responsibilities

• Design, build and operate scalable batch and streaming data pipelines.
• Develop data architecture across ingestion, processing, storage and transformation layers.
• Build reliable, reusable datasets and data models for downstream applications and teams.
• Implement data quality, validation, monitoring, lineage and governance.
• Design for reliability through idempotency, deduplication, schema evolution, recovery and historical accuracy.
• Implement appropriate access controls and protection for sensitive data.
• Work closely with engineering and infrastructure teams to operate production data services.
• Optimise performance, scalability and infrastructure costs.
• Contribute to technical standards, documentation and code reviews.

What We're Looking For

• 8+ years of professional data engineering experience.
• Strong Python and SQL skills, including PySpark or comparable distributed processing technologies.
• Strong production experience with AWS data services, particularly S3, Glue, Glue Data Catalog, Athena and Redshift .
• Experience designing and operating data lake or lakehouse architectures , including layered data transformation.
• Experience with ETL/ELT and integration technologies such as Airbyte, Fivetran or Meltano .
• Experience with streaming, event-driven or change-data-capture technologies such as Kinesis, MSK, SQS, SNS, DMS or Debezium .
• Experience with dbt or comparable data transformation and modelling tools .
• Familiarity with Parquet and modern table formats such as Iceberg, Delta Lake or Hudi.
• Experience with workflow orchestration tools such as MWAA, Step Functions, Dagster or Prefect .
• Strong understanding of data quality, governance, security and lineage.

AI & Advanced Data

Experience building data infrastructure for AI or other advanced data

#J-18808-Ljbffr