The Data Quality Engineer / Platform Specialist will play a hands-on technical role in ensuring that data within the Enterprise Data Platform (EDP) is accurate, reliable, discoverable, and fit for business use. Focusing on our GCP BigQuery ecosystem, you will design, implement, and automate data quality checks, manage platform health, and troubleshoot data pipelines. While experience is valued, we are highly focused on finding someone with strong SQL capabilities, an analytical mindset, and a deep understanding of modern, cloud-native data platforms.
Key Responsibilities
1. Technical Data Quality & Automation
- Develop, deploy, and maintain automated data quality rules and tests natively within GCP (e.g., using Dataform or BigQuery stored procedures) for critical data assets.
- Design and build monitoring, alerting, and scorecard dashboards to proactively detect data anomalies, schema drifts, and quality deterioration before they impact downstream reporting.
- Perform deep-dive root cause analysis using advanced SQL to trace data incidents back to their source, coordinating technical fixes with data engineers and system owners.
2. GCP Platform Operations & Management
- Assist in managing the GCP BigQuery data platform environment, ensuring efficient query performance, cost management, and reliable data transformation pipelines.
- Implement platform governance guardrails for data onboarding, transformation, and exception handling.
- Utilise native GCP tools (such as Cloud Composer, Knowledge Catalogue, or Cloud Functions) to orchestrate quality checks and remediate data pipeline failures.
- While GCP-native skills are the priority, experience with comparable enterprise data quality tools or data platforms like Databricks, AWS, Azure, IDMC will be considered as well.
3. Data Quality by Design (DQxD) Implementation
- Act as the technical execution arm for CAG’s Data Quality by Design framework across new digital and analytics initiatives.
- Partner with project teams to review data models and integrations, ensuring data quality tests are coded and implemented in the data pipeline before data is consumed.
4. Technical Metadata & Cataloging
- Utilise tools like GCP Knowledge Catalog to maintain technical metadata, ensuring the Data Catalogue, Data Dictionary, and Data Lineage remain highly accurate and discoverable.
- Automate the extraction and updating of metadata from source systems into the centralised catalog.
Requirements
Education & Experience
- Degree in Computer Science, Information Systems, Data Analytics, Engineering, Statistics, or a related highly quantitative discipline.
- 0–3 years of experience.
- Candidates with strong academic projects, relevant internships, or a demonstrable portfolio in data engineering, data quality, or cloud platforms are highly encouraged to apply.
Core Technical Skillsets
Advanced SQL & Query Optimisation
- Complex Querying: Strong proficiency in writing advanced SQL (including Common Table Expressions (CTEs), window functions, complex joins, and handling JSON/nested data structures).
- Analytical Troubleshooting: Ability to use SQL for deep-dive root cause analysis and data profiling to isolate pipeline failures and data anomalies.
GCP & BigQuery Native Ecosystem
- Platform Architecture: Hands-on experience or strong conceptual understanding of Google BigQuery architecture and navigating enterprise-scale data environments.
- Native Governance Tools: Familiarity with GCP-native metadata management and governance tools, specifically Knowledge Catalog, for automating data quality checks, data cataloguing, and mapping data lineage.
- Cloud Infrastructure: Basic understanding of the broader GCP ecosystem, including Cloud Storage (GCS), Identity and Access Management (IAM) for security guardrails, and Cloud Functions/PubSub for event-driven alerting.
Data Quality & Observability Engineering
- Quality Frameworks: Strong technical grasp of core data quality management dimensions (accuracy, completeness, consistency, validity, and timeliness) and how to translate them into executable code.
- Monitoring & Remediation: Experience designing automated monitoring routines, anomaly detection, and issue remediation workflows.
Data Pipelines, Modelling & Orchestration (ETL/ELT)
- Data Modelling: Solid understanding of modern data warehousing concepts, dimensional data modelling, and scalable ELT architectures.
- Transformation & Orchestration: Familiarity with SQL-first transformation frameworks like Dataform, and an understanding of orchestration concepts using tools like Google Cloud Composer.
- DevOps for Data (CI/CD): Understanding of version control (Git) and the basics of deploying data pipelines and quality tests through CI/CD pipelines.
Programming & Automation
- Python Proficiency: Intermediate skills in Python for programmatic data manipulation, scripting, and automating operational tasks.
BI, Analytics & Visualisation
- Dashboard Development: Familiarity with BI / analytics platform concepts to support downstream users.
- Quality Scorecards: Ability to build operational scorecards and data observability dashboards using visualisation tools such as Tableau, or Power BI to report on data health.
Core Competencies
- Strong analytical and problem-solving skills with a highly structured, code-first approach to root cause analysis.
- High attention to technical detail and the ability to translate business requirements into hard-coded data quality rules and monitoring mechanisms.
- Clear communication skills, specifically the ability to explain complex pipeline failures or data quality risks to non-technical stakeholders.
This is a 3-yr contract role.