Senior Data Engineer

Who We Are

iSelect and Compare the Market are trusted, well-known brands dedicated to helping millions of Australians compare and buy personal finance and household products such as insurance, energy, and loans. Together, we provide a seamless, customer-focused experience that makes it easier for Australians to find the right products to suit their needs.

What You’ll Be Doing

Reporting to the Data Engineering Manager, the Senior Data Engineer designs, builds, and evolves the organisation’s cloud data platform, delivering trusted, scalable, and secure data solutions that support business intelligence, advanced analytics, machine learning, and AI initiatives.

The role develops and optimises cloud-based data pipelines, products, and platform services, ensuring data is accessible, governed, and fit for reporting, analytics, predictive modelling, and Generative AI use cases.

Working closely with stakeholders across the business and technology teams, the Senior Data Engineer helps transform data into strategic assets that drive decision-making and innovation, while supporting AI-enabled solutions through strong data foundations and governance.

As a senior technical leader, the role provides expertise across data architecture, engineering standards, platform optimisation, automation, and DataOps/MLOps practices, contributing to the ongoing maturity of the organisation’s data and AI capabilities.

• Design, develop, and support scalable cloud-based data warehouse, lakehouse, and AI-enabled data platform solutions that support enterprise analytics, reporting and future ML/AI initiatives


• Build, optimise, and maintain batch, streaming, and near real-time data pipelines and data products to deliver trusted, high-quality, and governed data for business, analytics, and up-coming AI workloads


• Design, develop and deploy data workflows as an individual contributor with support of intelligent automation solutions which leverages AI agents, agentic workflows, LLMs, orchestration frameworks, and modern AI platform capabilities to improve operational efficiency and business outcomes


• Support advanced analytics and AI initiatives through the delivery of AI-ready datasets, vector search capabilities, RAG architectures, and secure, scalable data solutions that enable model development and operationalisation


• Implement and champion modern engineering practices including DataOps, SecOps, CI/CD, IaC automated testing, observability, platform automation, and responsible data access controls


• Ensure compliance with data governance, security, privacy, and quality standards, while collaborating with business and technology stakeholders to continuously improve platform capability, innovation, scalability, performance, and cost efficiency

What’s In It For You

At Compare the Market and iSelect, we’re all about helping people make smart choices – and joining us might just be yours.We offer an environment where your ideas are heard, your contributions are valued, and your growth genuinely matters. Collaboration is part of our DNA and you’ll be joining a supportive, forward-thinking team where you can do meaningful work (and be recognised for it!)

We’re guided by three shared values that shape how we work, how we treat each other, and how we show up for our customers - Drive, Noble Intent and All Together.

Plus, we offer a range of great perks to help you thrive both in and out of work:

• Vibrant and social community who have annual celebrations, family fun days and regular events


• Enjoy flexible work arrangements, including the option of one day working from home each week or a 9-day fortnight


• Enjoy additional leave days - ‘ME’ leave and ‘Volunteer Day’ leave


• Free coffee on Mondays


• Option to purchase additional leave


• Return To Work bonus for returning parents


• Quarterly Award and Recognition program

What Sets You Apart

Qualifications

• Minimum a bachelor’s degree qualified in Computer Science, Analytics, IT or a relevant discipline

Experience & Skills

• 7+ years' experience in Data Engineering, Analytics, Database Administration, Software Engineering, or Business Intelligence environments, with a proven track record delivering enterprise-scale data solutions


• 5+ years' experience designing, building, and supporting cloud-based data platforms, data warehouses, lakehouses, and data products across AWS, Azure, or GCP environments


• Strong expertise in modern data architectures and platforms, including Data Warehouses, Data Lakes, Lakehouses, Data Mesh, Digital Twin, Medallion Architectures, and platforms such as Databricks, Snowflake, Fabric, Synapse, Redshift, or BigQuery


• Extensive experience developing batch, streaming, and near real-time data pipelines using Apache Spark, PySpark, SQL, Kafka, Kinesis, Event Hub, and cloud-native storage and distributed data platforms


• Advanced proficiency in Python, SQL, and PySpark, with experience in additional programming languages such as Scala, Java, R, TypeScript, or Julia


• Strong experience delivering DataOps and software engineering practices, including ETL/ELT frameworks, orchestration platforms, CI/CD, IaC, DevOps tooling, automated testing, observability, monitoring, and platform automation


• Proven expertise in data modelling, governance, and data management, including dimensional modelling, Data Vault, metadata management, lineage, data quality, cataloguing, master data, access controls, and audit requirements


• Experience developing reusable frameworks, APIs, event-driven architectures, integration services, and metadata-driven automation solutions that improve scalability, maintainability, and operational efficiency


• Experience supporting current and future advanced analytics, ML and Gen AI initiatives, including AI-ready datasets, MLOps, AI platform integration, vector search, RAG, semantic search, embeddings, knowledge retrieval, and intelligent automation solutions


• Demonstrated technical leadership and stakeholder engagement, including mentoring engineers, contributing to solution architecture, establishing engineering standards, integrating enterprise and SaaS platforms, and applying security, privacy, compliance, and responsible data governance principles

At Compare the Market and iSelect we’re proud to be an Equal Opportunity Employer and we embrace diversity.

This role is available to candidates with unlimited Australian working rights only. Sponsorship is not available.

iSelect and Compare the Market, brands of iCompare Holdings, are committed to protecting your privacy. All applications and personal information are kept confidential and securely stored in line with Australian privacy laws, used only for recruitment purposes, and not shared with third parties outside iCompare Holdings without your consent.

At Compare the Market and iSelect we’re proud to be an Equal Opportunity Employer and we embrace diversity.

This role is available to candidates with unlimited Australian working rights only. Sponsorship is not available.

iSelect and Compare the Market, brands of iCompare Group, are committed to protecting your privacy. All applications and personal information are kept confidential and securely stored in line with Australian privacy laws, used only for recruitment purposes, and not shared with third parties outside iCompare Group without your consent.