Salary: $105 - $120 per hour
Data Engineer – Databricks / PySpark
Canberra | Hybrid | Baseline Clearance Required
$105–$120 per hour incl. super
We are seeking an experienced
Data Engineer to support the design, migration and operation of data solutions within a Federal Government environment.
This is a hands-on engineering role focused on
Databricks, PySpark, Spark SQL, Delta Lake and SAS-to-Databricks migration.
You will work within the Schools Data Production Team to build scalable data pipelines, migrate existing SAS processes and establish secure, maintainable and well-documented data engineering practices.
Key responsibilities- Design, build and maintain scalable Databricks data pipelines
- Migrate SAS code, workflows and scheduled processes to Databricks
- Develop solutions using PySpark, Spark SQL and advanced SQL
- Build ETL/ELT pipelines and medallion-layer data models
- Implement Delta Lake solutions
- Establish source control, deployment and testing practices
- Work with Azure DevOps, Git and CI/CD
- Implement automated testing, monitoring and performance optimisation
- Apply data security, privacy, metadata and governance controls
- Produce clear technical documentation and support knowledge transfer
Essential experience- Strong Databricks, PySpark, Spark SQL and SQL capability
- Demonstrated experience designing scalable Databricks solutions
- Experience building maintainable ETL/ELT pipelines
- Experience with Delta Lake and medallion architecture
- Strong experience with Azure DevOps, Git and CI/CD
- Experience with automated testing, monitoring and performance optimisation
- Ability to produce secure, maintainable and well-documented data solutions
Highly desirable / strongly preferred:- Demonstrated experience migrating SAS code, workflows and reporting processes to Databricks
EligibilityApplicants must:- Be able to work from the Canberra office under hybrid arrangements
- Hold a current Baseline security clearance or higher
ApplyRespond with your tailored CV. Suitable candidates will be asked to complete and application pack, including a One-page pitch for the role.
Candidates not meeting requirements abve, Citizenship and Security clearance requirements, will not be contacted.