Contract: September 1, 2026 – September 1, 2027
We are seeking a Senior DataOps / Cloud Data Engineer to support a large public‑sector data modernization initiative in Toronto. The successful candidate will design, develop, optimize, and support cloud‑based data pipelines and lakehouse solutions using Azure Data Factory, Databricks, Informatica, Python, and SQL.
This is a highly hands-on role focused on data integration, ETL/ELT development, Medallion Architecture, data migration, pipeline orchestration, automation, and DataOps.
Key Responsibilities
- Design, develop, and optimize Azure Data Factory and Databricks pipelines from Oracle and other on-premises data sources to cloud Lakehouse environments.
- Develop and implement solutions using Databricks Medallion Architecture and Delta Lake.
- Design and optimize relational, dimensional, and enterprise data models.
- Translate existing Informatica ETL processes into Azure Data Factory and Databricks ELT solutions.
- Develop data pipelines, workflows, jobs, scripts, and automation using Python, SQL, T-SQL, PL/SQL, ADF, Informatica, SSIS, and/or Microsoft Fabric.
- Implement data orchestration, parallel processing, data movement, and pipeline automation.
- Support large-scale data migration from OLTP/OLAP environments to cloud-based platforms.
- Implement data quality, validation, profiling, cleansing, monitoring, and data governance practices.
- Develop and maintain data lineage and documentation to provide traceability of data movement and transformations.
- Support CI/CD, deployment automation, and data provisioning using tools such as Azure DevOps.
- Troubleshoot production data pipeline issues and implement corrective solutions.
- Work with technical teams, business stakeholders, and project teams to support solution design and integration.
- Create technical and solution documentation, including data mappings, models, requirements, and design documentation.
- Provide technical guidance and knowledge transfer to internal team members.
Required Skills & Experience
- Strong experience as a Senior Data Engineer, Cloud Data Engineer, or DataOps Engineer.
- Hands-on experience with Databricks, Azure Data Factory, and Informatica.
- Strong development experience with Python and SQL, including T-SQL and/or PL/SQL.
- Experience designing and implementing Databricks Medallion Architecture.
- Strong knowledge of Delta Lake and Lakehouse architecture.
- Experience developing and managing enterprise data pipelines and workflows.
- Experience with ETL/ELT development, orchestration, automation, and deployment.
- Experience integrating cloud platforms with on-premises databases and data sources, particularly Oracle.
- Experience with data warehousing, data lakes, lakehouses, dimensional modelling, fact/dimension models, and star schemas.
- Experience with CI/CD and Azure DevOps.
- Experience working with large and complex datasets and high-volume data environments.
- Experience with data quality, data governance, data lineage, and data security practices.
- Strong troubleshooting, analytical, problem-solving, and communication skills.
- Experience working in Agile and/or Waterfall environments.
AI-enabled tools may be used to sort applications based on job-related criteria. All AI generated results are vetted by our team and the decision of which candidates move forward is always made by a human.