About Data Elephant
Data Elephant is a Canadian Data, Analytics & AI consulting firm helping organizations solve complex data, analytics, AI, and machine learning challenges. We combine business understanding with hands‑on technical delivery to build modern data platforms and turn data into useful outcomes.
The role
We are looking for a hands‑on Senior Data Engineer to build reliable data pipelines and transformation solutions across Microsoft Azure and Fabric. You will turn source data into trusted, usable datasets for reporting, analytics, AI, and machine learning. Working closely with architects, analysts, and clients, you will own engineering work from source analysis and development through testing, deployment, and production support.
What you will do
- Build and maintain ingestion and orchestration pipelines using Fabric Data Factory and Azure Data Factory, integrating databases, APIs, files, and enterprise applications.
- Develop ETL/ELT transformations using SQL, Python, PySpark, Fabric notebooks, and Dataflow Gen2, selecting the right approach for the workload.
- Implement OneLake, Lakehouse, Warehouse, and Delta Lake solutions, organizing raw, refined, and curated data using medallion architecture where appropriate.
- Develop full and incremental load patterns, change data capture where supported, parameterized pipelines, and reusable ingestion frameworks.
- Build dimensional models and curated datasets with analysts and business stakeholders; implement business rules, historical tracking, and source-to-target mappings.
- Implement data quality checks, automated tests, reconciliation, and validation. Handle schema changes, duplicate records, missing values, and late‑arriving data.
- Make pipelines reliable through logging, monitoring, alerts, retries, dependency management, and safe reruns; troubleshoot failures and document operational runbooks.
- Use Git and CI/CD to promote changes across environments; optimize SQL, Spark, and pipeline performance, implement agreed access controls, and participate in code reviews and mentoring.
What you bring
- Strong experience building and operating production data pipelines, with ownership of complex engineering work through deployment and support.
- Hands‑on delivery experience with Microsoft Fabric, including OneLake, Lakehouse or Warehouse, Fabric Data Factory pipelines, and notebooks.
- Strong SQL, Python, and PySpark skills, with practical experience processing data in Apache Spark and Delta Lake.
- Practical Azure experience with Azure Data Factory, ADLS Gen2, and Azure SQL, including integrating Azure services with Fabric.
- Solid understanding of ETL/ELT, orchestration, incremental processing, data modeling, and structured and semi‑structured data.
- Experience with Git, deployment automation, testing, data quality, monitoring, and performance tuning in production environments.
- Ability to work directly with clients to clarify requirements, explain issues, and manage dependencies; confidence working independently and collaborating across delivery teams.
Additional experience we value
- Dataflow Gen2 and Power Query, OneLake shortcuts or mirroring, and Power BI semantic model integration.
- Azure Databricks, Azure Synapse, Microsoft Purview, or ERP and SaaS integration experience.
- Streaming workloads, Fabric Real‑Time Intelligence, KQL, or Eventstreams.
- Microsoft Certified Fabric Data Engineer Associate or other relevant Azure or Microsoft data certifications.
How you work
- You are naturally curious, stay current with emerging technologies, and enjoy learning from colleagues and clients.
- You take ownership, ask thoughtful questions, and bring practical ideas to fast‑paced, changing environments.
- You collaborate without ego, welcome feedback, and share knowledge freely. You care about the quality of your work and the success of the team.
- You communicate clearly with technical and business audiences, explain trade‑offs, and build trust with clients.
Why join Data Elephant
At Data Elephant, you will work directly with clients on meaningful challenges, contribute to how we deliver, and learn alongside experienced colleagues. You will have opportunities to take ownership, deepen your expertise, and help shape a growing consulting practice.