Data Engineer

WSP India is currently seeking Data Engineer for Australia Mining team. They shall be responsible for supporting our AU business through GCC. This role is responsible for designing, building, and maintaining the data engineering layer of our Microsoft Fabric platform. The Data Engineer will own the end-to-end data pipeline lifecycle — from ingesting raw data out of engineering and construction tools through to delivering clean, modelled, AI-ready datasets that power reporting, dashboards, and decision-making across the business.

This is a greenfield opportunity — the successful candidate will be instrumental in shaping the platform from the ground up, making foundational decisions on standards, and patterns that will scale across the organisation. We are looking for someone who takes genuine ownership, thrives in ambiguity, and is energised by the challenge of building something from scratch rather than maintaining an existing setup.

• Build and maintain Data Factory pipelines for API-based data ingestion (REST, OData).

• Develop Python ETL scripts for custom extraction, API polling, and file ingestion.

• Develop and maintain Spark/Python notebooks (Python/PySpark) and SQL-based transformation logic.

• Create Dataflows Gen2 (Power Query) for handling Excel/CSV-based messy data.

• Implement incremental loads, watermarking, and CDC patterns.

• Integrate data from Dataverse and other enterprise sources into Fabric.

• Own Bronze → Silver transformations (cleansing, deduplication, standardisation)

• Support Silver → Gold transformations (star schema, fact/dimension modelling)

• Prepare feature datasets for AI/ML workloads and support notebook-based model development

• Implement data quality frameworks (validation rules, anomaly detection)

• Build and optimise Power BI semantic models (Direct Lake)

• Prepare datasets for reporting and basic analytics use cases.

• Monitor pipeline performance, platform health, and Fabric capacity (F64+), including cost optimisation.

• Maintain documentation (data dictionaries, pipeline inventory, architecture decisions)

• Manage Fabric capacity — pause/resume scheduling, scaling, and cost optimisation

Key Competencies / Skills:

• Microsoft Fabric - Hands-on with Lakehouse, Warehouse, Data Factory, Dataflows Gen2, Notebooks, OneLake.

• Python/SQL - Strong experience in ETL scripting, data engineering, transformation logic and notebook development.

• Working knowledge of file formats, partitioning strategies, ACID transactions, schema evolution

• Power BI - Semantic Model Creation, DAX fundamentals, Direct Lake mode configuration.

• Star Schema Design – Fact/dimension modelling, slowly changing dimensions (SCD), aggregation patterns.

• Data Factory / ETL — Pipeline design, REST/OData connectors, incremental loads, error handling.

• Git / CI/CD — Version control practices, Fabric deployment pipelines, Azure DevOps integration.

• Must be able to prioritize, multitask as well as work independently.

• Must be able to maintain quality of work and at the same time meet multiple deadlines.

• Excellent interpersonal skills cum responsiveness to others

• Strong technical skills which demonstrate the ability to implement and maintain an enterprise level reporting solution.

Desirable:

• Experience with Microsoft Dataverse and Power Apps integration

• Familiarity with Microsoft Purview for governance and cataloguing

• Understanding of the EPCM project lifecycle — feasibility through construction and commissioning

• Experience with Agile / iterative delivery in small, cross-functional teams

• Experience working in enterprise environments, particularly in the engineering and construction sector

• Exposure to project controls data — schedules (P6/MSP), cost systems, document management, EVM.

• Must have minimum 7-10 years of experience with

• Microsoft Fabric certification – DP 600, DP700s would be advantageous.

BGV:

• Employment with WSP India is subject to the successful completion of a background verification (“BGV”) check conducted by a third-party agency appointed by WSP India.

• Candidates are advised to ensure that all information provided during the recruitment process — including documents uploaded — is accurate and complete, both to WSP India and its BGV partner”.