PURPOSE OF THE POSITION
The Data Engineer is responsible for the maintenance and the improvement of all the data flows in the data analytics platform(s). The Data Engineer designs and develops scalable ELT packages and routines from the source systems in order to create information (datasets), defining and building the data pipelines.
The Data Engineer additionally analyzes complex data models and IT Applications, data flows, dependencies and relationships in order to contribute to conceptual physical and logical data models.
Moreover, the Data Engineer acts as architect of the Data & Analytics platform defining the logical data model and the physical data model(s). He/She also keeps up with industry trends and best practices, advising the management on new and improved data engineering strategies.
Main responsibilities
- Define the optimal architecture of the Data & Analytics platform enabling the availability and re-usability of all required data for the creation of analytics products
- Manage and monitor data pipelines to source and transform data to the Data & Analytics Platform(s)
- Integrate and transform new data sources (internal and external) by creating a full pipeline from ingestion to ETL process
- Deliver Data Provisioning to operational applications requiring transformation of data from several sources
- Optimize and expand the Data & Analytics platform data flows and data models, supporting in deploying analytics products
- Define and document detailed Data Engineering processes to guarantee efficient building and support of the data & analytics platform
- Assess the stability, robustness and efficiency of the implemented ELT processes and data pipelines and eventually re-design them
- Gather requirements and write technical specifications document
- Ensure and monitor high data quality standards with focus on data consistency
- Timely resolve incidents related to data interfaces and ELT processes
- Expand the current Data & Analytics platform to embed “big data” capability by investigating and sharing best practices
- Master’s degree in IT / Economics
- 5-6 years of data engineer experience
- Fluent English
- Experience with data modelling (conceptual, logical, physical) and data modelling documentation
- Professional experience and conceptual knowledge of building and maintaining physical and logical data models, including data streaming
- Ability to think beyond technology requirements to build ONE logical Data & Analytics platform
- Excellent knowledge and proven professional experience with MS Azure and Lakehouse architecture, in-depth knowledge and/or certification in Databricks (Spark)
- Well structured, analytically thinking and demonstrating the ability to explain processes in a clear and understandable manner to a non-technical audience
- System management expertise with monitoring, disaster recovery, backup, automated testing, automated schema migration, and continuous deployment
- Ability to take ownership and proven strive for excellence