Senior Data Engineer

Job Purpose

This Senior Data Engineer will play a key role in the evolution of our data platform, driving innovation and delivering high-quality data solutions. This role will focus on both migration from our current Data Warehouse to an AWS ecosystem, and developing cutting edge real-time data solutions including automation, business intelligence, and enabling analytics.

Job Duties

• Design, develop, and maintain data pipelines and ETL processes using AWS services such as AWS Glue, AWS Lambda and AWS S3.
• Support the migration of the existing Data Warehouse from SQL Server to AWS through S3 and Redshift.
• Develop and implement data quality checks and validation procedures.
• Design and implement best practice data lakehouse architectures and data warehousing solutions.
• Collaborate with data scientists and analysts to support the deployment of machine learning and advanced analytical solutions.
• Develop and maintain data documentation and operational procedures.
• Investigate and resolve data quality issues and performance bottlenecks.
• Stay abreast of the latest data technologies and industry best practices.
• Mentor junior data engineers and provide technical guidance to other team members where applicable.
• Contribute to the development and improvement of data platform best practices.

Knowledge, Skills and Experience

• Good understanding of AWS services such as AWS Glue, AWS Lambda, AWS S3, AWS Redshift, and Amazon EMR.
• Proficiency in Python, SQL, Pipeline Orchestration, and data warehousing concepts. Ability to diagnose and resolve complex data issues.
• Ability to effectively communicate with both technical and non-technical stakeholders.
• Experience with data lakehouse architectures and data warehousing solutions.
• Experience in migrating data from on-premises systems to cloud environments would be desirable.
• Experience with Agile development methodologies.
• Experience with data security and privacy best practices.

What is in it for me?

• A competitive rate of pay and pension contribution (£55,000 - £80,000)
• Generous discretionary bonus schemes, incentives and competitions
• An annual leave entitlement that increases with length of service
• Access to an online GP 24/7, 365 days a year for you and your immediate family.
• Employee wellbeing support through our Employee Assistance Programme
• Enhanced Maternity & Paternity Pay
• Long Service Recognition
• Access to a pay day savings scheme, financial coach and up to 40% of your earned wage ahead of payday, through Wagestream.