The Data Engineer is responsible for building, supporting, and optimising scalable, repeatable, and secure data pipelines and analytical platforms. Operating as a core member of an agile engineering team, the role focuses on enabling business insights through the integration, transformation, and management of large-scale batch and real-time data solutions. The Data Engineer works across cloud and big data technologies to architect, develop, and support modern data platforms that enable analytics, reporting, automation, and data-driven decision-making.
Key Responsibilities
- Design, build, and maintain scalable data analytics frameworks and data platforms.
- Translate complex functional and technical requirements into scalable architecture and high-performing data solutions.
- Develop and support batch and real-time data processing solutions using modern big data technologies.
- Create and maintain secure and reliable data pipelines between on-premise systems and AWS cloud environments.
- Develop and maintain ETL processes and data transformation workflows using Talend or similar ETL tools.
- Manipulate, process, and analyse data using Python and related technologies.
- Develop and support Big Data and Business Intelligence solutions including automated testing and deployment practices.
- Process large-scale datasets using Hadoop paradigms and AWS EMR services.
- Support production data feeds and resolve operational issues through break-fix support activities.
- Contribute to database development, optimisation, and operational management.
- Participate in software development, DevOps, and data operations activities.
- Ensure alignment with policies, standards, procedures, business continuity, and disaster recovery practices.
- Research and evaluate emerging technologies, tools, and frameworks relevant to data engineering and analytics.
- Collaborate with business and technical stakeholders to understand data requirements and deliver effective solutions.
Essential Skills & Experience
- 5+ years’ experience in Data Engineering or Software Engineering roles.
- 2+ years’ experience working with Big Data technologies and platforms.
- 2+ years’ experience with ETL processes and tooling.
- 2+ years’ experience working with AWS cloud technologies.
- Strong experience with Python for data manipulation and processing.
- Hands-on experience with AWS services including EMR, EC2, and S3.
- Experience with PySpark, Spark, Hadoop, or distributed data processing frameworks.
- Strong understanding of data modelling, data structures, and scalable data architecture design.
- Experience designing highly scalable distributed systems using open-source technologies.
- Strong understanding of object-oriented design, coding standards, and testing practices.
- Experience working with large-scale data infrastructure and analytical platforms.
- Strong analytical, troubleshooting, and problem-solving skills.
Desirable Skills & Experience
- Experience with real-time streaming and event-driven data processing.
- Exposure to cloud-native data engineering architectures.
- Experience implementing automated deployment and CI/CD practices for data platforms.
- Exposure to business intelligence and analytics tooling.
- Experience working within agile software delivery environments.
Qualifications
- Bachelor’s degree in Computer Science, Computer Engineering, Information Technology, or equivalent practical experience.
- AWS certification would be advantageous.
Behavioural Competencies
- Strong analytical and problem-solving mindset.
- Excellent communication and stakeholder engagement skills.
- Strong attention to detail and commitment to quality.
- Ability to work effectively both independently and within collaborative agile teams.
- Adaptability and willingness to learn evolving technologies and tools.
- Proactive mindset with focus on innovation and continuous improvement.
What We Offer
- Competitive salary and benefits package.
- Opportunity to work on modern cloud and big data platforms.
- Exposure to large-scale data engineering and analytics projects.
- Collaborative and innovative engineering environment.
- Professional growth and learning opportunities.
- Opportunity to contribute to impactful data-driven solutions.
#J-18808-Ljbffr