Data Platform Engineer
LinuxRecruit · London, United Kingdom ·
- Category
- Data engineering
Designs and maintains large-scale data pipelines on AWS for an AI-driven platform, ensuring data quality, security, and integrity while collaborating with cross-functional teams. Core stack: SQL, Python, and AWS services such as Athena, S3, Glue, and Spark.
Overview
As a Data Platform Engineer, you design and maintain data pipelines for a high-volume AI-driven platform. You ensure data quality, security, and integrity while collaborating with cross-functional teams to solve complex data challenges. You will stay current with trends and drive continuous improvement in data processing. This role offers impact by shaping scalable data infrastructure and enabling data-driven insights in a fast-paced scale-up.
Responsibilities- design and maintain data processing pipelines at scale
- ensure data quality, integrity, and security
- collaborate with cross-functional teams to tackle data challenges
- drive continuous improvement and innovation in data processing domains
- stay current with trends and technologies to enhance data platforms
- experience in a heavy AWS environment
- strong capabilities in data security, data management, DevOps, and software engineering
- proficient in SQL and Python
- experience with AWS services Athena, S3, Glue, and Spark
- familiarity with big data technologies and orchestration tools
- strong critical and statistical thinking and problem-solving abilities
- cross-functional collaboration
- problem solving
- continuous learning
- SQL
- Python
- AWS (Athena, S3, Glue)