Data engineer at Adyen (payments fintech) in Amsterdam: designs, builds, and operates production ELT pipelines and data architectures, integrating data from many sources and optimizing Spark-based performance. Core stack: Python, PySpark, Spark, Airflow, Kafka, SQL, Git.
Описание: Adyen provides payments, data, and financial products in a single solution for customers such as Meta, Uber, H&M, and Microsoft. It operates as a financial technology platform focused on helping businesses to achieve their ambitions. *Instagram и Facebook принадлежат компании Meta Platforms Inc., деятельность которой признана экстремистской и запрещена на территории РФ
Задачи:
Engage with data scientists, analysts, software engineers, product managers, and customers to understand requirements and craft effective solutions;
Design, develop, deploy, and operate high-quality production ELT pipelines and data architectures;
Integrate data from various sources and formats to ensure compatibility, consistency, and reliability;
Help establish and share best practices for performance, code quality, data validation, data governance, and discoverability;
Participate in mentoring and knowledge-sharing initiatives;
Ensure data is accurate, complete, reliable, relevant, and timely;
Implement testing, monitoring, and validation protocols for code and data using tools such as Pytest;
Identify and resolve performance bottlenecks in data pipelines and systems;
Improve query performance and resource utilization to meet SLAs and performance requirements using Spark optimizations.
Требования:
3+ Years of experience working as a Data Engineer or in a similar role;
Solid understanding of Software and Data Engineering practices;
Proficiency in Python, PySpark, Airflow, Hadoop, Spark, Kafka, SQL, and Git;
Ability to communicate complex data-related concepts and outcomes effectively to diverse stakeholders;
Ability to identify opportunities, devise solutions, and handle projects independently;
Experimental mindset with a “launch fast and iterate” mentality;
Ability to promote a data-centric culture and advocate for standards and continuous improvement.
Условия:
Office role based in Amsterdam; Office-first company with in-person collaboration; remote-only roles are not offered.