Experteer Overview
¿Posee las habilidades y la experiencia adecuadas para este puesto? Siga leyendo para descubrirlo y envíe su solicitud.
As Data Engineer - Developer, you will design and maintain data pipelines and platforms to ingest, transform, and publish data for data products across our data hub. You will work in diverse data squads to enable reliable analytics and decision-making at scale. This role offers the chance to modernize big data infrastructure, migrate to cloud, and contribute to cross-border data initiatives within an inclusive, international team.
Compensaciones / Beneficios
• Model and build data pipelines with Spark and Scala to ingest and transform data from Kafka, APIs, HDFS, databases and files into storage layers and databases
• Manage big data storage solutions (HDFS, IBM Cloud Storage, SQL/NoSQL)
• Implement data transformation and quality controls using Scala, SQL and Spark
• Set up CI/CD pipelines for automated deployment, tests, and development workflows
• Migrate Hadoop infrastructure to cloud tech (Kubernetes Engine, IBM Cloud storage, Spark as a service, Airflow as a service)
• Define schemas, queries, and views in SQL/NoSQL databases (Oracle, Postgres, MongoDB)
• Configure and schedule data pipelines with shell scripts and Airflow
• Perform unit and validation testing and maintain technical documentation
• Improve existing code for performance and maintainability
• Configure Dremio for data virtualization and expose xqbhyrx data in products
• Ensure performance and security of the data stack following best practices
Responsabilidades
• Experience in IT international companies
• Minimum 2 years in related roles
• Languages: English B2/C1; French bonus
Requisitos principales
• Training programs
• career plans
• internal mobility opportunities
• diversity and inclusion committee
• Corporate volunteering program
• Hybrid telecommuting model