Data Engineer
Capgemini Engineering · Bogotá, Colombia ·
- Category
- Data engineering
- Experience
- 2+ years
Build and maintain ETL pipelines with Azure Data Factory and Databricks, process large datasets with Spark, and deploy ML models using Python and Azure.
Data Engineer
Choosing Capgemini means choosing a company where you will be empowered to shape your career the way you'd like, where you'll be supported and inspired by a collaborative community of colleagues around the world, and where you'll be able to reimagine what's possible. Join us and help the world's leading organizations unlock the value of technology and build a more sustainable, more inclusive world.
Job Description
Develop and maintain efficient ETL pipelines using Azure Data Factory and Databricks to process and transform large datasets for analysis. Analyze large datasets and build machine learning models using Python and Spark to extract actionable insights and solve business problems. Utilize Apache Spark to process and analyse large volumes of data in a distributed environment, ensuring scalability and performance. Deploy machine learning models to production using Azure and Databricks, and monitor their performance for continuous improvement. Work closely with cross‑functional teams (data engineers, business analysts, etc.) to understand requirements and communicate insights and results effectively to stakeholders.
Requirements
- Advanced English skills (B2+)
- Experience in ETL and Azure tools: A minimum of 2-5 years of experience in developing ETL processes using Azure Data Factory, Azure Databricks, and other Azure cloud tools.
- Knowledge of Apache Spark and Databricks: Experience using Apache Spark for distributed data processing and knowledge of Databricks for data integration and transformation.
- Proficiency in Python and SQL: Advanced skills in Python for automating processes and manipulating data, along with a strong understanding of SQL for querying and transforming data.
- Handling large volumes of data: Ability to manage, transform, and optimise large volumes of data using cloud platforms and processing technologies like Spark.
- Team‑work and communication: Collaborative skills with multidisciplinary teams and effective communication ability to interact with both technical and non‑technical stakeholders.
What You'll Love About Working Here
- Flexible work arrangements, including remote work and flexible hours, to support a healthy work‑life balance.
- Career growth programmes and diverse professions to explore a world of opportunities.
- Certification opportunities in the latest technologies.
Location
Location: Bogota, D.C., Capital District, Colombia
Seniority level
- Mid‑Senior level
Employment type
- Full‑time
Job function
- Information Technology