Data engineer at Devoteam, a technology consultancy in the Netherlands, designing, building, and optimizing scalable data pipelines with Databricks, Spark, and SQL on Azure, AWS, or GCP, delivering data solutions for clients alongside data scientists and business stakeholders.
Devoteam is an AI-driven technology consultancy with more than 11,000 technology experts across 25 countries in Europe and the Middle East. The company partners with Google, Microsoft, AWS, and ServiceNow and specializes in Cloud, Cyber, Data, and AI in the Netherlands.
Задачи
- Design, develop, and deploy robust and scalable data pipelines using Databricks;
- Extract data from databases, APIs, and streaming platforms;
- Transform and cleanse data using Spark and load it into data lakes, data warehouses, and other target systems;
- Optimize pipeline performance for efficiency and cost-effectiveness;
- Use Databricks SQL, Delta Lake, Databricks Runtime, and Databricks Workflows to orchestrate data workflows;
- Integrate Databricks with Azure, AWS, or GCP cloud services for storage, compute, and security;
- Contribute to data models and architectures for data lakes and data warehouses;
- Implement data quality checks and validation procedures throughout data pipelines;
- Follow data governance policies and best practices;
- Collaborate with data scientists, analysts, business intelligence specialists, and business stakeholders;
- Provide technical expertise on Databricks and data engineering best practices;
- Stay up to date with Databricks features, cloud technologies, and data engineering trends;
- Continuously improve solutions and deliver innovative results for clients.
Требования
- Hold a university degree in Computer Science, (Business) Engineering, or a specialized master's in Machine Learning or AI;
- Have 2+ years of experience as a Data Engineer with demonstrated expertise in Databricks;
- Have hands-on experience with at least one major cloud platform: Azure, AWS, or GCP;
- Understand cloud data services and infrastructure;
- Have strong programming skills in Python or Scala;
- Have experience with data manipulation libraries such as PySpark and Spark SQL;
- Understand data warehousing principles, ETL processes, data modeling techniques, and database systems;
- Have advanced SQL skills for data querying, transformation, and analysis;
- Have excellent communication and interpersonal skills;
- Have strong analytical and problem-solving abilities with a proactive approach to data-related challenges;
- Nice to have: Proficiency in Dutch.
Условия
- Excellent primary and secondary employment benefits, including salary, bonus, lease car, laptop, and phone;
- Personalized career development plan with training and certifications;
- Opportunities to grow as a thought leader through knowledge sharing, blogs, and keynote sessions;
- The option to work from within Europe for 4 weeks per year;
- Internal knowledge-sharing community with sessions including Tech Thursdays and KISSes.