Data Engineer (Cloudera)

With just over 9 years of experience in the Portuguese market, we share our DNA with more than 200 workers and position our offer according to 3 lines of service:

• Strategy (Outsourcing, NeXel, Team as a Service, Tech Academies);
• Nearshore.

In ADENTIS, we focus on PEOPLE. This is our emotional salary:

• Great Work-Life balance;
• Very flexible organizational routine;
• Health benefits (for you and your family);
• Over 300 protocols to offer you great discounts in different areas;
• Continuous professional development sponsored by our Training and Certification Department;
• Regular feedback on your performance through a personalized plan;
• Comprehensive career plan and progression involving assertive performance reviews.

Responsibilities:

• Distributed Data Processing: Develop and optimize distributed data pipelines using Apache Spark, ensuring efficient processing of large datasets
• Caching & Computation Optimization: Implement advanced caching techniques and optimize distributed computation workflows to improve data retrieval times and resource utilization
• Partition Key Join Optimization: Optimize partition keys joins to speed up query processing and enhance overall system performance in distributed environments
• SQL Query Optimization: Optimize SQL queries for better performance and scalability, particularly in large-scale, distributed database systems
• Cloudera Ecosystem: If applicable, leverage the Cloudera platform to manage and scale data workflows in big data ecosystems, ensuring reliability and efficiency
• Wants to learn more about DevOps to be autonomous, with the following DevOps responsibilities (5-10% of the time)

Requirements:

• Expertise in Apache Spark and distributed computation techniques
• Experience with advanced SQL optimization and caching strategies
• Knowledge of the Cloudera ecosystem is a valuable asset for managing big data platforms
• Ability to work independently and take initiative to solve technical problems
• Excellent communication skills to effectively collaborate with technical and non-technical teams.
• Version Control & Branching
• Containerization & Orchestration
• Automation & Monitoring
• Available to go to the office twice per week