Data Engineer – Scala / Spark / Kafka

Role: Data Engineer – Scala / Spark / Kafka

Location: India

Employment Type: Contract

Experience: 5+ Years

Duration: 6–12 Months

Start Date: ASAP

Job Description

We are looking for an experienced Data Engineer to design, build, and optimize scalable data platforms capable of processing large volumes of batch and real-time data.

The ideal candidate will have strong hands-on experience with Scala, Apache Spark, Kafka, and Big Data technologies. You will be responsible for developing robust data pipelines, designing scalable data models, troubleshooting performance issues, and collaborating with engineering and analytics teams to ensure reliable access to high-quality data.

The role involves extensive work with distributed processing technologies, streaming platforms, and modern cloud-based data infrastructure.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines for batch and real-time processing.
• Develop data processing applications using Scala and Apache Spark.
• Build and maintain real-time streaming pipelines using Kafka and Spark Structured Streaming.
• Design and implement scalable data models and data processing solutions.
• Work with Hive, SQL, S3, Presto, Delta Lake, and other Big Data technologies.
• Analyze and optimize Spark jobs and distributed data workloads.
• Troubleshoot data pipeline, performance, and production issues.
• Ensure data quality, reliability, scalability, and availability.
• Collaborate with engineering, analytics, and other technical teams to deliver data solutions.
• Contribute to the design and development of modern data lake and lakehouse architectures.

Required Skills

• 5+ years of professional experience in Data Engineering, Big Data Engineering, or a related field.
• Strong programming skills in Scala.
• Hands-on experience with Apache Spark for both batch and streaming workloads.
• Strong proficiency in Hive and SQL.
• Hands-on experience with Kafka and real-time event streaming.
• Strong understanding of Big Data technologies and ecosystems.
• Experience with S3, Hive, Presto, and Delta Lake.
• Experience designing and implementing scalable data models.
• Strong debugging and problem-solving skills.
• Experience with performance analysis and optimization of distributed data workloads.
• Bachelor's or Master's degree in Computer Science, Engineering, or a related field.