What we want:
We are looking for a skilled Big Data Engineer to design, develop, and maintain scalable big data solutions. The role involves working with Hadoop ecosystems, real-time and batch data processing frameworks, and cloud-based platforms. The ideal candidate will contribute to end-to-end data architecture, ensure efficient data processing, and collaborate with cross-functional teams to deliver reliable and high-performance data solutions.
Who We are:
Vertoz (NSEI: VERTOZ) is an AI-powered MadTech and CloudTech platform offering Digital Advertising, Marketing & Monetization (MadTech) and Digital Identity and Cloud Infrastructure (CloudTech) solutions. We cater to Businesses, Digital Marketers, Advertising Agencies, Digital Publishers, Cloud Providers, and Technology companies.
What you will do:
•Design, develop, and maintain scalable Hadoop-based applications and data pipelines.
•Work on documentation, system design, development, and architecture of big data solutions.
•Implement and manage batch and real-time data processing using Spark, Spark Streaming, Kafka, and related technologies.
•Develop efficient data workflows using Hadoop ecosystem tools such as Hive, Impala, and HDFS.
•Work with stream-processing frameworks including Spark Streaming, Storm, and Flume.
•Integrate and manage data across relational SQL and NoSQL databases, including Vertica.
•Support deployment and operations in cloud-based environments.
•Perform cluster management and monitoring using Cloudera Hadoop Distribution and related tools.
•Write and maintain shell scripts to automate operational tasks.
•Collaborate with teams to ensure data reliability, performance optimization, and scalability.
•Support data visualization and analytics using tools such as Superset.
Requirements
•1+ year of hands-on experience working with Big Data technologies.
•Strong knowledge of Hadoop ecosystem tools including Hadoop, Hive, Impala, Spark, Spark Streaming, and Kafka.
•Experience with batch and real-time data processing frameworks.
•Proficiency in at least one programming language: Java, Python, or Scala.
•Experience with stream-processing systems such as Spark Streaming, Storm, or Flume.
•Good understanding of relational SQL and NoSQL databases, including Vertica.
•Exposure to cloud services and distributed systems.
•Hands-on experience with Cloudera Hadoop Distribution and cluster management.
•Basic to intermediate shell scripting skills.
•Strong problem-solving skills and ability to work in a fast-paced environment.