Lead Data Engineer
CoffeeBeans · Bangalore, India ·
- Seniority
- Lead
- Category
- Data engineering
- Experience
- 6+ years
CoffeeBeans · Bangalore, India ·
Jobskey Consultancy
Advance Auto Parts · Hyderabad, India
EXL · Pune, Maharashtra, India
Caterpillar Inc. · Chennai, Tamil Nadu
Blend 360 · Buenos Aires, Buenos Aires, ar
CoffeeBeans, a Bangalore-based technology consulting firm, is hiring a Lead Data Engineer with 6–10 years of experience to design, build, and maintain scalable data pipelines and platforms. The core stack includes SQL, Python or Java, Apache Spark, Airflow, Databricks/Snowflake, and a major cloud provider.
Founded in the year 2017, CoffeeBeans specialises in offering high end consulting services in technology, product, and processes. We help our clients attain significant improvement in quality of delivery through impactful product launches, process simplification, and help build competencies that drive business outcomes across industries. The company uses new-age technologies to help its clients build superior products and realise better customer value. We also offer data-driven solutions and AI-based products for businesses operating in a wide range of product categories and service domains.
Experience - 6 - 10 years
We are seeking an experienced Data Engineer to join our data team. In this role, you will be responsible for designing, building, and maintaining scalable data pipelines and platforms that enable advanced analytics and data-driven decision making across the organisation.
Key Responsibilities:
● Design, develop, and maintain efficient, reliable, and scalable data pipelines using modern data engineer tools and best practices.
● Build and optimise data flows between various internal and external data sources and destination ● Implement and maintain data quality checks, monitoring, and alerting systems.
● Write efficient and maintainable code following best practices and coding standards
● Participate in code reviews and contribute to technical documentation.
● Troubleshoot and resolve data pipeline issues and performance bottlenecks.
Required Technical Skills:
● Strong proficiency in SQL and data modelling
● Hands-on experience with Python or Java programming.
● Proven experience with Apache Spark for large-scale data processing.
● Experience with Apache Airflow for workflow orchestration
● Expertise in working with modern data platforms and warehouses, particularly Databricks and/or Snowflake (good to have Snowflake / databricks certification).
● Experience working with at least one major cloud provider (AWS, GCP, or Azure).
● Strong understanding of data engineering concepts, ETL processes, and data warehousing principles.