Lead Data Platform Engineer
gen-digital · IND - Pune ·
- Work mode
- Onsite
- Seniority
- Lead
- Category
- Data engineering
- Experience
- 15+ years
gen-digital · IND - Pune ·
We're seeking an exceptional Data Platform Engineer to shape the strategic direction and architecture of our data infrastructure. This is a high-visibility, high-impact role where you'll co-own the overall platform architecture, set technical standards across the entire Data Engineering organization, and drive initiatives that directly impact our data strategy. You'll be a force multiplier—inspiring teams, mentoring senior engineers, and making architectural decisions that affect how the entire organization builds and operates data systems. We are in the midst of transforming our legacy infrastructure into a modern cloud-first platform, and you'll be instrumental in leading this technical evolution.
• Co-own and evolve the overall data platform architecture across all domains
• Guide the technical vision for the entire Data Engineering organization
• Drive platform initiatives that directly support the company's data strategy
• Lead the modernization of legacy systems while ensuring business continuity
• Identify and solve problems that span the entire Data Engineering org before they impact delivery
• Set the bar for excellence and serve as an inspiration across all engineering teams
• Design and drive implementation of highly scalable platform architectures for multi-TB+ daily workloads
• Author and sign off on all RFCs and technical specifications that touch your domain(s)
• Define best practices and patterns that become standard across the organization
• Build self-service infrastructure that empowers hundreds of data consumers while maintaining reliability
• Design event-driven, real-time data architectures using Kafka and stream processing at scale
• Contribute to any codebase across the organization, setting quality standards through exemplary code
• Architect comprehensive Infrastructure as Code strategies and GitOps frameworks
• Design enterprise-grade security frameworks including encryption, zero-trust access controls, and compliance
• Establish SLAs/SLOs for critical platform services and drive observability strategy
• Lead complex incident response, conduct thorough post-mortems, and implement systemic improvements
• Drive platform reliability and performance optimization initiatives
• Mentor IC9 and IC10 engineers, accelerating their technical growth and impact
• Collaborate across the entire Engineering organization as a thought partner
• Interact with executives and external stakeholders as a domain expert
• Democratize data access by empowering more teams to contribute to the data stack safely
• Identify hiring gaps and help define interview panels for critical technical roles
• Actively participate in and contribute to Data Engineering communities (conferences, Stack Overflow, vendor forums)
• Expert-level proficiency in SQL, Python, and at least one additional programming language
• Demonstrated mastery of object-oriented design and software engineering principles
• Deep expertise with databases across paradigms: PostgreSQL, MySQL, Snowflake, Oracle, DynamoDB, MongoDB
• Extensive Kafka ecosystem expertise: cluster management, Debezium, schema registry, Connect frameworks, stream processing
• Advanced container orchestration with Kubernetes and Docker at scale
• Expert knowledge of Infrastructure as Code, GitOps, and cloud-native architectures
• Proven track record architecting platforms processing multi-TB daily workloads
• Experience designing and operating systems supporting hundreds of concurrent users
• Deep understanding of distributed systems, cloud architectures, and scalability patterns
• Expert-level experience with observability stacks (Prometheus, Grafana, ELK, Jaeger)
• Mastery of data orchestration platforms like Airflow, Prefect, or Dagster
• Demonstrated success modernizing legacy data systems while ensuring zero business disruption
• 10-15+ years of professional experience in data platform engineering or related field
• Track record of leading complex, org-wide technical initiatives
• History of driving architectural decisions that scale across large organizations
• Experience in high-growth or large enterprise environments
• Deep expertise with both modern and legacy technologies (Oracle, SQL Server, Informatica, SSIS, Business Objects)
• Proven ability to influence technical direction across multiple teams and domains
• Excellence in mentoring senior engineers and accelerating their growth
• Strong executive presence and ability to communicate complex technical concepts to all levels
• Comfortable operating in high ambiguity and defining problems before solving them
• Track record of building collaborative relationships across engineering and business organizations
• Recognition as a thought leader in the Data Engineering community
• Deep experience with modern data stack tools (dbt, Looker, etc.)
• Machine learning infrastructure and MLOps architecture expertise
• Contributions to open-source data infrastructure projects
• Published technical articles, conference talks, or thought leadership in data platforms
• Expert knowledge of data privacy frameworks (GDPR, CCPA) and implementation at scale
First 90 Days: Understand our architecture deeply, identify systemic opportunities and risks, and begin influencing key technical decisions. Establish yourself as a go-to expert in your domain(s).
First Year: Drive at least one major architectural initiative that impacts the entire Data Engineering organization. Establish yourself as a trusted advisor to leadership and a mentor to senior engineers. Author multiple RFCs that set new standards.
Ongoing: Continuously shape the technical direction of the organization, mentor the next generation of technical leaders, and maintain your position as one of the most knowledgeable experts in your domain(s) both internally and in the broader Data Engineering community.
Cloud & Infrastructure: AWS (EC2, EKS, RDS, extensive cloud-native services)
Data Technologies: Apache Kafka, Apache Airflow, SQLMESH, Snowflake, dbt
Languages & Frameworks: Python, SQL, Java, Scala, Docker, Kubernetes
Monitoring & Observability: Prometheus, Grafana, DataDog, ELK Stack, Jaeger
Legacy Technologies: Apache Spark, Oracle, Postgres, SqlServer, Informatica, SSIS, Business Objects