Senior Architect - Databricks
Celebal Technologies · Hyderabad; Jaipur; Bangalore ·
- Seniority
- Senior
- Category
- Architecture
- Experience
- 3+ years
Celebal Technologies · Hyderabad; Jaipur; Bangalore ·
Job Description :
Celebal Technologies is looking for an experienced Senior Architect Databricks to join our growing Databricks Practice.
In this customer-facing consulting role, you will work closely with enterprise customers, including CDOs, Data Architects, and Engineering Leaders, to design and deliver scalable Lakehouse solutions on the Databricks Data Intelligence Platform.
You will lead enterprise data modernization initiatives, drive large-scale EDW migrations, and provide architectural guidance while leveraging the latest Databricks capabilities, including Lakeflow, Unity Catalog, Serverless Compute, MLflow, and AI/BI solutions.
Key Responsibilities :
- Lead customer discussions from discovery, solution design, and architecture through implementation.
- Design end-to-end Databricks Lakehouse architectures covering ingestion, transformation, governance, machine learning, and analytics.
- Drive enterprise EDW modernization and migration programs from platforms such as Teradata, Oracle, Netezza, SAS, Greenplum, Hadoop, Synapse, and Snowflake to Databricks.
- Define migration strategies, wave planning, risk mitigation, and ROI/TCO assessments.
- Implement and recommend advanced Databricks capabilities, including :
1. Lakeflow Declarative Pipelines (LDP)
2. Lakeflow Connect
3. Serverless Compute
4. Unity Catalog
5. Databricks Asset Bundles (DAB)
6. Databricks SQL, AI/BI Dashboards, and Genie
7. MLflow, Model Serving, and Mosaic AI
- Build production-grade ETL pipelines using DBT and Databricks Auto Loader.
- Conduct architecture workshops, PoCs, executive presentations, and technical deep dives.
- Prepare architecture documentation, including HLDs, LLDs, migration roadmaps, reference architectures, and runbooks.
- Collaborate with Sales, Delivery, and Databricks teams to support proposals, solutioning, and Statements of Work (SoWs).
- Stay current with the Databricks product roadmap and promote new capabilities internally and externally.
- Mentor architects and data engineers on Databricks best practices.
Required Experience :
- 12+ years of experience in Data Engineering, Data Platform Architecture, or Solution Architecture.
- Minimum 8+ years in Data Engineering/Data Platform Architecture with at least 3+ years of hands-on Databricks experience in a consulting or customer-facing role.
- Experience leading multiple enterprise-scale EDW modernization or Lakehouse migration programs.
- Strong expertise in Databricks Data Intelligence Platform and Lakehouse Architecture.
- Hands-on experience with :
1. Delta Lake
2. Apache Spark and PySpark
3. Unity Catalog
4. Lakeflow Declarative Pipelines
5. Lakeflow Connect
6. Serverless Compute
7. Databricks SQL
8. AI/BI Dashboards
9. MLflow
10. Mosaic AI
11. DBT
12. Databricks Auto Loader
- Strong SQL and PySpark programming skills.
- Expertise in Medallion Architecture, Dimensional Modeling, and Data Governance.
- Experience with cloud platforms such as Azure, AWS, or GCP, including networking, IAM, and storage services (ADLS Gen2, Amazon S3, or Google Cloud Storage).
- Excellent stakeholder management and communication skills with experience engaging technical teams and C-level executives.
- Willingness to travel internationally for customer engagements when required.
Preferred Skills :
- Disaster Recovery architecture (Delta Deep Clone, cross-region replication, RPO/RTO design)
- MLOps and Generative AI implementation using MLflow, Model Serving, Vector Search, and Mosaic AI
- Experience with Informatica, ODI, SAS DI, IBM DataStage, or Talend
- Infrastructure as Code using Terraform and Databricks Asset Bundles
- Governance tools such as Immuta, Collibra, Microsoft Purview, or Alation
- Domain experience in BFSI, Insurance, Retail, Manufacturing, or Healthcare
What We're Looking For :
- Proven experience owning solution architecture from discovery through production deployment.
- Ability to act as a trusted advisor for enterprise customers adopting Databricks.
- Strong leadership and mentoring capabilities.
- Experience building reusable accelerators, reference architectures, and migration frameworks.
- Strong collaboration with Databricks field teams on strategic customer engagements