Databricks Architect - Delta Lake
Idexcel Technologies · Bangalore ·
- Category
- Architecture
- Experience
- 10+ years
Idexcel Technologies · Bangalore ·
Job Description :
Databricks Architect (10+ Years Experience | 5+ Years Databricks)
Position Summary :
Seeking a highly experienced Databricks Architect with 10+ years of IT experience and 5+ years of hands-on Databricks expertise to architect, design, and deliver enterprise-scale Data, Analytics, AI, and Lakehouse solutions. The candidate will lead architecture initiatives, build PoCs, drive cloud modernization, mentor engineering teams, and engage directly with customers and business stakeholders.
Key Responsibilities :
- Architect and implement enterprise Data Lakehouse solutions using Databricks, Delta Lake, Unity Catalog, Lakeflow, Delta Live Tables (DLT), Workflows, MLflow, Mosaic AI, Genie, Vector Search, and AI/BI Dashboards.
- Design batch, streaming, CDC, real-time, and Medallion Architecture data pipelines.
- Build Proof of Concepts (PoCs), accelerators, reusable frameworks, reference architectures, and conduct customer demonstrations.
- Lead Databricks Genie implementations including conversational analytics, prompt engineering, governance, security, performance optimization, and enterprise integrations.
- Perform Spark and Databricks performance tuning, query optimization, workload optimization, AQE, partitioning, clustering, Liquid Clustering, caching, Photon optimization, and cost optimization.
- Drive data governance, security, Unity Catalog adoption, data lineage, and platform observability.
- Lead data modernization, cloud migration, architecture reviews, estimation, presales support, and stakeholder management.
Required Skills :
- Databricks Lakehouse Platform, Delta Lake, Unity Catalog, Lakeflow, DLT, Databricks Workflows, Databricks Asset Bundles (DAB), MLflow, Genie, Vector Search.
- Expert knowledge of PySpark, Spark SQL, Python, SQL, Data Modeling, Data Warehousing, Kafka, and CDC architectures.
- Experience with AWS, Azure, or GCP cloud platforms.
- Strong understanding of Databricks security, multi-tenant architectures, governance, and FinOps.
- Experience integrating Databricks with PostgreSQL, Snowflake, SQL Server, Power BI, Tableau, Superset, REST APIs, Salesforce, and SAP.
DevOps & Platform Engineering :
- Hands-on experience with Databricks Asset Bundles (DAB), GitHub Actions, Terraform, Databricks CLI, and Infrastructure as Code (IaC).
- Design CI/CD pipelines, automated deployments, environment promotion strategies, testing frameworks, and DevSecOps practices.
Certifications :
Mandatory :
- Databricks Certified Data Engineer Professional
- Databricks Certified Developer for Apache Spark
Preferred:
- Databricks Certified Machine Learning Associate
- Databricks Generative AI Fundamentals
- Databricks Data Analyst Associate
- AWS Solutions Architect Professional
Education:
Bachelor's or Master's Degree in Computer Science, Engineering, Information Technology, Data Science, or related field.
Experience:
10+ Years Overall IT Experience | 5+ Years Databricks Experience
Role:
Databricks Architect / Lead Architect / Senior Solution Architect
Preferred Domains:
Banking, Financial Services, SaaS Platforms, Data & AI Modernization Programs.