Senior Data Engineer
Ztek Consulting · Bengaluru, India ·
- Seniority
- Senior
- Category
- Data engineering
- Experience
- 7+ years
Mandatory skills: Snowflake, Databricks, Spark
- 7–10 years of Data Engineering experience, with expertise in Snowflake, advanced SQL, OLAP platforms, and scalable reporting/data-store design.
- Strong data-modelling skills, including dimensional models, star/snowflake schemas, facts/dimensions, SCD Types 1/2, and Data Vault fundamentals.
- Hands-on experience building ETL/ELT pipelines using Python, Apache Spark, Databricks, dbt, and orchestration tools such as Airflow, ADF, or Databricks Workflows.
- Experience designing data lake/Lakehouse solutions for structured, semi-structured, and unstructured data using Parquet, Avro, JSON, and CSV, with effective partitioning and optimization.
- Proficiency in Snowflake features such as Snow pipe, Streams and Tasks, Dynamic Tables, Time Travel, clustering, secure data sharing, incremental processing, and cost/performance tuning.
Additional Desired Skills:
- Cloud experience with AWS, Azure, or GCP, including object storage such as S3, ADLS, or GCS; exposure to Kafka, CDC, Spark Structured Streaming, APIs, and integration tools is preferred.
- Strong understanding of data quality, reconciliation, observability, lineage, cataloguing, governance, RBAC, masking, encryption, and PII controls using tools such as Unity Catalog, Collibra, Alation, Great Expectations, or Soda.
- Knowledge of Git, CI/CD, automated testing, Terraform/IaC, Docker, Agile delivery, production monitoring, troubleshooting, and SLA management.
- Familiarity with Power BI, Tableau, or Looker, including semantic layers, query optimization, and workload management.
- Proven ability to own architecture and design reviews, translate business and reporting requirements into secure data products, communicate with stakeholders, estimate delivery, and mentor engineers.