Senior Data Engineer designing, building, and maintaining production-grade ETL/ELT pipelines and on-premises lakehouse solutions, while driving platform performance, security, and engineering best practices. Core stack includes SQL Server, Python/PySpark, Spark, Kubernetes/Docker, Airflow, and Iceberg.
Job Summary
We are seeking a senior data engineer to design, build, and maintain production-grade data pipelines and lakehouse solutions in an on-premises environment, while driving data platform initiatives, performance, security, and technical best practices.
Mandatory Skill Set
8+ years of IT experience with 5+ years in data engineering/data pipelines
Expert SQL/SQL Server - query optimization, indexing & performance tuning
Advanced Python & PySpark
Apache Spark and distributed data processing
Kubernetes & Docker for on-premises deployments
Data modelling - relational & NoSQL hands-on MongoDB/Cassandra
Lakehouse/Data Lake architecture, including Iceberg, partitioning & metadata
ETL/ELT orchestration - Apache Airflow or similar
CI/CD, Git, automated testing & IaC
Data security, governance, RBAC, masking & compliance.
Desired Skill Set
Experience in data virtualization, logical data warehouses, Data Mesh/Fabric architectures, metadata management, and data lineage
Familiarity with real-time streaming technologies such as Kafka/Flink, along with experience in technical leadership, mentoring, and Agile data architecture.
Responsibilities
Design and develop production-grade ETL/ELT pipelines using Python/PySpark
Build and manage multi-layer lakehouse architecture across raw, curated, and consumption layers
Deploy scalable data pipelines using Kubernetes/Docker in on-premise environments
Develop data models and optimize complex SQL Server queries and performance
Establish CI/CD, automated testing, monitoring, and version-control practices
Implement data security, governance, and access controls
Mentor engineers, conduct code reviews, and drive engineering best practices
Collaborate with data scientists, BAs, and stakeholders to deliver business-ready datasets
Provide L3 support and lead resolution of complex data engineering challenges.
Should you be interested in this career opportunity, please send in your updated resume to
(HIDDEN TEXT)
at the earliest.
When you apply, you voluntarily consent to the disclosure, collection, and use of your personal data for employment/recruitment and related purposes in accordance with the SCIENTE Group Privacy Policy, a copy of which is published at SCIENTE's website (
Privacy Policy | SCIENTE).
Confidentiality is assured, and only shortlisted candidates will be notified for interviews.
EA Licence No. 07C5639