Role Overview:
We are looking for a motivated Data Engineer to design, build, and maintain scalable data pipelines and data platforms that enable efficient data collection, storage, processing, and analytics. You will work closely with data scientists, analysts, and business stakeholders to develop reliable data solutions that support business intelligence, machine learning, and digital transformation initiatives.
Key Responsibilities:
- Design, develop, and maintain ETL/ELT pipelines for structured and unstructured data from multiple sources.
- Build and optimise scalable data pipelines to support batch, near real-time, and real-time data processing.
- Develop and maintain data models, data warehouses, and data integration solutions.
- Collaborate with data scientists and business stakeholders to understand data requirements and deliver high-quality datasets.
- Monitor data pipelines to ensure reliability, availability, and performance against defined SLAs.
- Perform data validation, cleansing, and quality assurance to maintain data integrity.
- Implement data governance, security, and compliance requirements across data platforms.
- Troubleshoot and resolve data pipeline issues while continuously improving system performance.
- Support solution implementation, testing, deployment, and post-implementation maintenance.
Requirements:
- Degree in Computer Science, Information Technology, Data Engineering, or a related discipline.
- 2-5 years of experience in building and maintaining data pipelines and ETL solutions.
- Hands-on experience with ETL/ELT tools such as Informatica, Talend, or similar platforms.
- Strong knowledge of SQL and Python.
- Familiarity with data modelling, metadata management, and data warehousing concepts.
- Exposure to big data technologies such as Apache Spark, Hadoop, or Hive is advantageous.
- Experience working with multiple data sources including SQL databases, PostgreSQL, SAP, flat files, APIs, and unstructured data is preferred.
- Exposure to Power BI, SharePoint, Kubernetes, Docker/Podman, Microservices, or DevSecOps practices will be an added advantage.
- Strong analytical, problem-solving, and communication skills.
(EA Reg No: 20C0312)