Data Engineer
Valleysoft · Cairo, Egypt ·
- Category
- Data engineering
analyticsdata-engineeringdata-modelingdata-pipelinesdata-qualitydata-warehousingetlhivekafkanosqlsql
Description
Valleysoft, a reputable IT services company, is seeking a skilled data engineer to join our dynamic team. The successful candidate will be responsible for designing, developing, and maintaining scalable data solutions to support business analytics and decision-making processes.
Key responsibilities:
- Design, build, and optimize data pipelines and ETL processes for collecting, transforming, and storing data across multiple sources.
- Develop and maintain data warehouses, data marts, and data lakes to enable efficient data access and analysis.
- Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver relevant solutions.
- Ensure data quality, consistency, and security throughout the data lifecycle.
- Monitor and troubleshoot data pipeline performance and implement improvements.
- Keep up-to-date with emerging data engineering technologies and best practices.
Requirements
- Build and maintain data warehouse and data marts, including ODS and internal departmental marts.
- Gather, process, maintain, and analyze large datasets, ensuring data accuracy and quality.
- Design, automate, and support ETL pipelines and data flows across multiple applications.
- Create logical and physical data flow models and mapping designs for ETL processes.
- Strong SQL skills (performance tuning, index management) and solid understanding of database structures.
- Strong understanding of data modeling concepts.
- Familiarity with SQL and NoSQL storage, and big data technologies (Hive, Impala, YARN, Kafka, etc.).
Informatica-specific requirements:
- Hands-on experience with Informatica PowerCenter development (mappings, sessions, workflows).
- Strong knowledge of Informatica transformations (e.g., Lookup, Joiner, Aggregator, Expression, Router, Sorter) and performance tuning.
- Experience with Informatica Developer.
- Experience with Informatica B2B is a plus.
- Ability to design, automate, monitor, and troubleshoot ETL jobs, including scheduling.
Banking sector requirements (preferred):
- Previous experience in the banking sector, preferably business banking, including working with banking datasets.
- Strong understanding of data quality and reconciliation expectations in banking environments.
- Experience working under regulated environments and change control practices is a strong plus.
Benefits
- Private health insurance
- Training & development