Manager - Data Engineering (TPA)

Job Description

Role Purpose: To ensure the organization has reliable, well-governed, and analytics-ready data by building and maintaining robust data pipelines and models. The role exists to make trusted data available securely and cost-effectively to support enterprise reporting, BI, and data-driven decision-making.

Key Responsibilities

1. 1- Enterprise Data Ingestion and Data Engineering;
   • Build reusable, parameterized mappings and task flows in Informatica IDMC to standardize data ingestion.
   • Implement change data capture (CDC), idempotent loads, schema evolution, and data-quality gates.
   • Optimize Big Query loads using partitioning, clustering, and the appropriate load-versus-stream approach.
   • Set up version control and CI/CD pipelines for data engineering assets.
2. 2- Data Mapping & Transformation Design;
   • Profile source systems and define field-level mappings and transformation rules.
   • Specify business logic, including joins, lookups, and derivations.
   • Define data-quality rules, exception handling, and reject criteria.
3. 3- Orchestration, Automation & Reliability Engineering;
   • Parameterize task flows and configure schedules and dependencies.
   • Implement retries, backoff, and checkpointing to ensure reliable processing.
   • Integrate monitoring and alerting through the Ops console and ChatOps.
4. 4- Data Roles and Privacy management;
   • Define least-privilege IAM roles and service accounts for data access.
   • Apply dataset, table, row, and column-level security and data masking.
   • Enable audit logging and retention policies, and classify PHI/PII data.
5. 5- Modern Data Architecture & Data Modelling;
   • Implement Medallion Architecture across the Bronze, Silver, and Gold layers.
   • Develop scalable, curated data models for analytical and reporting needs.
   • Define data quality, lineage, and governance standards for curated data.
   • Collaborate with business and analytics teams to create trusted datasets.
6. 6- Workload Monitoring, Performance & Cost Optimisation;
   • Track data freshness, job health, volumes, and anomalies.
   • Monitor SLAs across job duration, errors, cost per TB, and slot usage.
   • Tune BigQuery performance through query optimisation and resource management.
   • Optimize cost through storage lifecycle management, query tuning, and caching.

Skills

• Data Engineering Tools: Informatica IDMC, Airflow, Apache Spark.
• Programming Languages: SQL, Python.
• Cloud Technologies: Google Cloud Platform (BigQuery, IAM), Azure/AWS (optional).
• Visualization & BI Platforms: Power BI, Looker, Looker Studio, Tableau.
• Strong in BigQuery (SQL, modelling, tuning).
• Proficient with Informatica IDMC/IICS ETL/ELT.
• Experience with end-to-end enterprise data pipelines using GCP and Informatica IDMC.
• Knowledge of Medallion Architecture.
• Experience building reporting-ready data marts and BI datasets.
• Experience supporting enterprise BI dashboards and self-service analytics is preferred.
• Knowledge of healthcare data (claims, providers, members).
• Understanding of cloud security (IAM, service accounts, private endpoints).

Education

Bachelor’s degree in Computer Science, Information Technology, Information Systems, or a related