Position Summary
IT Excellence is seeking an experienced, highly technical, and accomplished Data Engineer with 3 to 5+ years of hands‑on data engineering, cloud data architecture, and ETL/ELT pipeline development experience to join our technology team on a full-time permanent basis in Dubai, UAE. In this specialized data engineering role, you will spearhead designing, building, and maintaining scalable and reliable data pipelines and cloud infrastructure to support advanced Data Science, GenAI applications, and enterprise analytical reporting. You will work closely with data scientists, LLM engineers, and product analysts to deliver robust, high‑performance data systems. Ideal candidates bring a robust academic degree in computer science or engineering, deep practical mastery of Azure data platforms (Azure Data Factory, Databricks, Synapse Analytics), PySpark, Python, and SQL, and immediate availability for deployment in Dubai.
Detailed Job Description
As a Data Engineer at IT Excellence in Dubai, UAE, you will take full ownership of end‑to‑end data pipeline development, cloud data lake modeling, workflow orchestration, and infrastructure automation. Your day‑to‑day responsibilities encompass designing and maintaining ETL/ELT pipelines using Azure Data Factory, Azure Databricks, PySpark, Scala, and SQL, building and optimizing data models in Azure Synapse Analytics, and deploying Apache Airflow pipelines on Red Hat Linux using Kubernetes. You will collaborate proactively with cross‑functional data science and analytics squads to implement data quality checks, validation rules, error handling, and comprehensive monitoring. Working in a fast‑paced technology consulting environment in Dubai, you will drive data reliability, pipeline scalability, and cloud cost optimization.
Key Responsibilities
- Design, develop, and maintain robust ETL/ELT pipelines using Azure Data Factory, Azure Databricks, PySpark, Scala, and SQL.
- Build and optimize scalable data models and analytical tables using Azure Synapse Analytics and Azure Data Lake Storage Gen2.
- Build and deploy Apache Airflow pipeline orchestration workflows on Red Hat Linux environments using Kubernetes.
- Implement comprehensive data quality checks, schema validation, automated error handling, and pipeline monitoring.
- Manage Azure data platform components with an uncompromising focus on performance tuning, security, and cost optimization.
- Collaborate proactively with Data Scientists, LLM Engineers, and Product Analysts to deliver clean, accessible enterprise data.
- Develop real‑time and batch data ingestion solutions from external APIs, transactional databases, Azure Event Hubs, and Kafka.
- Optimize large‑scale data processing and storage using partitioning, indexing, Parquet, Delta Lake, and related technologies.
- Implement enterprise data governance, access control, and security best practices across all data stores.
- Troubleshoot complex pipeline and cloud platform issues to ensure high availability, reliability, and uptime.
- Automate infrastructure workflows and deployments using CI/CD pipelines and Infrastructure‑as‑Code tools (ARM, Bicep, Terraform).
Required Qualifications & Skills
- Bachelor’s or Master’s degree in Computer Science, Software Engineering, Information Technology, or a related technical discipline.
- Minimum 3 to 5+ years of hands‑on professional Data Engineering experience in enterprise cloud environments.
- Strong SQL skills and professional proficiency in Python or Scala programming.
- Strong hands‑on operational experience with Azure Data Factory, Azure Databricks, Azure Synapse Analytics, Azure Data Lake Storage, and Azure Blob Storage.
- Extensive practical experience with Apache Spark, PySpark, or Scala distributed data processing frameworks.
- Strong understanding of enterprise data warehousing, data modeling principles, and ETL/ELT design patterns.
- Proven experience with data pipeline orchestration and scheduling tools (Apache Airflow).
- Familiarity with Git version control, CI/CD pipelines, Kubernetes, Linux environments, and data governance practices (Application email: [email protected]).
Nice-to-Have Skills
- Professional certifications such as Microsoft Certified: Azure Data Engineer Associate or Databricks Certified Data Engineer.
- Hands‑on experience with streaming data platforms such as Apache Kafka or Azure Event Hubs.
- Prior working experience in tier‑1 enterprise technology consulting firms or data‑driven startups in the UAE / GCC region.
- Advanced performance tuning experience for Delta Lake storage formats.
- Bilingual proficiency in English and Arabic.
Application Information
- Recruiter: IT Excellence Talent Acquisition Practice
- Contact Name: Harsh (Recruitment Lead)
- Email: [email protected]
- Phone: Unspecified
- Application URL: Unspecified
- Salary/Rate: Competitive market rate commensurate with experience
- Deadline: Open until filled
- Notice Period: Immediate to short notice preferred
- Contract Duration: Permanent / Full‑Time Onsite (Dubai, UAE)
Recruitment Pro Tip
When applying for senior data engineering roles in Dubai, ensure your resume explicitly highlights your Azure Databricks Spark depth, Airflow orchestration experience, and cloud cost optimization track record.