Data Engineer (Cloud & Analytics Platform)

The incumbent will design, build, and maintain scalable data solutions that support analytics, machine learning, and business insights. Responsibilities include developing robust data pipelines, cloud-based data platforms, analytics solutions, and supporting AI/ML initiatives while ensuring data quality, governance, and operational excellence.

Responsibilities

• Design, develop, and maintain ETL/ELT pipelines across cloud and on-premise environments, integrating internal and external data sources, including compliant ingestion of public data where required.
• Build, optimize, and maintain scalable data models (e.g., star and snowflake schemas) to support analytics, reporting, and business intelligence requirements.
• Ensure data quality, integrity, and availability through validation, cleansing, transformation processes, and implementation of monitoring controls.
• Implement and manage cloud-based data solutions on Microsoft Azure, including Azure Data Factory, Data Lake Storage, Azure SQL, and related services.
• Support infrastructure configuration and administration, including compute, storage, networking, identity management, and security controls to ensure reliable and scalable platform operations.
• Implement CI/CD pipelines, automate deployments, and establish monitoring, logging, and alerting capabilities to maintain performance, reliability, and SLA adherence.
• Support the development and optimization of interactive dashboards and reports using Power BI, ensuring seamless integration between data pipelines and reporting layers.
• Collaborate with business stakeholders to understand requirements and translate them into effective data solutions and visualizations.
• Collaborate with Data Scientists to deploy, support, and monitor machine learning models in production environments and integrate AI/ML capabilities into data pipelines and applications.
• Ensure adherence to data governance, security, compliance, access control, and data lineage requirements while following software engineering best practices such as version control, code reviews, and modular design.
• Contribute to continuous improvement of data platform standards, operational performance, scalability, and cost optimization initiatives.

Requirements

• Possess a bachelor’s degree in Computer Science, Computer Engineering, or a related field.
• 2-3 years of experience in data engineering, software engineering, or related roles with exposure to data platforms, cloud technologies, and analytics solutions.
• Strong analytical thinking, problem-solving, and troubleshooting abilities.
• Good communication and collaboration skills with the ability to work effectively across technical and business teams.
• Self-motivated, detail-oriented, and able to manage multiple priorities in a dynamic environment.
• Possess initiative and willingness to learn new technologies, tools, and methodologies.

Technical Skills Required

• Experience with data warehousing concepts, data modeling, database optimization techniques, and ETL/ELT development using Python or ETL tools (e.g., SSIS, Informatica).
• Proficiency in relational database technologies such as Microsoft SQL Server and Oracle, and familiarity with big data technologies such as Spark and Databricks.
• Proficiency in Python for data processing, data engineering tasks, and basic API development.
• Hands-on experience with Microsoft Azure services, including Azure Data Factory, Data Lake Storage, and Azure SQL.
• Familiarity with CI/CD pipelines, DevOps practices, and automation tools such as Azure DevOps.
• Familiarity with streaming or real-time data processing concepts and exposure to monitoring, performance tuning, and optimization of data systems.
• Experience with Power BI, including DAX and Power Query, for dashboard development and reporting.
• Basic understanding of machine learning workflows and tools, including Scikit-learn and Azure Machine Learning.