Data Engineer

Key Responsibilities

• Design, develop, and maintain data pipelines and ETL/ELT processes using AWS services.

• Develop and optimize SQL queries for data extraction, transformation, validation, and analysis.

• Use Python for data processing, automation, and pipeline development.

• Manage and process data using AWS S3 and AWS Redshift.

• Develop and maintain ETL workflows using AWS Glue.

• Configure and manage AWS IAM roles, users, and policies to support secure data access.

• Monitor data pipelines, jobs, and AWS resources using AWS CloudWatch.

• Perform data synchronization and ensure consistency across systems.

• Troubleshoot data pipeline failures, performance issues, and data quality concerns.

• Implement data validation and reconciliation processes to ensure accuracy and completeness.

• Collaborate with Data Analysts, Software Engineers, Business Analysts, and other stakeholders to understand data requirements.

• Maintain technical documentation for data pipelines, processes, and AWS environments.

• Follow data security, governance, and access control best practices.

Qualifications

• Bachelor's degree in Computer Science, Information Technology, Engineering, Data Science, or a related field.

• 3+ years of experience in Data Engineering or a related role.

• Strong hands-on experience with SQL and Python.

• Experience working with AWS S3 and AWS Redshift.

• Hands-on experience with AWS Glue for ETL/data processing.

• Knowledge of AWS IAM, including Roles, Users, and Policies.

• Experience with AWS CloudWatch for monitoring and troubleshooting.

• Experience with data synchronization, ETL/ELT, and data pipeline development.

• Strong understanding of data processing, transformation, and validation.

• Strong analytical and problem-solving skills.

• Good communication and collaboration skills.

Preferred Skills

• Experience with additional AWS data services such as AWS Lambda, Athena, RDS, or Step Functions.

• Experience with Apache Spark/PySpark.

• Knowledge of data warehousing and dimensional modeling.

• Experience with CI/CD and Git.

• Knowledge of data quality and data governance concepts.

• Experience working in an Agile/Scrum environment.

Key Skills

Data Engineering | SQL | Python | AWS S3 | AWS Redshift | AWS Glue | AWS IAM | AWS CloudWatch | Data Synchronization | ETL/ELT | Data Pipelines | Data Warehousing | Data Processing | Data Validation | AWS