AWS Data Engineer – Data Warehouse & Netezza Migration
Astra North Infoteck Inc. · Toronto, Canada ·
- Category
- Data engineering
- Experience
- 5+ years
Astra North Infoteck Inc. · Toronto, Canada ·
Astra-North Infoteck Inc. ~ Conquering today’s challenges, achieving tomorrow’s vision! · Toronto, Canada
Inizio Partners Corp · Newark, New Jersey, United States
Facctum · Bangalore North, India
Capgemini
Hybrid (2 days/week in-office) AWS data engineer role in Toronto: build and maintain scalable data pipelines, data lakes and warehouses on AWS (S3, Glue, Lambda, Redshift, Step Functions) and migrate IBM Netezza warehouse logic to an AWS data lake for a banking/financial client. 5+ years in the bank/financial domain required.
· Amazon S3 (Data Lake)
· AWS Glue (ETL)
· AWS Lambda
· Amazon Redshift
· AWS Step Functions
· AWS IAM & CloudWatch
· IBM Netezza
Role and
Responsibilities
· Required to have strong data warehouse technical
knowledge.
· Must have knowledge in Bank and financial domain
(Minimum 5+ years of experience).
· Design, develop, and maintain scalable data
pipelines and workflows on AWS Build and manage data lakes and data warehouses
(e.g., S3, Redshift).
· Develop ETL/ELT processes using tools like AWS
Glue, Lambda, or Spark Strong solution.
· Knowledge in AWS Cloud and hands on experience
with ETL process (like Kafka message processing, batch interface data injection
and other business layer process).
· Optimize data ingestion, transformation, and
loading processes.
· Work with stakeholders to understand and deliver
data requirements.
· Ensure data quality, governance, and security
compliance.
· Monitor and troubleshoot data pipelines and
workflows Automate deployments using CI/CD pipelines Improve performance, cost
optimization, and scalability.
· Coordinate with clients, data users and key
stakeholders to understand feature requirements needed merge them to create
reusable design patterns.
· Data onboarding using the developed frameworks
Understand and make sense of available code in Netezza to design a best way to
implement its current features in AWS Data Lake.
· Unit test code and aid with QA/SIT/Perf testing
Migration to production environment
Good to have
· AWS Certifications (Solutions Architect / Data
Analytics)
· Experience with Airflow (or Managed Workflows
for Apache Airflow - MWAA)
· Exposure to Snowflake, Databricks, or Delta Lake
Ambition · Singapore, Singapore