- XPT SoftwareAustralia PTY Ltd, incorporated in 2016, is a Software Services company
- XPT works with topclients across Australia in Banking, Insurance, Telecom,Retail, Energy, Mining and Manufacturingdomains.
- We have 120+technocrats in Australia working at our clientlocations.
- XPT SoftwareAustralia is part of group companies which has globalpresence across India & Europe.
- We have served100+ clients globally, fulfilling their onsite-offshoreneeds.
Job Overview
We are seeking an experienced Redshift Data Engineer with 8+ years of experience in data engineering and strong hands‑on expertise with Amazon Redshift. The ideal candidate will be responsible for designing, developing, optimizing, and maintaining scalable data platforms and ETL/ELT pipelines on AWS.
Key Responsibilities
- Design and develop scalable data pipelines and ETL/ELT workflows using AWS services.
- Develop and optimize complex SQL queries, stored procedures, views, and data models in Amazon Redshift.
- Perform Redshift performance tuning, query optimization, distribution/sort key optimization, and workload management.
- Build and maintain data ingestion pipelines from multiple sources into Amazon Redshift.
- Work with AWS Glue, S3, Lambda, Step Functions, and CloudWatch for data integration and orchestration.
- Develop robust data transformation processes using Python and SQL.
- Implement data quality, validation, reconciliation, and monitoring frameworks.
- Design and maintain dimensional data models, including fact and dimension tables.
- Work with large datasets and optimize data processing for performance and cost.
- Implement incremental data loading and CDC (Change Data Capture) strategies.
- Troubleshoot data pipeline failures, performance issues, and production incidents.
- Collaborate with Data Architects, Business Analysts, Developers, and other stakeholders.
- Follow best practices for data security, governance, access control, and AWS IAM.
- Participate in code reviews, technical design discussions, and Agile delivery processes.
Mandatory Skills
- 8+ years of experience in Data Engineering
- Strong hands‑on experience with Amazon Redshift
- Advanced SQL skills
- Strong Python programming/scripting experience
- Experience developing ETL/ELT pipelines
- Strong experience with AWS
- Hands‑on experience with Amazon S3
- Experience with AWS Glue
- Experience with data warehousing and dimensional data modeling
- Strong knowledge of Redshift performance tuning
- Experience with large‑scale data processing
- Experience with CI/CD and Git
- Good understanding of data quality and data governance
Good to Have
- Experience with AWS Lambda, Step Functions, EMR, Athena, and CloudWatch
- Experience with Apache Spark / PySpark
- Experience with Airflow
- Knowledge of Kafka / streaming data
- Experience with CDC tools such as AWS DMS, Debezium, or equivalent
- Experience with Terraform / Infrastructure as Code
- Exposure to Databricks, Snowflake, or other cloud data platforms
- Experience working in Agile/Scrum environments
Technical Skills
Cloud: AWS, S3, Glue, Lambda, Step Functions, CloudWatch
Database: Amazon Redshift, PostgreSQL
Programming: Python, SQL
ETL/ELT: AWS Glue, Airflow, custom Python pipelines
Big Data: Spark, PySpark
DevOps: Git, CI/CD, Terraform
Data Modeling: Star Schema, Snowflake Schema, Dimensional Modeling
Streaming: Kafka – desirable