Data Engineer Specialist

Scotiatech is a world-class engineering group! We are seeking to modernize bank technology by delivering a trustworthy, unique, and top-quality service.

We want you to be part of a Team that is always in personal and professional growth, for this reason we are looking for our new Data Engineer Specialist.

What will you do?

• Build, test, and maintain data pipelines and ETL/ELT processes on Azure and Azure Databricks (Delta Lake, medallion architecture) to ensure reliable and efficient data flow for data products.
• Collaborate with the Senior Data Engineer and cross-functional teams (Data Scientists, Product Owners, Architecture, and Platform Engineering) to understand requirements and deliver high-quality data solutions aligned with international business goals.
• Implement data models, schemas, and transformations (dimensional and Data Vault patterns) to support analytics and reporting needs.
• Ensure data quality, integrity, and performance by monitoring and optimizing data pipelines, including performance tuning and cost optimization (FinOps) of Spark workloads.
• Adhere to and help apply data governance, security, and compliance standards within Azure environments, including Unity Catalog, role-based access control, data lineage, encryption, and key management.
• Troubleshoot and resolve issues in data pipelines to minimize downtime and ensure operational efficiency, performing root-cause analysis where needed.
• Contribute to the adoption of best practices and tools for data engineering, including documentation, automated testing, and DevOps/DataOps practices.
• Support the migration and rationalization of legacy workloads (including existing Google Cloud Platform estates) onto the Azure Databricks target platform, minimizing business and regulatory risk.
• Stay updated on modern Azure and Databricks data engineering trends and tools to enhance pipeline capabilities.

What do we offer you?

• Growth opportunities
• Diverse, dynamic and multicultural environment
• Benefits for your financial and emotional well-being
• competitive wages

What do we expect from you?

Experience

• Have a B1 level of English.
• 3-5 years of experience in Data Engineering, designing and developing scalable data solutions in cloud environments.
• Hands-on experience with Azure Databricks, including data pipeline development, Delta Lake, Spark, notebooks, workflows, Unity Catalog, and Lakehouse architecture.
• Strong proficiency in Apache Spark (PySpark) and Python for large-scale data processing.
• Experience building batch and streaming data pipelines using modern data engineering practices.
• Knowledge of data migration, cloud modernization, and platform transformation initiatives is highly desirable.
• Strong SQL skills and experience working with large-scale analytical databases and data warehouses.
• Experience with data modeling, schema design, and data warehousing concepts.
• Familiarity with DevOps and DataOps practices, including Git-based version control, CI/CD pipelines, automated testing, and deployment processes.
• Understanding data governance, security, monitoring, and observability practices within cloud environments.
• Experience working in Agile delivery environments and collaborating with cross-functional teams.
• Strong problem-solving skills and ability to troubleshoot complex data processing challenges.