Data Engineer

Tasks and responsibilities

• Design, develop, and optimize ETL and ELT data pipelines for structured and unstructured data;


• Integrate data from multiple sources into centralized platforms, including data lakes and data warehouses;


• Develop and maintain logical and physical data models to meet analytical, artificial intelligence, and reporting needs;


• Optimize the performance, scalability and reliability of data systems;


• Implement validation, cleansing, and monitoring processes to ensure data quality, consistency, and compliance;


• Collaborate with data scientists, artificial intelligence engineers, and business teams to deliver reliable and structured data;


• Develop reusable components, APIs, and automation scripts to optimize data flows;


• To ensure data security, privacy, and regulatory compliance;


• Document data pipelines, templates, processes, and best practices;


• Conduct a technology watch to evaluate new tools, cloud technologies and best practices in data engineering.

Qualifications

• Bachelor's degree in software engineering, computer science, or master's degree in computer science;


• Possess a minimum of six (6) years of experience in a role related to data engineering, MLOps, software development, or machine learning;


• Master Python;


• Experience with AWS or Azure cloud platforms;


• experience with Google Cloud Platform (GCP) is an asset;


• Possess knowledge of machine learning and an understanding of the associated mathematical foundations;


• Master Linux environments and containerization technologies, including Docker;


• Be comfortable with the Microsoft environment;


• Actively contribute to the continuous improvement of internal processes;


• Participate in the deployment and operationalization of machine learning solutions;


• Experience with MLOps tools and practices (an asset);


• Understanding the principles of computer networking (an asset);


• Have experience in data engineering, ETL, MLOps, pipeline development, and CI/CD (an asset);


• Knowledge of MLflow, DVC or equivalent model management solutions (an asset);


• Proficient in machine learning model monitoring tools (an asset);


• Experience with Infrastructure as Code approaches, including Terraform and Ansible (an asset);
  


• Experience in CI/CD applied to machine learning solutions or data pipelines (an asset);


• Knowledge of the R language (an asset);


• Demonstrate an interest in data architectures and cloud environment optimization (an asset);


• Certification in MLOps or related technology (an asset).