Azure Data Engineer

NATURE OF POSITION:

AI Fire is seeking a skilled and driven Data Engineer to design, develop, and maintain scalable data warehouse solutions and migration of data from a variety of small business ERP platforms into our NetSuite ERP system. You’ll play a key role in enabling data-driven decision making by building reliable data pipelines, standardizing data from multiple systems, data management and optimizing cloud data infrastructure.

RESPONSIBILITIES:

Design and implement efficient data pipelines using Azure Fabric, Data Factory, Databricks, and Synapse Analytics
Develop ETL/ELT processes to ingest, transform, and load large volumes of data
Create and maintain data models, Lakehouse architecture, and data warehousing solutions
Collaborate with business analysts and business stakeholders to understand data requirements
Monitor and optimize data workflows for performance and cost
Ensure data quality, security, and compliance with governance standards
Automate data validation and testing procedures
Building dashboards as needed in PowerBi
Normalizing and cleaning data

REQUIREMENTS:

Strong proficiency in Azure data services: Fabric, Data Factory, Databricks, Synapse, Logic Apps, Azure SQL, etc.
Experience with Python, SQL, Spark, or Scala
Experience with Workflow Definition Language (WDL)
Experience with Celigo
Solid understanding of cloud architecture and data lake principles
Familiarity with CI/CD tools and DevOps practices in Azure
Working knowledge of data governance, privacy, and security practices
Certifications: Microsoft Certified Azure Data Engineer Associate
Understanding of general data in ERP systems
Experience with Power BI and Power Apps
Knowledge of machine learning workflows and integration
Analytical mindset and problem-solving attitude
Strong communication and teamwork abilities
Adaptability in a fast-paced, agile environment
Experience in other platforms such as AWS data fabric (Redshift, Glue, Athena) or Databricks will also be considered
Bachelor’s degree in Computer Science, Engineering, or related field