Data Engineer

Job Summary

As a Data Engineer on the Enterprise Data Platform team, you will be working on a team that brings data-centric applications to life. In this role, you will be working with analytics, data science, and business partners to understand capability requirements and develop data solutions based on priorities. This is a developer role in the areas of data techniques including data access, integration, modeling, visualization, mining, design and implementation.

Job Description

Technical Skills Required:

• Databricks – Delta Lake Warehouse concepts
• Airflow – Orchestration and data pipelining
• Pyspark + SQL + Python
• Cloud Platform Exposure – ( Any cloud platform but Azure would be preferred)
• Knowledge on Medallion Architecture
• Knowledge on Data Modelling
• 2+ years of relevant technical experience working with various data engineering methodologies such as data integration and data pipelines (ETL/ELT) to activate against data at scale.
• 1+ years of experience of data modeling for analytic projects activities that include design, curation, and management of large datasets
• Experience with Apache Airflow for orchestrating and scheduling complex data pipelines to ensure reliable and automated ETL workflows.
• 1+ years’ experience developing on big data technologies with Spark and Hive, preferably leveraging such as DataBricks, AWS, and Azure equivalent technology.
• Experience leveraging RESTful web services to collect and publish data.
• Experience in software engineering development and testing life cycles using but not limited to Python, R, Linux, Java, JavaScript, Lambda, and SQL programming
• Bachelor's degree in Computer Science

Job Responsibilities

• Develops and implements end-to-end complex data projects and technical solutions through translating business requirements into technical solutions and data-flow architectures.
• Build and automates data pipelines that clean, transform, and aggregate unorganized data into data sources that are ready for analysis.
• Use expertise to apply various analytic methods to discover and interpret information about data from multiple data sources to implement analytics solutions
• Use expertise in database design to implement, operate stable and scalable dataflows from multiple marketing platforms into a cloud data lake
• Experience with data visualization tools Tableau, PowerBI, and Looker with data modeling
• Design, implement and deploy data applications and mechanisms using big data technology
• Provides subject matter expertise for multiple projects concurrently through all phases of the development lifecycle.
• Develop, enhance, govern, and administer for data platform to: collect data, transform, enrich, unify, segment, and integrate data
• Strong adherence data ethics rules around PII data sets
• Work collaboratively with IT teams, Performance Marketing team, and business leaders to ensure actionable is provided key stakeholders’
• Off hours on-call support rotation as required
• Experience with agile or other rapid application development methods a plus
• Retail industry experience a plus