Pipeline Automation & Data Ingestion: Design, build, and optimize automated data feeds for external CPG vendor cost updates, competitor market pricing, and store spatial layout data. Modern Data Stack Governance: Manage and structure modular pipelines using dbt, Databricks Unity Catalog, and Python, ensuring high performance, proper data governance, and schema control. Complex Data Transformations: Write advanced, production-grade SQL and Python scripts to clean, structure, and stitch disparate vendor datasets into scalable data models. CI/CD & Code Quality: Enforce continuous integration, unit testing, version control, and automated deployment practices using GitHub. Cloud Architecture Optimization: Monitor and tune compute performance, data storage, and pipeline execution within AWS and Databricks.
Experience: 5+ years of hands-on data engineering experience, ideally in retail, CPG, e-commerce, or supply chain environments. Core Tech Stack: Expert proficiency in SQL, Python, dbt, Databricks Unity Catalog, and AWS data infrastructure. Automation & Version Control: Proven success building automated ingestion frameworks and utilizing GitHub for repository management and CI/CD pipelines. Data Modeling: Solid understanding of dimensional modeling, Star Schema, and building unified enterprise catalog spaces. Soft Skills: Excellent remote communication skills, proactive problem-solving mindset, and the ability to work independently in a nearshore setup. Proficiency in English: C1–C2
Soft Skills:
- Creativity and innovation.
- Attention to detail.
- Accountability and ownership.
- Time management and prioritization.
- Adaptability in high-volume environments.
- US holidays.
- 15 vacation days per year.
- Professional growth opportunities within the company.