Advanced Data Scientist

Responsibilities

• Design, optimize, and maintain scalable ETL pipelines using PySpark and Databricks on cloud platforms (Azure/GCP).

• Develop automated data validation process to proactively perform data quality checks.

• Employ key Databricks modules (DeltaLiveTables, Unity Catalog, MLFlow) to facilitate creating, running experiments, automating, and scheduling jobs on Databricks.

• Optimize the allocation of cloud resources and Databricks DBUs to manage and control cloud and Databricks consumption cost.

• Employ GitHub repositories to ensure that production code management best practices are being strictly adhered to.

• Build and tune ML/AI and optimization models, identify algorithmic performance improvement opportunities, and perform experiments to demonstrate incremental value delivered.

• Have frequent conversations with Business Stakeholders to understand their requirements and concerns. Explain data deficiencies, model performance/root cause analysis, and model output.

• Follow best practices in Data Architecture, Coding, and Project Management operations.

• Collaborate with cross-functional teams, such as Customers’ Stakeholders, Engagement Managers, Data Ops/Job Monitoring, Product Management & Software Engineering.

• Expand the use of analytics, ML/AI, mathematical optimization, Gen-AI/LLMs and Agentic-AI in the context of Retail/CPG business use cases such as anomaly detection, demand forecasting, price elasticity modeling, promotions features & strategy simulation, product cannibalization and halo modeling, markdown optimization, product allocation, reorder/replenishment, size and pack optimization, workforce scheduling & task optimization

Qualifications

Job Requirements:

• Minimum Education:

• Master’s degree in engineering, computer science, data science, operations research, statistics, mathematics, quantitative sciences or relevant work experience

• Minimum Work Experience (years):

• 6+ years of experience in Data Science/Data Engineering with emphasis on the full lifecycle of Data Science-ML/AI projects.

• Within that timeframe, experience is expected in: Python/PySpark, SQL, and relational or NoSQL databases, and cloud resource management.

• Key Skills and Competencies:

• Experience working with AWS, Azure, or GCP cloud environments.

• Experience implementing advanced analytics, ML/AI algorithms (such as: ,

• Statistical Time Series: Exponential Smoothing Models, S/ARIMA;

• Machine Learning: Random Forests, Gradient Boosting Methods;

• Neural Networks: TiDE (Google), DenseNet & Prophet (FB-Meta);

• Foundational Time Series Models: TimesFM (Google), Chronos (AWS)

• Mathematical (constrained linear, non-linear and network) optimization models

• Proven experience building end-to-end production grade Data & ML/AI pipelines using PySpark, Python (Pandas/NumPy) and SQL.

• Experience working with Git (or similar code management repositories) as a collaboration tool. production-grade

• Experience with orchestration tools like Databricks, Airflow (or similar tools like Snowflake, Dagster, etc.).

• Understanding of Retail/CPG industry business challenges with an emphasis on Supply Chain, Pricing/Promotions/Markdowns, Inventory Allocation, Assortment Mix Planning, Size & Pack Optimization, Retail Shrink/Fraud & Anomaly Detection, and Workforce optimization applications are highly desirable.

• Excellent verbal and written communication skills, especially as it relates to technical communications. Ability to present technical analysis to business stakeholders.

• Demonstrated ability to learn new technologies quickly and independently, particularly as technology in this domain rapidly advances. In this context, a working knowledge of GenAI/LLMs, Agentic-AI and related frameworks (e.g. LangChain) will be a plus.

• Ability to work independently with minimal supervision and achieve stretch goals in a highly innovative and a fast-paced environment.