How to Model and Forecast New Products in Demand Planning
Demand forecasting for new products presents a unique challenge in demand planning. Unlike established products with historical sales data, New Product Introductions (NPIs) lack any past performance records, making it difficult to predict demand accurately. Traditional time series forecasting methods heavily rely on past sales patterns to predict future demand, but for NPIs, there is no historical baseline, rendering conventional approaches less effective.
In this article, we’ll explore how to tackle this problem using Bayesian hierarchical modeling—a flexible statistical approach that allows us to “borrow” information from similar products launched in the past. This modeling framework can adapt to different levels of product grouping (or hierarchy) and incorporate a transition curve that captures the unique adoption phase of NPIs. By leveraging information from similar products within the same hierarchy, Bayesian hierarchical models allow us to create reliable forecasts for NPIs, even in the absence of historical data. Let’s dive into the concepts, approaches, and practical steps for forecasting new product demand using this approach.
The Challenge of Forecasting New Products
For established products, demand forecasting typically involves analyzing historical sales data, seasonal patterns, and external factors. However, NPIs present a different landscape, as there’s no prior sales data for the new product. This leads to several challenges:
1. Lack of Data: No historical baseline exists for identifying trends or seasonal patterns.
2. Uncertain Demand Patterns: New products often exhibit unique adoption curves, particularly in the early stages, which makes demand highly variable.
3. Difficulty in Systematizing Expert Judgement: In the absence of data, demand planners often rely on expert judgement, which can be subjective and inconsistent.
Bayesian hierarchical modeling offers a robust solution to these challenges. By designing a hierarchical model, we can pool information across similar NPIs, account for various levels of grouping (such as product category or retailer), and model product-specific variations in a systematic way.
Bayesian Hierarchical Modeling for NPIs
Bayesian hierarchical modeling is a statistical technique that allows us to build flexible models with structured relationships across different levels of data. For NPIs, this means we can leverage information from past NPIs within the same product category or retail environment to model demand. By structuring the model hierarchically, we can estimate demand patterns that are common across similar products, while also capturing nuances unique to each new product.
Why Bayesian Methods?
Bayesian methods are particularly well-suited for NPI forecasting due to several advantages:
• Borrowing Patterns from Similar NPIs: A hierarchical structure allows the model to pool information from similar new products launched in the past within the same category or retailer. This establishes a reasonable baseline for NPIs.
• Quantifying Uncertainty: Bayesian models generate full posterior distributions for each parameter, providing credible intervals that allow us to quantify the uncertainty in each forecast—a critical feature when demand is highly unpredictable.
• Flexibility in Hierarchical Structure and Transition Modeling: Bayesian models can handle one-level or multi-level hierarchies and adapt to different grouping structures, such as product categories or retailers. Additionally, they can model the initial transition phase (or adoption curve) that is common in NPIs.
Building a Bayesian Model for NPI Forecasting with Hierarchical Flexibility
In our approach, we’ve developed a single, flexible Bayesian hierarchical model that can be tailored to different demand forecasting needs based on the product hierarchy and complexity. This model is adaptable, allowing for both one-level and multi-level hierarchies, depending on the product’s structure in the organization. It also incorporates a transition curve that captures the initial adoption phase for NPIs, which is essential for modeling demand for new products without historical data.
Let’s explore how the hierarchical structure and the transition curve work within this model.
Hierarchical Flexibility: One-Level and Multi-Level Structures
The model is designed to support different levels of product hierarchy, which allows it to handle a variety of product groupings. Here’s how this flexibility works:
1. Single-Level Hierarchy: In most scenarios, products have only a single dimension —such as product category. Hence, the model can be structured with a single grouping level which makes it simpler to implement. For example, if we’re forecasting demand for a new type of snack bar launched across multiple stores, we might only need to capture variations across stores, without further grouping.
2. Multi-Level Hierarchy: When products are organized within broader categories or regions, the model can be structured to capture multiple levels of grouping. For instance, if a company launches a new line of eco-friendly cleaning products, each product might fall under broader categories (like “kitchen cleaners” or “bathroom cleaners”) or be specific to certain retailers. In this case, the model can incorporate multiple levels of hierarchy to account for both the individual product trends and the category or retailer-level patterns.
Our model’s flexibility means we can configure it based on the appropriate grouping dimensions for each forecasting scenario. By adjusting the hierarchy levels (e.g., adding or removing group levels), the model can capture the unique demand behavior of each NPI while still benefiting from shared information within similar groups.
Role of Transition Curve & Launch Date for Capturing Early-Stage Adoption
New products often exhibit distinct demand phases, particularly in the period immediately following their launch. This early demand phase might involve a rapid increase in sales, a peak, and then a stabilization period. To model this dynamic, we include a transition curve which allows us to accurately capture the adoption phase for new products.
Each product within the hierarchy—down to the most granular level—has its own transition curve that reflects its unique adoption characteristics. The parameters of the transition curve, such as the rate of adoption, the peak demand, and the stabilization period, are modeled hierarchically. This means that products belonging to the same group, like a product category or retailer, share common patterns while still allowing flexibility to represent product-specific variations.
Importantly, the launch date of the new product acts as an additional parameter for inference. It defines when the transition curve begins—essentially the point in time when the business starts putting the product in stores. For new products, the launch date is a critical input because it anchors the transition curve, ensuring the model can accurately capture the timing and shape of the demand build-up. However, for products included in the training data (past products), the launch date is not required; their observed historical data already provides the demand trajectory.
To illustrate:
• A highly anticipated product, like a gaming console, will have a sharp, early peak in demand right after the launch date, as the transition curve kicks in immediately.
• Niche products may see a slower adoption phase, with demand ramping up more gradually over time, depending on the launch date and the dynamics of their respective product category.
By treating the transition curve’s parameters (adoption rate, peak, stabilization) hierarchically, the model pools insights from similar products while retaining the flexibility to reflect individual adoption patterns. The launch date provides a starting point for the transition function, ensuring that the model can project future demand for new products based on when they hit the market.
To further strengthen the model, informative priors are set for the transition curve parameters. These priors act as a guide, particularly when data for a new product is sparse or limited. They ensure that the model converges accurately and avoids unrealistic forecasts, while still allowing room for learning as new data becomes available.
By structuring the transition curve hierarchically, the model is able to capture both the stationary demand characteristics (long-term behavior) and the dynamic adoption phase (transition behavior) for NPIs.
How the Model Works in Practice
Our Bayesian hierarchical model, implemented using CmdStanPy (a Python interface for Stan), follows a structured workflow to provide robust demand forecasts for NPIs:
1. Define the Model Structure: Based on the product’s grouping complexity, we configure the hierarchy as either a one-level or multi-level structure. The choice of grouping dimensions, such as product category or retailer, depends on what best represents the relationships between similar products.
2. Incorporate Priors and Transition Curve: The model uses Bayesian priors, informed by similar NPIs, to guide parameter estimates. It also incorporates the transition curve, with hierarchical parameters to capture group-specific adoption patterns.
3. Fit the Model Using MCMC: CmdStanPy runs MCMC (Markov Chain Monte Carlo) sampling to estimate the posterior distributions of the model parameters, capturing both group-level patterns and product-specific effects.
4. Generate Predictions with Uncertainty: The model produces forecasts with credible intervals, providing a range of likely demand outcomes that account for the uncertainty inherent in NPI forecasting.
This workflow allows us to generate demand forecasts with quantified uncertainty, making the model well-suited for NPIs where limited data is available but reliable, data-driven estimates are essential.
Benefits of a Flexible Bayesian Hierarchical Model for New Product Forecasting
Using a single, flexible Bayesian hierarchical model for NPIs provides several advantages:
1. Reliable Forecasts for New Products: By leveraging patterns from similar products launched in the past and incorporating the transition curve, we create reasonable demand estimates for new products. The transition curve parameters allow us to model the unique adoption phase of NPIs, capturing the dynamic early-stage demand that is typical for new products.
2. Quantification of Uncertainty: Bayesian models provide full posterior distributions, allowing us to estimate credible intervals for each forecast. This is particularly valuable for NPI forecasting, where demand is often unpredictable and quantifying uncertainty is critical.
3. Scalability for Complex Product Lines: The hierarchical structure of the model allows it to scale across different levels of product complexity. Whether forecasting demand for a single level (like individual products in multiple stores) or multiple levels (such as products grouped by category and retailer), the model is adaptable to large, complex product portfolios.
4. Flexibility to Add Exogenous Variables: Bayesian methods can easily incorporate additional features, such as seasonality effects or external factors, which further refine forecasts and make the model adaptable to different contexts.
Conclusion
Forecasting demand for new products without historical data is challenging, but Bayesian hierarchical modeling provides a robust and flexible solution. By designing a single model that can adapt to different hierarchy levels and incorporate a transition curve for the adoption phase, we can create reliable forecasts for NPIs, even in the absence of direct historical data.