“New products are probably one of hardest things to forecast because they have no history, which is the basis of statistical forecasts,” says Chase.

Revolutionary products, Chase explains, are those that change the marketplace and the way we buy things. The introduction of digital cameras is a good example. “No one uses regular cameras with film anymore. There are no more one-hour film developers in parking lots. People print their own photos or just display them digitally.”

Most new products, however, are evolutionary, meaning they build off an existing product line, such as an ice cream maker’s introduction of single churn. Sales of a particular flavor of the original version can be used to forecast the same flavor in single churn, Chase says. “The new product is an evolution of an existing product, with new features added to enhance sales.”

To compensate for the lack of sales history on revolutionary new products, forecasters in the past have sifted through large amounts of data to identify a “surrogate” product with similar market characteristics to use in predicting demand. SAS has developed a patented process call Structured Judgment that improves this activity. “Today we create a product profile with the technology, hit a button and it goes out and in nanoseconds will find hundreds of surrogate products,” he says. “It brings this information into a graphical format where it can be standardized and adjusted for seasonality. That data is then used to forecast the new product in a way that enables decision making and allows continued adjustment.”

This technology also allows users to supplement structured data with unstructured data from internet sources like Facebook and Twitter, he says. “Using a tool called ‘sentiment analysis’ we can track how people feel about the new product during the launch. This data and technology were not available five years ago.”

Combining unstructured and structured data to improve forecasting is in its infancy and is one of the biggest overall challenges for users of big data, Chase says. “Combining structured, historical data with real-time unstructured data – what people are saying about the product during launch – holds tremendous promise for more successful product launches,” he says.

To view the video in its entirety, click here