.Demand prediction in retail with Amazon Forecast

As businesses grow, forecasting becomes an increasingly important task. Imagine an electronic store having a shortage of goods during Black Friday, that would be a huge amount of income lost. Accurate forecasting can help companies optimise inventory, manage supply chain logistics, and increase profitability. Amazon Forecast is a machine learning service that can help businesses predict future demand with high accuracy. In this blog post, we'll cover the basics of using Amazon Forecast for demand prediction, look at what-if analysis and possible MLOps solution.

Setting up Amazon Forecast

As with every other service in AWS, there is always an option of setting it up from AWS Console, but if there is a need for automation, it is also possible to use Infrastructure as Code solutions, like CloudFormation or CDK. The automation will take additional time to set up, but will make a solution more reusable and less error-prone. I, personally, prefer familiarising myself with any AWS service through AWS Console first - it gives better overview and understanding of basic features.

Data preparation

Our first step is to gather historical data on past demand for your product/service and structure it into datasets. Data collection and cleaning is a huge topic in itself that we will not cover fully in this blogpost, but the general rules are:

• Dense data will create more accurate forecast (as less zeroes/nulls for data values as possible)
• Latest data is better than old data
• Data has to be clean (no duplicates, no incorrect data formats, etc.)

Our example case is big retail store that sells all kinds of products: electronics, groceries, sport goods, etc. There is a data for the last 30 days of sales available, that we've cleaned before further processing.

Data import

Now it is time to work with Amazon Forecast. First we have to create an S3 bucket, that will store all our data. After the bucket is created, we have to import our datasets (they can be stored either in CSV or PARQUET format). There are 3 types of datasets that you can import into Amazon Forecast:

• Target time series (required)- for this dataset your training data has to be time series and it has to include the field that you want to generate a forecast for. This field is called the target field. In our case (prediction of demand) the target field is called demand and the structure of target time series dataset's looks like this:

• Related time series (optional) - includes historical time-series data other than the target field, such as price. Because related time series data must be connected to target time series data, each related time series dataset must contain the same identifying fields. In our case, these would be item_id and timestamp. A related time series dataset might contain data that refines the forecasts made with target time series dataset. For example, you might include price data in your related time series dataset on the future dates that you want to generate a forecast for. This way, Forecast can make predictions with an additional dimension of context. Here is the example of related time series dataset's structure for our case:

• Item metadata (optional) - choose this dataset type when your training data isn't time-series data, but includes metadata information about the items in the target time series or related time series datasets. For instance, in our case, an item metadata might have a category, color and brand as attributes. Item metadata is especially useful in cold start forecasting scenarios, in which you have little direct historical data with which to make predictions, but do have historical data on items with similar metadata attributes. When you include item metadata, Forecast creates cold start forecasts based on similar time series, which can create a more accurate forecast. An example of item metadata dataset's structure for our case:

Importing each of these datasets into Amazon Forecast takes around 30 minutes, however it is possible to run these jobs in parallel and speed up the process.

Training a Predictor model

This is where the real magic happens! Amazon Forecast uses advanced algorithms to train a model based on configurations that we specify in this section. The most notable options are:

• Forecast frequency (required) - a numeric value for forecast frequency, supports following time-frames: minutes, hours, days, weeks, months, year.
• Forecast horizon (required) - a numeric value specifying how far into future should the forecast be generated
• Forecast quantiles (optional) - a numeric value between 0.01 and 0.99, or mean. Quantile of 0.90 means that the actual value of demand (in our case) would be lower than a value on a dashboard with 90 percent confidence and higher than the value on a dashboard with only 10 percent chance.

This is the most time-consuming part of interactions with Amazon Forecast. The job could take 2-3 hours, depending on amount of data.

Creating and using Forecast

Now that we have datasets ready and predictor model trained, we can create a Forecast. For that we just have to pick a name and choose a predictor, that we created before. After that there are 2 options: we can either query newly-created forecast using dashboards in service itself or export the forecast to Amazon QuickSight. In my opinion, the last option feels more refined and feature-rich.

Integrating Amazon Forecast and QuickSight

The setup is quite simple. We just have to export our forecast to S3 bucket and then reference it during analysis creation in Amazon QuickSight. After the initial steps are done, we can get a forecast dashboard for a specified period of time, either for all items or for a specific item. The dashboard below shows a demand prediction for a single item (coffee) for 30 days period:

The forecast will give you transparent and accurate prediction of future demand for a specific item.

What-if analysis

We can go even further and implement what-if analysis. Using this insight we can go through additional scenarios and see how the demand will change, if the target value is different from the predicted.

Implementing automation

As I mentioned before, automation takes time but will pay off eventually. Below is a sample architecture that will do most of the job for you: from data preparation to forecasting and evaluation. Setup is implemented using CloudFormation.

If you would like to dive deeper in Amazon Forecast for retail, please follow this link.

If you are interested in various forecast-samples from Amazon, please follow this link.

Conclusion

Demand forecasting is crucial for businesses of all sizes. The applications of Amazon Forecast are vast and varied, ranging from demand and financial planning to resource allocation, energy management, and weather predictions. This powerful tool not only helps you generate predictions with great accuracy but also enables you to conduct what-if analysis to assess the potential impacts of various business decisions. By setting up a project, training a model, and harnessing the forecast generated by Amazon Forecast, you can streamline your business operations and secure a competitive edge.