Page 6 - Logistics News - July August 2023

AI

Conversely, ML algorithms are programmed with training dataset and generates forecasts for 2018. Forecast
statistical principles, allowing the machine to choose values are then compared to the actuals for 2018 to
between multiple options to improve upon its performance determine the accuracy of the forecasts.
without human intervention.
Comparison metrics
Pros, cons and requirements of ML Three evaluation metrics were selected for the model
Machine learning algorithms: comparison:
• Can handle vast amounts of data in a short time, for 1. Weighted average percentage error (WAPE).
example, causal datasets can be considered by the AI 2. Penalty value.
without considerable effort. 3. Development time.
• Require large datasets for learning – to produce best Each metric is compared to the forecast currently used by
results. the client to generate a matrix by which the new forecasting
• Algorithm optimisation for ML can be complex and time- methods can be compared and a simple score is applied to
consuming. each metric.

Case study Scoring legend
Traditionally, forecasting is done by looking back at
history and manually adding a growth factor to the data.
Some organisations also employ sophisticated statistical
methods. With the hype around ML and the purported
ease of causal modelling, the team at Business Modelling
Associates put together a comprehensive comparison to pit
statistical methods against ML methods. Using a set of real- 1. WAPE
world client data, we compared the following: One of the most popular measurements of forecast
• Commercial statistical tool. accuracy is the mean absolute percentage error or MAPE.
• Commercial ML tool. This method measures the size of the error (between
• In-house statistical methods (MS Excel). actuals and forecast) in percentage terms (ForecastPro,
• In-house AI methods (Python). 2019). The challenge with MAPE is that equal weighting
is given to all measurements, so a small error on a large
Background measurement would count as equal to a large error
To test the capability and robustness of the methods, the on a small measurement. Thus, WAPE was chosen as
use-case chosen for this study is a company with a unique a comparison metric as it assigns a weight to each
set of parameters: A monthly order pattern with limited measure.
SKUs and the typical South African peak periods around
Christmas and Easter. 2. Penalty value
For each technique, a penalty value is assigned based on
Input data and set-up quantified business consequences.

Historical data from 2014 to 2018 was made available for

the study. Due to significant pattern changes from 2014 to 3. Development time
2015, 2014 data was excluded from the study for the in- For each method, the time spent to develop the model is
house statistical and in-house ML models. The ML models also considered.
were set up as follows:
• Training data: 2014/2015-2017. Results
• Test data: 2018. For each tool, multiple models were developed. This
comparison will detail only the best performer for each
What this means is that the ML model learns from the tool.

4 JULY/A U GU S T 2023 www .l o g ist i csn e w s .c o .z a