Page 8 - Logistics News - July August 2023

AI

WAPE (lower is better) in-house ML method exhibits the best WAPE score at 21
percent and the commercial ML tool comes in second at
23 percent. This translates into a 38 percent drop in the
penalty score for the in-house ML method and a 9 percent
drop for the commercial ML tool. Both statistical methods
exhibited inferior performance on the penalty metric to the
baseline model.

The next metric brings a very interesting perspective
to the analysis. For our best-performing model, the time
Penalty (lower is better) investment is 160 hours. This translates into a highly
skilled employee being occupied for a full month. If
the penalty score could be translated into a monetary

value, the investment might be justified, but for most
organisations, translating something like lost sales into
a concrete monetary value is challenging. The time
investment also does not consider the time it takes for the
practitioner to become skilled and most companies do not
have AI practitioners on staff.
Development time*
In the final comparison table, the ML models obtained

equal scores of 6 each, followed by the in-house statistical
model at 3. The greatest differentiator between the ML
models is the time it took to develop the in-house model
versus the commercial model. For the commercial model,
a complete novice with no ML experience was able to learn
the ML tool and build a very compelling model in a single
working day’s worth of hours.

* A third place was not awarded for the ‘time’ metric as the Given that the final score is a draw, the selected
duration to develop a model in the commercial statistical method would depend on the priorities of the client. If
tool is unknown (it was done by the service provider) the penalty outweighs the importance of time, a highly
and the time taken to develop the in-house ML model is customised solution would be recommended; however, if
considerably more than the other models. a more balanced solution is desired, the commercial tool
would be recommended.
Total scores (higher is better)
Conclusion
Some view AI as a passing fad and some as a cure-all
silver bullet. From this analysis, it seems that the truth lies
somewhere in-between. Great strides are being made in

the field of ML, but there is a lot of work that still needs
to be done. Companies need to be educated on what is
available in the industry and that the advantage of ML
From the above analysis, it becomes clear that the ML is becoming more attainable every day. It is, however,

methods have a definite advantage over the statistical undeniable that AI has started to close the demand
methods. Considering the WAPE and penalty metrics, the forecasting gap. •

6 JULY/A U GU S T 2023 www .l o g ist i csn e w s .c o .z a