We’ve passed into the Analyst Age without fanfare despite its importance. Like the asteroid that wiped out the dinosaurs, big data – which at its most basic, comprises data sets so large that traditional data processing techniques are inadequate – and its analysis carries with it the same destructive payload for companies who choose to continue to operate in the same, inward-looking manner they always have.

“They don’t see each other. They only see what they want to see,” to quote Cole from Sixth Sense, “They don’t know they’re dead.”

To be fair, part of this is the extreme difficulties in grappling with how to use big data when the terms of engagement can change so rapidly. It’s a nebulous term to encompass the vast swathes of information we leak without realising it in interactions with systems, particularly online. What about ‘thick data’? Or ‘data lakes’?

Relax. None of that’s important. This column isn’t to lambaste, but instead to help provide context, with the help of Gartner. Gartner conducts a regular annual survey on big data, and its latest from late 2014 reveals some interesting points. I’ve referenced Gartner before, and any supply chain company should really be reading anything the company puts out, but in particular I’d like to focus on some of the myths of big data and where you’ll likely fit in.

Firstly, if you feel like you’re being left behind, don’t. Gartner’s survey determined that only 13% of its respondents had actually managed to deploy big data implementations in 2014. Most were still in the planning and knowledge-gathering phase, with a quarter saying they had “no plans” to implement big data in some capacity (See: Cretaceous–Paleogene extinction event, Sixth Sense reference, first paragraph.)

Given the volume of big data, it’s also not surprising where companies are choosing to focus their efforts.

According to Gartner, some 79% of companies are looking at transactional data as their main focus, with log data serving as the secondary source. While this might not seem useful, performing analysis on transactional data can yield huge benefits, such as identifying the real costs of different shipping options across multiple service providers.

There’s also a belief that volume somehow makes up for quality, and while the power of big data does lie in both its quantity and variety, this ‘myth’ is one of my biggest bugbears. As a company that provides detailed analysis at multiple levels of the supply chain, the quality of data is of paramount importance – while the individual impact of any one flaw is smaller, the volume means there’s many more to deal with.

Combine it with exogenous data that is necessary to make big data work, and you have a potential analysis car wreck on your hands. Working on the quality of your data can help improve your own collection and documentation systems, which will be key when you need to leverage external service providers.

And you will. One of Gartner’s predictions is that, by 2017, more than 30% of enterprise access to big data will be done through data brokers and data intermediaries. To stay with the asteroid metaphor, data brokers are to logistic companies as Jupiter is to astral bodies – Jupiter’s mass serves as a sink for our solar system, sucking up most of the asteroids and comets that would pose a threat, and letting us get on with it.

It’s not feasible or possible for logistics companies to collect all that data when what’s important contextually isn’t immediately apparent. Arbitrary things like the weather and comments about a brand’s coolness factor on Twitter can mean the difference between increasing internal inventory stock or negotiating long-term distribution contracts with third-party warehousing providers.

Ultimately, big data is bigger than your company – to get the most benefit, you’ll need data brokers and analysts, with the ability to vacuum up huge amounts of data from various sources and provide insight into business-specific situations. Get comfortable with the idea. Start small, start with data you’re comfortable with, start with the questions you want big data to answer.

But, most importantly, get started. •