Does big data mean big $$ and big headaches for boards?

One of the hottest tickets in the technology world at present—alongside mobility and cloud—is "big data". The term is pervasive: I hear it mentioned or see it in print almost every day. Technology types—especially software companies and information consultants—are promoting big data as if it is some sort of nirvana, where all of the hassles of processing and making sense of seemingly unscalable mountains of data that pervade businesses simply go away. Consequently, many companies seem to be rushing towards expensive big data deployments. Some are ending up very disappointed.

It's true that the results of big data analytics can reveal some interesting correlations about various things of interest. The results can be helpful to decision-making, but only if you know what questions to ask. The challenge for the board is to ensure that it is clear as to why big data is important:

• How does the proposed system expedite the achievement of our agreed corporate strategy?
• What material benefits will we accrue from its adoption?
• What is the cost of not deploying the proposed system?

Boards need to ask these questions before the not insignificant cost of deploying a system is authorised. (Actually, they are no different the questions a board should ask before any major capital decision.) Even if satisfactory answers to these important questions are forthcoming, one crucial limitation remains. A Financial Times article, published earlier this year, sums it up well:

Big data do not solve the problem that has obsessed statisticians and scientists for centuries: the problem of insight, of inferring what is going on, and figuring out how we might intervene to change a system for the better.

Big data is not a substitute for critical thinking, the careful consideration of strategic options, or smart decision-making. It is all well and good to buy a system to crunch a (very large) set of numbers. So-called big data systems can be very helpful at this task. But don't expect them to make sense of the answers that they spit out. If you do, there is a fair chance that you will end up disappointed.