You have probably been seeing the term Big Data popping up inmany places. It has entered the business vernacular. Learning moreabout Big Data stimulated the following thoughts forme.

INTRODUCTION

Johns Hopkins University School of Medicine used data fromGoogle Flu Trends (a free, publicly available database) to predictincreases in flu-related emergency room visits well before theCenters for Disease Control and Prevention issued its warning.

Twitter updates tracked the spread of cholera in Haiti after the2010 earthquake as accurately as and weeks earlier than officialreports. Pharmaceutical companies use data from social media sites,such as Facebook, to track reports of side effects of their drugs,correcting formulas much more rapidly than awaiting results ofclinical trials.

Ford Motor Co. opened a Silicon Valley office in 2012 focused onBig Data, innovation and the user experience, stating, “It's timeto prepare for the next 100 years.” Walmart records well over 1million customer transactions every hour, saving them to databasesestimated to have over 200 times the information in all of thebooks in the US Library of Congress.

SO WHAT IS BIG DATA?

Big Data has moved from technology circles into the businessmainstream. But what is it? This article addresses the terms, thebusiness case and management and business issues associated withBig Data.

Big Data describes the volumes of data generated by anenterprise, including Web-browsing trails, point-of-sale data, ATMrecords and other customer information generated within anorganization, plus huge stores of data from new externalsources such as social networks like Facebook, Twitter, YouTube andLinkedIn, sensor and even surveillance data and massive public andprivate databases.

These data sets can be so large and complex that they becomedifficult to process using traditional database management toolsand data processing applications. The data are often unstructured,unformatted and unwieldy. But there can be important businessinformation ready to be unleashed; “the signal in the noise.”

Ever-improving computer hardware tools, such as virtuallyunlimited storage and continually faster processing speeds,combined with software tools such as artificial intelligence,machine learning and pattern recognition, can be applied to thesevast troves of data.

THE BUSINESS CASE

As the examples in the introduction demonstrate, Big Data hasthe potential to bring about a fundamental transformation of theeconomy. Almost no area of business activity will remainuntouched.

There is a payoff from data-first thinking. Studies show thatthe more companies characterized themselves as data-driven, thebetter they performed on objective measures of financial andoperational results. Harvard Business Review recentlyreported that companies in the top third of their industry in theuse of data-driven decision making were, on average, 5% moreproductive and 6% more profitable than their competitors.

McKinsey Global Institute, the research arm of McKinsey and Co.,has analyzed Big Data's effect on business, concluding that the useof data and analytics are going to be a basis of competition goingforward for individual firms, for sectors and even for countries.The ones that are able to use data effectively are more likely towin in the marketplace.

A NEW CULTURE OF DECISION MAKING

What Managers Must Do

Managers can measure and analyze more precisely than everbefore, thus allowing much more insight about their businesses anda better understanding of their customers. This knowledge cantranslate into more-accurate predictions, wiser decisions andstronger performance.

Managers can analyze vast amounts of data in a rigorous anddisciplined way. This enables acting on carefully distilledinformation, perhaps in place of instinct and intuition. Inessence, the manager allows instinct to be overruled by thedata.

It is critical to note that Big Data does require managementvision and human insight. There needs to be an understanding of howthe mathematics differs from reality and an understanding of thebusiness theory underlying the analytical results. Mostimportantly, the assumptions underpinning the analysis call forthorough understanding and critical evaluation.

Senior leadership identifies the strategic goals and uses thedata to make better, faster decisions. The use of the data isdirectly linked to strategy, with a goal to clarify, define andimplement it most effectively.

Challenges for Senior Management

The challenges to a business's culture can be great. A seniorexecutive team might see a shift in who is considered the “expert”.Often, the most senior person or most highly paid person in abusiness area was automatically considered the expert. With BigData injected into decision-making, the role of expert could shift.The expert might be a junior person who knows what questions to askand how to use data to get the best answers.

Senior decision makers have a duty to embrace evidence-baseddecision-making. The data-driven CEO welcomes, encourages andcreates a culture that supports it. For example, when the CEO has agut feeling about a business trend, the data might not support thatintuition. Senior executives that are genuinely data-driven willoverride their intuition when the data do not agree with it.

The culture should allow a junior manager to use the data toexplain results that differ with the boss's instinct. This requiresthat the junior manager has a thorough understanding of the dataand its relation to the business, courage, and a high level ofinterpersonal skills, in addition to a supportive culture on thepart of the organization.

UNDERSTANDING THE PITFALLS

Managers must understand the pitfalls and limitations, as wellas the potential, of Big Data. Good data scientists should in partbe pessimistic with a great concern about what the information istruly indicating, being sensitive to what can go wrong withpredictions and model designs.

Correlation is not causation. With such large data setsand well-honed measurement there is substantial risk of “falsediscoveries.” Just because elements of the data are highlycorrelated, does not mean that there is a causal relationshipbetween them; indeed, the correlation might not be meaningful atall from a business viewpoint. Management must use skill andexperience to avoid this trap.

Furthermore, any mathematical model inevitably is asimplification. Modeling is used very successfully in the physicalsciences; for example, countless phenomena are accuratelypredictable according to the laws of physics, such as the flow ofwater or the path of a rocket. This is not the case insubstantially more complex systems, such as economics and socialsystems, which are disciplines directly affecting business.

Managers must understand the observable business theory orinsight that explains the statistical inferences. Conclusions aremuch stronger and more valuable when there is this businessinsight. The numbers do not speak for themselves; managers speakfor them, giving them meaning.

It is human nature to use analysis to confirm one's own biasesand prejudices. It can be all too easy to use massive data trovesto see what management wants to see without realizing it is doingso. Big data might provide raw material for biased fact-findingostensibly based on statistics.

CONCLUSION

Big Data is a powerful tool to support smart decision-making. Itcan allow better understanding of what is most important to thesuccess of an enterprise. Managers must employ it thoughtfully,being fully aware of the obstacles to maximizing its utility,including organizational cultural issues, personal biases and“false discoveries.' With that caveat, management can realize thevalue of Big Data to better serve customers and the enterprise as awhole.

Stuart R. Levine is chairman/CEO of Stuart Levine &Associates. He can be reached at (516) 465-0800 or stuartlevine.com.