Big Data: A revolution in decision-making improves productivity

There is a fundamental change underway in the way that companies make decisions. Instead of relying on a leader’s gut instincts, an increasing number of companies are embracing a new method that involves data-based analytics. This ‘Big Data’ revolution is occurring mainly because technology enables firms to gather extremely detailed information from and propagate knowledge to their consumers, suppliers, alliance partners, and competitors.

Companies that use this type of ‘data driven decision making’ actually show higher performance. Working with Lorin Hitt and Heekyung Kim, I analyzed 179 large publicly-traded firms and found that the ones that adopted this method are about 5% more productive and profitable than their competitors.  Furthermore, the study found a relationship between this method and other performance measures such as asset utilization, return on equity and market value. There is a lot of low-hanging fruit for companies that are able to use Big Data to their advantage.

Where does the data come from? In part, it’s due to the widespread diffusion of technology that captures vast amounts of information such as Enterprise Resource Planning, supply chain management and customer relationship management systems.

There are multiple sources of this data outside of companies as well. Mobile phones, vehicles, factory automation systems and other devices routinely generate streams of data on their activities. In addition, manufacturers and retailers use RFID tags to track specific items as they go through the supply chain.

Clickstream data and keyword searches from Websites also provide great insight into what customers care about without having to set up focus groups or customer behavior studies.

For example, every month there are over 100 billion Google searches and each search is someone expressing an interest in or demand for something. We’ve used that data at the MIT Center for Digital Business to accurately predict housing sales up to three months in the future for each of the 50 states in the U.S.

This data also can be used to make accurate predictions for a variety of products and services including appliance sales and automobile features. It’s the world’s biggest focus group.  Google searchers have even allowed researchers to accurately predict flu season trends well before the Center for Disease Control issued its own reports.

Leading-edge firms don’t just passively collect this data. Instead, they actively conduct experiments to develop and test new products. Companies such as Amazon, eBay, and Google are good examples of successful firms that rely heavily on field experiments as part of a system of rapid innovation, using high visibility and large amounts of online culture experimentation. That culture has spread to other information-intensive industries like retail financial services, retail, and food services.

While this Big Data revolution is growing, the majority of companies are still relying on traditional methods to make decisions such as hunches and intuition. My colleagues and I at the MIT Center for Digital Business are getting more and more involved in studying the economics of Big Data and we welcome your input and advice. I’m also working with Alex ‘Sandy’ Pentland of the MIT Media Lab to teach a workshop on this topic. We’ll cover what Big Data is, how to get it, and how to use it to an organization’s advantage. Join us if you can.

Prof. Erik Brynjolfsson is director of MIT’s Center for Digital Business and coauthor of the paper, “Strength in Numbers: How Does Data-Driven Decision Making Affect Firm Performance?” He also is teaching in the upcoming MIT Sloan executive education program, Big Data: Making Complex Things Simpler on March 27-28.

Tags: