Why guess and get it wrong when you could do a little experimenting and get it right?
The business pages are filled with examples of companies that have taken big hits to their brands because they’ve made marketing decisions that ran afoul of customer expectations. Take Netflix, and its aborted scheme to divide its streaming and DVD video offerings. Netflix could have avoided its embarrassing reversal if it had experimented on this decision before publically announcing the change.
My recent article* in Harvard Business Review is about how marketing departments need to adopt the “test and learn” approach through simple business experiments. Experiments are very easy to conduct: Take one action with one group of customers, take an alternative action–or often no action at all–with a control group, and then compare the outcome. The results are simple to analyze, and causality is usually clear. When the best experiments are scaled up, companies often see a powerful improvement in profits.
The Netflix debacle is a strange case. In July, Netflix announced a plan to divide its streaming and DVD video offerings, creating a new service called “Qwikster” which would deliver DVDs by snail mail. The division also came after a big price hike. But in October, after a massive customer outcry, Netflix retreated from its plan. (The price increase remains intact.)
To me, it looked as though Reed Hastings, the Netflix CEO, and his management team were just guessing. It’s a management team that seems to take pride—perhaps too much pride—in making gut level decisions, and this is one where they just horribly misjudged what their customers would tolerate.
And Netflix has the perfect opportunity to do experiments. They have separability of customers, and they can vary things remotely over the Internet. If the company wanted to direct traffic to a particular section of its website, for instance, it could easily send emails directing different customers to different URLs. This type of variation is much harder to implement in retail stores, where experiments are often about physically changing signage or merchandising fixtures.
There is no excuse for Netflix not experimenting with alternative services and pricing, and measuring the response of customers before announcing it across the board. The company learned its lesson the hard way: In the third quarter, Netflix lost 800,000 customers – the first time in years that Netflix’s US customer base shrank instead of grew.
Companies and organizations today are doing a great job of using analytics on the supply side. When designers at Ford Motor Company are devising a new steering wheel column they takes into account reams of data on performance and safety. When the management of the Oakland A’s are creating a baseball team, they do a rigorous statistical analysis of on-base percentage and slugging percentage to put the best players it can afford on the field. On the demand side, though, most companies are not looking at data at all; rather they’re just guessing at customer responses. And so often they guess wrong.
But marketing departments needn’t get bogged down in analytics – which focuses on dissecting past data – to gauge customer responses. Instead, most companies will get more value from simple business experiments. This shifts the focus from past data to future data and for most companies promises a much easier path to unlock profits from customer data.
*A Step-by-Step Guide to Smart Business Experiments, Harvard Business Review, March 2011, Eric T. Anderson and Duncan Simester
Tags: