Uncontrolled: A case for experimentation in social sciences including management

Imagine a company that operates 10,000 convenient stores of which 8,000 are named QwikMart and 2,000 are named FastMart. It is observed that the average revenue per store is $1 million for QwikMart and $1.1 million for FastMart. Will the company benefit by renaming all QwikMarts to FastMarts? One way to answer such a strategic question is to build an analytical model of a typical convenient store with several variables including the name of the store as one. And then perform the analysis under different scenarios. An alternate method is to actually rename a few dozen randomly chosen QwikMart stores and test the revenue impact against another randomly chosen QwikMart stores whose name is not changed. Author Jim Manzi argues in “Uncontrolled: The surprising payoff of trial-and-error for business, politics and society” that our current methods carry a huge bias for the former (analysis) and can benefit from doing more of the latter (experimentation).

Causal density – key challenge:  Experimentation hasn’t been a popular method in social sciences especially if you contrast it with correlation analysis. A classic example is presented in the bestseller Freakonomics where the authors argue that a significant fraction of US crime reduction can be linked to legalization of abortions in 1970s. The research involved rigorous correlation analysis. However, in subsequent analysis two Federal Reserve economists found a bug in the software model and with a small change in assumptions the result shows no correlation between abortion legalization and crime reduction. Back-and-forth has continued without any conclusive result.

Building analytical models in social setting where behaviours are involved has a significant challenge. The causal density of a social setting is very high compared to physics laws applied to large objects. That means the number of variables that can impact the observed outcome can be very large, very difficult to find out and hence building a reliable analytical model is difficult. If finding all the causes is too cumbersome, why not experiment and find out? That is the view Jim Manzi presents at least for those situations where experimentation is practical.

Experimentation and business strategy: In my earlier articles, I have observed that Strategy gurus like Michael Porter and Richard Rumelt don’t emphasize experimentation. They present analytical models through which you decipher the internal and external context and create a game-plan as an outcome. And then you implement it. It has been over a quarter of a century since Rumelt-Henderson-Porter started publishing frameworks. It hasn’t worked predictably.  It is time managers consider experimentation as a complementary method to successful strategy building. Note that Manzi is not denying the role of Porter-style analytical models in creating hypothesis. He is suggesting that it is worth checking if we can test the crucial assumptions behind the strategy at low-cost quickly. And do those experiments whenever possible.

A throwaway prototype of what later became AdSense was built overnight at Google. Within a week hundred Googlers experienced and assessed the usefulness of content targeted ads.  This eventually led to creating a successful monetization model for Google. Google today performs tens of thousands of experiments on search algorithm alone every year.

Experimental revolution in business:  Manzi cites companies like Capital One which has turned business into a scientific laboratory. Every decision about product design, marketing, channels of communication, credit lines, customer selection, collection policies and cross-selling could be subjected to systematic testing using thousands of experiments. In fact, Manzi’s own company Applied Predictive Technologies is helping 30 to 40 percent of largest retailers, hotel chains, restaurant chains and retail banks in America perform repeated standardized tests on its platform.

Whether experimentation really becomes a revolution worldwide is to be seen. However, if you want to understand how experimentation is pushing the boundaries in social sciences, Uncontrolled is an excellent place to start.

Note: I am thankful to Prof. Stefan Thomke of Harvard Business School for suggesting this book to me. Thomke himself is an authority on experimentation and has written an excellent book – “Experimentation matters: Unlocking the potential of new technologies for innovation”.

Four steps where “Frugal innovation” meets “8 steps to innovation”

When times are tough, anything frugal gains currency. “Frugal innovation” is no exception. Last week I attended a Tech Event of a global MNC where “frugal innovation” was the theme. CTO of the organization presented it as one of the key elements of his innovation strategy. Is there a process for frugal innovation? Perhaps there is no single answer to this question. In this article I would like to explore one such approach where “frugal innovation” meets “8 steps to innovation” - the framework in our book.

What is frugal innovation? Again there is no single answer here. However, it generally means creating products / services which cater to the needs of masses in emerging economies (check Wikipedia). That implies innovations which are affordable and accessible to a vast underserved population. Some of the examples of frugal innovation that I have come across are:  Low-end mobile phones, GE EKG, Tata Nano, Cisco cell-site router, Embrace infant-warmer, Aravind Eye Hospital, Vaatsalya Healthcare etc.

Now let’s look at the 4 steps which enable frugal innovation:

Challenge book (step-2): How you frame the problem significantly influences the solution you create. Vijay Govindarajan identifies an interesting characteristic of such products: 50% performance at 15% cost in his book “Reverse innovation”. However, that is just the starting point. If Jane Chen and team had applied the 50%-15% rule to their incubator project, they would have created a $3000 incubator. Instead they created an infant-warmer which costs $200 (1% of traditional incubator) and looks nothing like a traditional incubator. Jane & team visited villages in India and Nepal and met people like Savitha who lost her baby because she couldn’t travel to the nearest town with incubator facility in time (watch the TED video). After talking to several mothers like Savitha the team realized that they need a local solution that can work without electricity, simple enough for a mother or midwife to use, portable, that can be sterilized and used across multiple babies, and of course ultra-low-cost compared to the traditional $20,000 incubators.

Building the context into the challenge statement is extremely important and it is very difficult to do it without experiencing the situation in these markets first-hand. Govindarajan categorizes these aspects of the challenge as various gaps such as performance gap (50%-15%), infrastructure gap (e.g. lack of electricity), preference gap (e.g. usable by mid-wife) etc.

Experiment with low-cost at high-speed (step-4): Simple looking Embrace infant warmer has already helped over 22,000 low birth-weight and premature infants. However, during the design process the team had to iterate and test the solution dozens of time by going into the field and talking to doctors, moms and clinicians to ensure that it meets the need of the local communities. For a frugal innovation, doing each experiment at low-cost and with high-speed is highly desirable. Govindarajan calls this "focus on learning based on mini-experiments".

Iterate on the business model (step-6): Traditional business models may not work in these situations. Typically several iterations are required on who (customer), what (offering) and how (to reach and monetize). Vaatsalya healthcare, a hospital chain focused on rural & semi-urban areas, experimented on two business models for six months: full-service 20-bed hospital in Gadag and consulting mode no-bed clinic in Karwar. Gadag model worked and was replicated across 17 locations. Do you rent new space or do you partner with existing practitioners? Vaatsalya had to experiment to figure out that senior doctors whose kids are not interested in father's business are the ideal partners while entering a new town. You not only get a hospital but also relationships. Iteration of this kind is inevitable during frugal innovations.

Build an innovation sandbox (step-7): For Embrace infant-warmer the phase change material (wax-like substance) that melts at human body temperature and maintains it for 4 to 6 hours is the only non-trivial technological element. However, in a product like Tata Nano or GE’s low-cost EKG, the technological complexity is much higher. Hence, dozens of testing iterations are not enough. You need thousands of experiments. That needs a dedicated team and infrastructure investment such that experimentation goes from low-cost high-speed to low-cost high-speed high-volume. And you need to have this closer to customer base so that iteration with customer happens fast. Check out sandbox stories of Tata Nano and Wright brothers. 

Govindarajan calls such a dedicated team Local Growth Team (LGT) in the context of a global MNC. Companies like GE Healthcare & John Deere created LGTs in India and China for frugal innovation.

Photo source: embraceblog.org

Why does Ram Charan say “It is a myth that innovation is expensive”?

A couple of weeks ago I got following question during a webinar to senior managers of an IT services firm: “Isn’t systematic innovation expensive?” That reminded me of what Ram Charan, a leading CEO coach, said in an interview in Economic Times, “It is a myth that innovation is expensive”. I want to explore this quote in this article. Why does Ram Charan say innovation is not expensive? 

Let’s begin with Ram Charan’s complete response. He was asked, “Is innovation expensive?” And he answered:

It is a myth that innovation is expensive. We need to consider it in three parts. Part one is sourcing of ideas, which can be done inexpensively. The second part is the conversion of that idea to the point where you can scale it up and execute it. The third part is actual execution. The middle part needs very small amount of total revenues; the large amount goes to the final part. That is a business decision, not an innovation decision. So you have to ring-fence a certain amount of money, select a few projects, focus on it, fail more and fail often.

For the sake of simplicity let’s call the three parts Ram Charan mentioned as – idea generation, incubation and execution. After having facilitated / witnessed several idea generation sessions, I can safely say that generation of ideas is not expensive. I don’t recall any session where less than 3 ideas per person were generated no matter how hard the challenge is. With the advent of Internet and collaboration tools, the cost is further reduced.

Ram Charan doesn’t consider financing the third part – execution - as an innovation expense.  Perhaps that can be debated. But for now let’s focus on the middle part – incubation where innovation efforts usually falls through. Question is: Are you ring-fencing a few incubation projects expecting them to fail often?  As Ram Charan says, you don’t need a big army of people here. Even Tata Nano got incubated with a four member team.

Resources is just one part of the story. The second part is related to design of experiments in order validate assumptions of an idea. During a workshop last month where 11 startup teams applied our “8 steps to innovation” framework, many found out very inexpensive experiments (1 day to 1 week duration) to validate some of the crucial assumptions behind their ventures. The third part is related to the rigor and rhythm of innovation review. I feel that most organizations have a long way to go in their effectiveness with which incubation projects are run.

Irrespective whether you consider execution of new ideas as part of your innovation budget, you definitely need a small part for running incubation projects and in all likelihood you will benefit from understanding the design of “low-cost high-speed experiments” (step-4 in our book) and what it means to “do the last experiment first” (part of step-8). Moreover, it will help to do effective reviews of incubation projects.

Photo source: www.ishaeducation.org

Prof. Karl Ulrich’s classification of problem types

I watched the Hindi movie Satyagraha last night whose story revolves around a grass-root level anti-corruption movement in India. As I was watching the movie, following questions came to mind, “Is corruption as a problem similar to any other problem like e.g. fixing a car or a mobile phone? Or is it different? And can such a problem be solved in any systematic approach?” The first question reminded me of a lecture by Prof. Karl Ulrich of Wharton in the coursera course “Design: Creation of artifacts in society”. In this lecture Ulrich presents a classification of problems which I find simple and useful. Here is a summary.

Design vs system-improvement problems: Problems can be divided into two broad categories: Design and System-improvement. Design creates new artifacts from nothing and system improvement problems begin with an existing operating system. For example when Ulrich built a sleeping shack in Montana (mock-up shown in the picture) he was building it where nothing existed other than a patch of ground and hence solving a design problem. On the other hand, if you are trying to reduce infection in hospitals due to increased hand washing (picture on top right) you are starting with an existing practice in a hospital and trying to improve it. Hence, it is a system improvement problem.

Selection and tuning problems: If you are selecting a new accounting system for your organization, you typically don’t build it from scratch (picture bottom left). You select one from a list of a few well-known alternatives. So there is a special class of design problems like the selection of account system which Ulrich calls “selection problems”. Similarly when you at the problem of attracting more traffic to your website, tools such a Google Analytics help you with specific parameters such as placement of words, graphics, search terms etc. It is like setting a bunch of knobs in order to identify the best performance. Landscape is known and parameters are understood and your job is to fine the combination of parameters resulting in high performance. Ulrich calls such system-improvement problems “tuning problems”.

Crises and wicked problems: Ulrich presents two more categories of problems which cut across both design & system improvement problems. The first one is – crises problems – a set of problems where time is very critical. For example, when Apollo 13 crew had to build a system to get oxygen from carbon dioxide, it was indeed a design problem. However, it had to be solved under sever time pressure and hence some solution quickly weighs much more than a great solution slowly (picture middle right). Wicked problems are problems where stakeholders have conflicting interest i.e. they disagree on the criteria of a good solution. For example, problems such as India-Pakistan conflict, improving public education or healthcare, reducing poverty and of course, corruption are wicked problems. Unlike a “fixing a car” problem, a solution to a corruption problem will dissatisfy at least one party (say the politicians who benefit from corruption).

Now that we have looked at what kind of a problem corruption is, let’s come back to the second question: Can wicked problems be solved in a systematic way? Or are there at least some good practices in solving / making progress on wicked problems? We will explore this in a separate article.

Source:

Lecture 8.2a – “Problem solving and Design” by Prof. Karl Ulrich in the coursera course “Design: Creation of artifacts in society”. This is based on chapter 2 of Ulrich’s book with the same title.