Innovation review has been an important element in my consulting work for the past decade and a half. And I have seen that it becomes trickier when it comes to reviewing big bets. In “Amazon unbound: Jeff Bezos and the invention of a global empire”, Brad Stone gives us a peek into how Jeff Bezos reviews his big bets like Alexa, Go, Fire Phone, AWS, India, and more. I found it very interesting. Here is an example of how such a review went for Alexa.
On January 4, 2011, Bezos sent an email to a few select executives including his technical advisor or TA, Greg Hart, “We should build a $20 device with its brains in the cloud that’s completely controlled by your voice.” By then Hart and Bezos would have perhaps discussed the advent of speech recognition multiple times including once in late 2010 when Hart demonstrated Google’s voice search on Android to Bezos. Soon after the email, Hart was assigned the secret project to build such a device and he moved out from his TA role.
By early 2013 the product codenamed Doppler was in beta testing mostly from a few hundred Amazon employees. The data was noisy and it wasn’t enough. The device didn’t look promising yet. Bezos kept asking, “How will we even know when the product is good?” Rohit Prasad was leading the effort in building a far-field speech recognition engine, a core part of Doppler. Prasad and Hart prepared a graph showing how Alexa would improve as data collection progressed. For each successive 3 percent increase in accuracy, they needed to double the data.
In one of the reviews with Bezos, Hart, Prasad, and team proposed to double the speech science team and postpone the launch from summer to fall. Bezos commented, “You are going about this the wrong way. First, tell me what would be a magical product, then tell me how to get there.” Then the question came whether the team had enough data. Prasad answered that they need thousands of more hours of complex, far-field voice commands.
Bezos factored in the team’s request for additional speech scientists and calculated how long the revised team would need to get the requisite data. He asked, “Let me get this straight. You are telling me that for your big request to make this product successful, instead of it taking forty years, it will only take us twenty years?” His math was correct and the team didn’t have a plan. Bezos ended the meeting abruptly say, “You guys aren’t serious about making this product.”
This review resulted in a serious introspect in the team and they came up with a plan of outsourcing the data collection. The new program called AMPED ended up renting neighborhoods first in Boston and later expanding it to 10 cities and recruiting contract workers. The rented homes and apartments were sprinkled with Alexa devices disguised as pedestal microphones, Xbox gaming consoles, TVs, and tablets. The contract workers worked eight hours a day, six days a week, reading scripts from iPad with canned lines and open-ended requests.
By 2014, the Alexa team had increased its store of speech data by a factor of ten thousand. When all this was presented to Bezos in the review, he said, “Now I know you are serious about it! What are we going to do next?”Bezos’ comment, “First tell me what would be a magical product, then tell me how to get there” would get repeated in different forms. For example, in the review of another of his big bet, India business, Bezos would ask Amit Agarwal, Head of Amazon India, “Tell me how to win. Then tell me how much it costs.” It reminded me of George Day’s innovation portfolio management framework, “Real-win-worth it” (HBR Dec 2007).