3Cs of idea communication illustrated through Andrej Karpathy’s LLM talk

Communicating your idea effectively is important be it to customers, team members, or investors. We presented 3 attributes of idea communication – curiosity, concreteness, and credibility in our book “8 steps to innovation”. In this blog, I would like to illustrate the 3Cs using Andrej Karpathy’s talk “Intro to large language models” which he published on his YouTube channel.

Andrej Karpathy is one of my favorite teachers in the deep learning area. The OpenAI founding member and ex-director of AI at Tesla has a hands-on approach to teaching involving Python, Pytorch, and technical papers. Hence, I was surprised when Andrej uploaded a PowerPoint presentation on his YouTube channel. I was familiar with half the information in the talk. And yet, there was a lot I could learn from the way Andrej presented. It is an excellent example to illustrate how 3Cs – curiosity, concreteness, and credibility improve the effectiveness of a presentation. Let’s look at each C one by one.

Curiosity: A good presentation not only makes you curious early on, it keeps you engaged by maintaining a curiosity flow. What does curiosity flow in Andrej’s talk look like? He begins with the question, “What is an LLM?” (21 min), then he moves on to the second part, “The promise and future directions of LLM” (17 min), and in the third and last part, Andrej talks about “the challenges in LLM paradigm” (13 min).

Within each part, Andrej is maintaining a curiosity flow. For example, while explaining what an LLM is, Andrej asks questions like “How do we get the (neural network) parameters?” “What does a neural network do?” “How do we obtain an assistant?” etc. While presenting the future directions in LLM research, Andrej explains problems like – what is equivalent of system-2 thinking? Or how do we get tree search in chess to language? How do we create a self-improvement sandbox environment for LLM like how it happened for AlphaGo? And, in the final part, he shows how different jailbreaks like “prompt injection” “data exfiltration” or “data poisoning” pose a security challenge for an LLM. In short, it helps to build a curiosity flow while designing an idea presentation.

Concreteness: Large Language Models are high-dimensional and abstract and as Andrej alludes to in the talk, how they work is not fully clear. Hence, it makes sense to use lots of concrete examples to make the concept understandable. And that’s what Andrej does. In many places, he shows how LLMs respond in certain situations by showing how ChatGPT behaves when you prompt it in a particular way. For example, he illustrates the “reversal curse” by showing how ChatGPT answers the question “Who is Tom Cruise’s mother?” correctly while saying “I don’t know” when asked, “Who is Mary Lee Pfeiffer’s son?” He gives a demo of how LLMs use tools like browser search, calculator, and Python libraries to solve a given problem and present the information as a plot. 

Andrej also uses several metaphors or analogies to explain concepts. For example, he says an LLM is like a zip file of the Internet, except that it is a lossy compression. Or, LLM is not like a car where you can understand and explain how different parts work together to give its function. Or, current LLMs are like speed chess which uses an automatic and fast system-1 mode of thinking, and while it is yet to learn how to solve problems like competitive chess where players use deliberate, slow, system-2 mode of thinking involving tree search.

My biggest takeaway from the talk comes in the form of a metaphor when Andrej explains that it is better to think of an LLM as the kernel of an emerging operating system (like Windows or Linux) rather than as a chatbot or a bot generator. To explain this, he maps various components of current OSes to LLM components. For example, he says the Internet is like the hard disk in the traditional OS, and context window is like the working memory or RAM, etc. I thought it was a powerful metaphor to convey the paradigm shift.

Credibility: Most idea presenters like you and me need to worry about making our ideas credible. Given his position and brand and given the popularity of LLMs, Andrej probably doesn’t have to pay special attention to this aspect. However, he is making forward-looking statements in this talk, and he needs to ensure he doesn’t divulge any information confidential to OpenAI. He achieves this by citing academic papers while mentioning future directions and security challenges. His demo also adds to the credibility. He is not making any “AGI is around the corner” kind of hyperbolic statements and devotes time to talking about the limitations and challenges of the current LLMs.

I hope this illustration helps one to see how the 3Cs - curiosity flow, concreteness, and credibility help in designing better presentations.

8-steps after 10-years: Why bother building participation?

It has been 10 years since the publication of our book “8-steps to innovation”. During this time, we got the opportunity to share the framework with various leaders. We also saw the framework being put into practice. Through this series of reflections, I will try to shine a light on situations where the framework might be weak. In this article, I will question the third step – building participation.

The 8-step framework suggests that if you want to build an idea pipeline, create a challenge book first (step 2) and then involve people in participative problem-solving (step 3). For example, if you have a brainstorming session around a specific challenge, you end up generating several ideas. Here are situations where this may not work.

Inventor’s challenge: I have met several inventors who prefer to keep their ideas to themselves. They have no interest in sharing it with their boss or colleagues because they feel they may steal their ideas. Perhaps they have had a bad experience in the past where others took credit for their idea. In my workshops, when we have brainstorms, these people suggest some ideas. And then meet me during the break or send a message to share their pet idea which they are not comfortable sharing with others. I understand the importance of secrecy until you have some form of protection. However, sometimes people end up carrying ideas with them for years without any form of validation. It helps to have a few sounding boards. Is ChatGPT a good brainstorming partner? Perhaps.

Manager’s fear: A few years ago, I wrote about 3 reasons why managers don’t throw their toughest challenge to their teams. The single biggest reason is fear of perceived incompetence. They feel they get paid to solve problems and if they share their challenge with the team, they might be perceived as incompetent. It gives confidence when you solve a problem and get your team to implement your solution. This works best when you have been in the system for a long time and know the domain very well. However, when you start managing an existing team, you may not be the domain expert. And this approach may not work.

Why care about small ideas? Continuous improvement as a systematic approach has been around for over a hundred years. Our book presents stories from Toyota, Titan, and TVS. Many organizations continue to highlight the number of ideas and the number of employees participating in continuous improvement programs in their annual reports. For example, the Asian Paints FY23 annual report says that there were 7000+ improvement suggestions submitted. Having said that I have met several leaders who don’t consider continuous improvement worth the cost. What matters to them are big bets. As a result, participation becomes unimportant. Participation thrives when small ideas are encouraged.

Participation in virtual teams: Virtual teams have been around for a while but their presence increased during and post-Covid era. As people started working from home, formal brainstorms and tea-coffee chats diminished. As video calls started taking time, initiatives focusing on not-so-urgent issues took a backseat. And participation in innovation-related initiatives went down. Getting people to participate in anything other than deliveries became a challenge at least in some organizations.   

In short, participation may be one way of building an idea pipeline. However, there are situations in which participation may not work or one may be uncomfortable sharing the ideas. Perhaps ChatGPT is your partner. Problems may be defined and solved by individuals and implemented through teams if they are the managers. So yeah, skip step 3 if you don’t need it.

Three enablers of innovation stamina

“We used to be innovative before the pandemic”. It is not uncommon to hear this in the corporate world when I visit clients now working in hybrid mode. That is not surprising if we look at innovativeness as a kind of stamina. It is similar to saying “I used to run 10K comfortably once upon a time”. Building and sustaining stamina is a sweaty process and needs discipline. If you don’t practice, your stamina goes down. Going to the gym helps or having a trainer or a buddy to exercise with help. Analogously, what kind of enablers help build innovation stamina? Let’s look at 3 of them in this article.   

Dashboard and review: Smartwatches have made parameters like the number of steps per day easily accessible. A gymgoer may watch a number of push-ups and pull-ups, bench-press weights and repetitions, etc. A dashboard makes a big difference when it comes to stamina building. However, we need to differentiate between a goal of running 10K in under 1 hour and doing 6K runs three times a week. So, a good dashboard should have an outcome goal (e.g. run 10K in under 1 hour) and a process goal (e.g. run 6K 3 times a week). Similarly, it helps to have outcome goals related to innovation stamina such as idea pipeline (no of ideas, ideas per person per year, no of big bets), idea velocity measured through experiments and customer validations, business impact measured through savings, revenue and profit, and participation measured through percentage of team members participating in innovation activity, etc. And it helps to have process goals such as the number of brainstorms, number of challenge campaigns, number of hackathons, etc. I have presented a few examples of dashboard parameters I gathered from annual reports here and also presented process goals here.

A dashboard without a review is of limited use. Hence, organizations need to review the innovation dashboard with rigor and rhythm (e.g. quarterly). This is where tough questions get asked and budget allocation / re-allocation happens. Here is an example of how Jeff Bezos reviews a big bet like Alexa and another one on how innovation reviews happened at P&G under A G Lafley.

Gyms and coaches: As I go out to jog in the morning, I see many people carrying their gym bags and heading for a workout. For many, a gym and perhaps a coach make a difference in bringing discipline to their stamina-building process. For innovation, gyms come mostly in the form of laboratories. There are different types of labs. For example, a tinkerers’ lab may house various tools for wood-cutting, metal-cutting, circuit-building, CAD modeling, 3D printing, etc. under one roof. Alternately, a technology-focused lab may focus on technology like quantum computing, IoT sensors, AR/VR, nano-materials for water purification, etc. A design studio creates space for using various materials and tools for prototype designs.

An innovation sandbox also has high experimentation capacity built through a lab but in addition, it has constraints coming from market use cases, cost, and product performance. For example, when the Lego company decides to create a center focused on creating a bio-plastic for building lego bricks, it is working under various constraints like malleability of the material, ability to hold paint, cost, and perhaps a few more.

A gym is far more effective with coaches and it welcomes newcomers and trains them. Likewise, a lab is more effective when there are coaches/mentors for newcomers.

Events and celebrations: Many runners get motivated when they decide to participate in an event such as a 10K run or a marathon. They form groups and practice together for months for this event. While such events are competitive for many, for most people the cooperative spirit may dominate the practice.

Companies also organize events related to innovation that instill the spirit of competition and cooperation. For example, there are day-long or week-long events showcasing promising ideas or prototypes. There are events like Innovation Day/week, Engineers’ Day, technology conferences, hackathons, challenge campaigns running over a month, etc.

Newsletters socialize these events. They showcase not only the winners but also the participants helping each other. Events generate stories that are discussed over lunch and they may motivate skeptics to participate in the next event.

Events can lose steam if they turn into just social events. The ability to spot good challenges, ideas, and prototypes and convert them into proposals, papers, and formal projects is important.

To summarize, we looked at three enablers of innovation stamina: dashboard and review, gyms and coaches, and events and celebrations.

Related blogs:

4 stamina of an innovator, Aug 27, 2015

Starting an innovation initiative: An ABCD approach, Sep 25, 2015. The enablers mentioned in the above article could be seen as an extension of the ABCD approach with an E for Enablers.

Is “8 steps to innovation” still relevant in the digital era?

It has been nine years since the publication of our book “8 steps to innovation: going from jugaad to excellence”. In a fast-paced world where technology becomes obsolete every two-three years, nine years is a long time. My co-author Prof. Rishikesha Krishnan has been nudging me and suggesting that we should re-look at the framework, especially in the context of the digital era. Is the framework still relevant? Here is an attempt to sketch some initial thoughts on this topic. The attempt is clearly biased and criticism is more than welcome.

Relevance of pipeline-velocity-batting average:  The framework addresses the question, “How to become more innovative systematically irrespective of strategy, size, sector, and culture?” This question is more relevant for organizations and teams and less relevant for individuals. The framework divides the main question into three sub-questions: How to build an idea pipeline? How to improve idea velocity? And how to enhance batting average? The pipeline problem addresses the generation of a constant stream of business-relevant ideas. Velocity problem explores validation of various assumptions associated with the ideas and finding relevant resources including investors for the promising ideas. Batting average problem looks at increasing the chance of success for big bets while building a margin of safety. 

Are pipeline, velocity, and batting average problems still relevant in the digital era? I have been presenting these sub-questions to MBA students and corporate executives. And nobody has questioned the relevance of any of these sub-questions. These questions were relevant when Thomas Edison was running his invention factory more than a hundred years ago and are relevant for the innovation engine at Mahindra today. Even a corner grocery shop if it plans to do systematic innovation would have to address these questions. So then, what has changed?

The eight steps are responses to these three questions. First three steps address the pipeline problem, the next three steps address the velocity problem, and the final two address the batting average problem. Let’s see how the relevance of each step changes in the digital era.

Pipeline problem: (Step-1, 2, 3)

Step-1: Laying the foundation: This step involves setting up the core processes like idea management process, buzz creation process, and learning and development process. It also involves establishing clarity on the scope, source, and sponsorship of innovations. I feel these things are not affected in the digital era. Any organization that is serious about innovation has these in place in some form or the other.

Step-2: Create a challenge book: This step emphasizes the creation of a challenge book and establishing collective clarity around it. The digital era has created new metaphors like Uber (marketplace), Tesla (EV, semi-autonomous, over the air upgrades), Zomato (home delivery), Amazon (shopping convenience), Paytm (mobile wallets), etc. In the past few years, we have seen new waves like the pandemic, sustainability-related regulatory norms, electric vehicles, cryptocurrency, machine learning, etc. gaining momentum. All these metaphors and waves contribute to building a challenge book. However, in my opinion, the relevance of challenge book doesn’t go away. In fact, it becomes more relevant because in a world of ever-increasing distractions, challenge book can bring focus to the innovation efforts.

Step-3: Build participation: This step assumes that navigating complex challenges may benefit from participation, the way it happens in a café or a conference. The assumption remains relevant in the digital era. However, the digital era highlights the importance of the customer experience dimension.  With steps like search, discovery, comparison, selection, payment, delivery, and returns associated with online shopping, end-to-end experience has become increasingly important. Moreover, this cuts across the shopping of products like mobile phones, grocery items, and services like blood testing and banking. Hence, a methodology like design thinking which puts experience design at its center and weaves empathy, participative problem solving, and experimentation in an iterative manner has gained significance.

Velocity problem: (Step-4, 5, 6)

Step-4: Experiment at low-cost with high speed: The digital era has seen the emergence of new tools – computational modeling tools, simulators, 3D-printers, etc. Many of these tools are now available on cloud making them easily accessible at low-cost. They are helping idea authors test their ideas or at least some assumptions associated with their ideas with less cost and at high speed. For consumer-facing digital applications, A/B testing – a form of randomized controlled experimentation – has become an important mechanism for testing ideas. No matter what the technology or tools, the relevance of low-cost high-speed experimentation hasn’t diminished over the years.

 Step-5: Find a champion: This step is based on the assumption – An idea either finds a champion or dies. Is the assumption still valid? Very much. Finding a strategic customer who endorses the idea or an investor who supports the development of the idea continues to be important today. Social media has helped ideas authors find champions by publicizing their idea through videos. Programs like Shark Tank are creating platforms for start-ups to find investors and/or mentors.

Step-6: Iterate on the business model: As the relevance of data increased, so did the importance of business models that leverage data. Dental insurance company Bento partnered with Philips which manufactures electric toothbrushes. This is because having the data on how many times a person brushes his teeth would help determine his dental insurance. EV companies like Ather Energy began to unbundle their product offering and started selling batteries separately as a subscription. Banks began to offer Buy-Now-Pay-Later (BNPL) payment option as an alternative to credit cards. Business model innovation continues to be an important lever for digital businesses.

Batting average problem: (step-7, 8)

Step-7: Build an innovation sandbox: Exploring big bets is inevitable for any company that is serious about survival. Google explores self-driving cars, Amazon experiments with Just-walk-out stores, and Facebook bets big on virtual/augmented reality. Small firms may have to consider automation and analytics seriously. The challenge is you can’t bet on all the big trends, you will have to choose. And even after choosing a trend, you may not know how this trend may lead to a new offering. You need to identify a few use-cases, invest in building experimentation infrastructure, and perform a large set of experiments to see what is both meaningful in your context and promising enough. In short, you need to build an innovation sandbox, unless you choose to acquire the innovation. Building an innovation sandbox is neither low-cost nor a short-term project. Technology platforms may speed up the process and open innovation may help in connecting ideas from remote corners of the world.  I haven’t seen its relevance diminished.

Step-8: Build a margin of safety: Big bets bring risky exposures. You can’t have one and not the other. Datacenter outages are a given once you adopt the cloud. If you are a bank and if you don’t worry about managing data center outages you will be in trouble sooner or later. HDFC Bank learned it the hard way. Shakespeare knew that a pound of flesh is a risky promise for the Merchant of Venice. And V G Siddhartha, the founder of Café Coffee Day was expected to know how much debt is enough. This step – building a margin of safety – could very well be the most challenging step to internalize. And it is evergreen.

In short, from my biased perspective, the core problems raised in the book – pipeline, velocity, and batting average are still relevant in the digital era. And 8-step responses are relevant too. However, your input is welcome and it is possible that I am missing something here.

A response from Mark Jarvis on "Defining characteristics of five levels of innovation maturity"

Mark W. Jarvis is an innovation guru at UNO MINDA Group and Head of International Business Development at INITIA Design Studio. We met at an innovation management session at IIM Bangalore a few years ago. He has been extremely kind to give constructive inputs on my blog "Defining characteristics of five levels of innovation maturity". He has separated the characteristics into two categories - one for innovation managers and practitioners inside the organization and the other for stakeholders and innovation benchmarking outside the organization. Moreover, he has expressed his concerns on some of the metric parameters like the number of patents that bother him and suggested better parameters. Thanks a lot, Mark. Appreciate your input.

----------------------

Hello Vinay,

I very much like your “Defining Characteristics of Five Levels of Innovation Maturity”.

I think it is important for any defining characteristic to be “easily” obtained … and distinguished by availability to the “inside the corporation” innovator or to the “outside the corporation” stakeholder.

Since I am always thinking of how to turn things into charts for PowerPoint slides … here is a chart summarizing your thoughts … with some added from me.

	Inside Corporation   (for Innovation Managers and Participants)	Outside Corporation   (for Stakeholders and Innovation Benchmarking)
Level 1     Ad-hoc      No Tracking	Likely an authoritarian workplace riddled with confusion, panic, fear, or lethargy. 😉	No Published Evidence Of Managed Innovation    (or Evidence Of Sustained Poor Market Performance)
Level 2     Foundation     Innovation Dashboard In Use     Pipeline, Velocity, and Impact (Process and Outcome) Indicators	Company-specific Innovation Metrics are gathered and reviewed by management and shared with innovation community.	Annual Report (R&D spend, Number of R&D centers, Number of patents, Number of new SKUs, Number of new product launches, etc.)
Level 3     Engaged     Rigor and Rhythm of Innovation Review	Rigorous Innovation reviews (feasibility, desirability, scalability, viability, etc) with consequences (for example, projects de-funded or extra-funded) that are well-communicated.    Observe who attends, clarity of strategic challenges, innovation champion support, and resource re-allocations.	?
Level 4     Aligned     Innovation Sandbox	Innovation Sandbox(es) that is focused by a strategic challenge area, has experimentation capacity, dedicated (politically protected) resources and patient management.	Press Release or other communications about Innovation Sandbox.     (Note – it is difficult to gauge the quality of the Sandbox effort from outside the corporation)   Another type of “Innovation Sandbox’ is a well-managed Outside Innovation effort.
Level 5     Excellence     30%+ Revenue from Innovations in the Last Five Years	Internal Innovation Dashboard metrics will explicitly include tracking of Revenue from Innovation in the Last Five Years and related long-view systemic innovation metrics.	Annual Report Revenue Contribution (or Growth Contribution)

Additional thoughts …

About counting the number of patents … this metric bothers me a bit … as I have seen it misused or misunderstood … but I agree it is a good place to start. As a quantity metric it is easy to measure and communicate. Much better would be a patent strategic quality measure … that tracks the business impact of a patent … or tracks the size of patent space covered … or tracks the quantity/quality of patents that reference your patent. But these quality metrics are not easy to come by … or are easily manipulated. I am optimistic that future innovators will have access to affordable software/analyzers to guide and assess a company’s intellectual property portfolio (including patent mapping (Clarivate’s Innography, etc) and patent valuation (Anaqua’s AQX, etc)). Note – one exception to my thinking on this metric comes to mind … perhaps the USA pharmaceutical industry … which seems to extract huge margins through the sheer quantity (sometimes hundreds) of slightly differentiated patents for a key pharmaceutical product.

About Innovation Sandboxes … It is extremely important that sandboxes be politically protected by patient and enlightened management. Typically, there are corporate jealousy, lack-of-cooperation, and unreasonable payoff time behaviors that distract and destroy Sandboxes. For “outside the corporation” folks … I added Outside Innovation as a “type of Sandbox” … since a company’s Outside Innovation efforts (by definition) are readily observed (Maruti-Suzuki’s MAIL, etc.).

About Level 3 Defining Characteristic To An Insider … In addition to robust Innovation Reviews, innovators will notice that leaders will innovate by walking around ("What new thing did you explore this week?") and have pleasant discussions about smart failure.

About Level 3 Defining Characteristic To An Outsider … I am wondering what this could be … to a company stakeholder (investor, financial journalist, innovation benchmarker, etc.)?

I am happy to know your (and others) additional thoughts.

Mark Jarvis

Innovation maturity, level 4 challenge and sandbox hesitancy hypothesis

My co-author Prof Rishikesha Krishnan and I have been presenting the innovation maturity mirror (see the picture above) to executives for close to 8 years. We feel many organizations that begin their innovation journey reach level 3 and then struggle during the next leg – to reach level 4. In this article, I would like to first spell out what the level 4 challenge looks like through the innovation maturity mirror. And then propose a hypothesis that hesitation to build a sandbox in a strategic opportunity area could be at the heart of the challenge.

What’s the main difference between level 3 and level 4? Level 3 indicates that the organization is engaged in experiments and reviews. 1 in 10 ideas get prototyped, incubation pipeline gets reviewed quarterly and 1 in 3 employees participate in innovation activities. That is, if you walk around the corridors or shop floor, innovation is palpable. Level 4 indicates that, in addition to the above things, the organization now has a balanced portfolio of small, medium, and large impact ideas. On average, the organization generates 1 idea per employee per year, and projected impact of the big idea pipeline is greater than 10% of the revenue and there is at least one operational innovation sandbox.

The run rate of 1 idea per employee per person is not easy to sustain. However, with the momentum of the experimentation and challenge campaigns, it is achievable. It is also not difficult to generate big impact ideas. Ask this question in a leadership meeting and you could get a few ideas. Your favorite search engine could also help you get a few. Things get tricky when the rubber meets the road for the big ideas – a champion, typically a CXO, taking a strategic bet, allocating resources, building experimentation infrastructure, put a dedicated team around a focus area. Sandbox hesitancy hypothesis says that organizations either hesitate to set up a sandbox or don’t give enough attention to it.

Organizations tend to be secretive about their sandbox setup and that is understandable. Amazon was secretive about the Kindle effort and Apple was secretive about iPhone. But many times organizations end up acquiring new resources – people/technology to build the sandbox. For example, in 2007 Google acqui-hired a team for starting its self-driving car sandbox which eventually became Waymo. The VueTool team was working on a digital mapping project and some of its members had won 2005 DARPA Grand Challenge related to robotic self-driving cars. Mahindra acquired Reva to strengthen the electric automobiles sandbox and Flipkart acquired an AR/VR company Scapic last November. In all these cases, in all likelihood, there was a champion at the top level (Bezos for Kindle, Sergey Brin for self-driving car, etc.)  

Is innovation sandbox applicable only for large companies? I don’t think so. I feel that even an SME would need to build a sandbox with all its characteristics – a champion, focussed challenge area, experimentation infrastructure, dedicated team, and failure protection.

If the sandbox hesitancy hypothesis has any merit, then a number of questions can be asked. Why do organizations hesitate to build an innovation sandbox? Is it a lack of ideas? Or lack of confidence? Or lack of clarity on the strategic bet? Or lack of resources? Or lack of urgency? Or lack of sandbox management experience?

I and Rishi hope to explore these questions in the coming months. If you feel you have some useful input, please let us know.

Doing the last experiment first: illustrated through Alex Honnold’s El Capitan free-solo

Last month, Reserve Bank of India issued an order to HDBC Bank stopping all launches of the digital business generating activities planned under its program Digital 2.0 and sourcing of new credit card customers. Reason? HDBC Bank suffered major outages in Internet banking and payment system due to a power failure in the primary data centre. These are temporary restrictions but such incidents could damage company’s brand. Question is: are such data outages avoidable? And could “doing the last experiment first” be helpful in such situations? Let’s explore these questions in this article.

“Doing the last experiment first” is one of my favourite levers of building margin of safety. We have mentioned the concept in our book “8 steps to innovation” and we borrowed the term from A. G. Lafley, ex-CEO of P&G. Doing the last experiment first involves validating the leap-of-faith assumption associated with an idea. What is a leap-of-faith assumption? An assumption that is (a) critical to the success of the idea, and (b) there is very little evidence available to support it. How does Alex Honnold’s El Capitan free-solo illustrate this concept? Let’s look at it next.

Alex Honnold is an American rock-climber. In 2017, he became the first rock climber in the world to free solo 3000-foot wall of El Capitan in California. If you want to get a feel of what that means, check out this 5-minutes video showing Alex’s free-solo climbing scenes. To us Alex’s endeavour appears almost like a suicide attempt. And Alex says the same thing in his TED talk, “Seems scary? Yeah, it is” (1:13). However, he says something strange immediately after, “But on the day that video was taken (i.e. his free-solo), it didn’t feel scary at all. It felt as comfortable and natural as a walk in the park.” Walk in the park? Was Alex serious or joking?

Alex explains in the TED talk his years of systematic effort in preparing for such a climb. But the part that is of interest to us is related to what Alex calls the most difficult part of the climb – the Boulder problem (8:06).  “It was about 2000-feet off the ground and consisted of the hardest physical moves of the whole route. (It involved) long pulls between poor handholds and with very small, slippery feet.” This manoeuvre culminated in a karate kick with left foot over to the inside of an adjacent corner. This required “high degree of precision and flexibility”. Alex had been doing a nightly stretching routine for this move for over a year (8:35).

Ability to navigate the boulder problem including the karate kick comfortably is an example of the leap-of-faith assumption in Alex’s climb. If he didn’t want to be a lucky climber, then he had to master the solution of the boulder problem. In this video, “What if he falls? The terrifying reality behind filming free-solo”, we see Alex practicing on the Boulder problem (6:00). And we see him practicing with a rope and actually falling in the process (6:07). What that means is that Alex would have experimented with his ideas to navigate the Boulder problem with rope first. And he would have failed in many of these attempts and learned valuable lessons on what may work. This is an example of doing the last experiment first.

Can Boulder problem be re-created in an indoor environment? Yes. You can see how an indoor wall climbing center VauxWall recreated the Boulder problem in this video. And see how Alex’s climb feels like a graceful dance on the wall here (10:50). I don’t know if Alex actually practiced in an indoor setup like this. But the point is it is possible to re-create a difficult situation in a controlled environment so that one can practice more easily, more often and at lower cost.

What would “doing the last experiment first” mean in the context of data centre outages in HDFC Bank? We can get a clue from what Dr. Werner Vogels, Amazon CTO says they do at Amazon. They started what was later called “Game days” where they pulled the plug from a data centre and see how their site held on. And like the indoor gym recreating the Boulder problem perhaps such experiments can be performed in a more controlled environment as well. At least it is worth considering it because the consequences of failure could be grim.

image source: youtube.com

Improving idea velocity: A webinar May 15, 2020

(Watch the video of the webinar - filesize 54MB)

Improving idea velocity is arguably the most important imperative of the current innovation efforts. In a challenging time like Covid-19, the speed of innovation becomes even more pertinent as we respond to create novel and affordable solutions such as vaccines, ventilators, disinfectants, social distancing interventions, etc. In this webinar, we will step back from covid context and explore ways of improving idea velocity, in general. This webinar is meant for practitioners, educators, students, researchers, and whoever is concerned with the speed of innovation. Familiarity with 8-steps to innovation framework is helpful but not required. To register, please send mail to vinay@catalign.com. 

Could “Create a margin of safety” be the toughest of the 8 steps to innovation to master?

Café Coffee Day founder V. G. Siddhartha’s unfortunate demise coincided with my class on “Margin of safety” in “Strategic Management of Technology and Innovation” course at IIM Bangalore. “Create a margin of safety” is the 8^th step of the "8-steps to innovation" book I co-authored. Siddhartha allegedly committed suicide by jumping into the Netravati river near Mangalore. We would never know the exact reasons why Siddhartha took such an extreme step. Given the debt situation of Café Coffee Day group, could it be possible that Siddhartha lost track of margin of safety? And, if a seasoned businessman like Siddhartha can overlook margin of safety, could it be the toughest step to master?

When I discussed this question with my friend and co-author of “8 steps to innovation”, Prof. Rishikesha Krishnan, he suggested I read the book “Failing to succeed: The story of India’s first e-commerce company” by K. Vaitheeswaran. It turned out to be a textbook case demonstrating how difficult it might be to internalize the principle of “margin of safety”. Let’s look at a few anecdotes from the book which illustrate this point. But before we look at it, let’s note that we are looking at a venture story when it hit a downward spiral. The Indiaplaza story contains several ups and many things that the founders should be proud of. Moreover, innovators and especially entrepreneurs should be indebted to K. Vaitheeswaran for the candid narration of his experience. It is so rare in the Indian context.

June 2009:  "A jewellery vendor from Delhi came to our office with a few thugs and abused me with choice expletives in front of all staff members and threatened to beat me up physically if I did not pay up the dues within two days."

"An apparel vendor from Surat came to the office accompanied by a local policeman. The policeman threatened to arrest me if we didn’t settle the dues in one week."

August 2012: "I had stopped drawing my salary from August 2012, and worse, I had made the mistake of using my personal credit cards to spend for the company. Every day private collectors visited our home on behalf of credit card companies and loudly demanded money to embarrass and shame me in front of my neighbours and family. Then I decided to withdraw my Provident Fund (PF) because we desperately needed money."

December 2012: "The last week of December was terrible. On 31 December 2012, New Year’s Eve, a group of drunk people banged on our apartment door loudly and in front of my neighbours, family and some friends abused me for non-payment of dues. I was falling into bouts of depression and my health was taking a severe beating."

April 2013:  "When this deal (a potential acquisition) fell through, the creditors became furious. In a few days, our office was swarming with creditors in person. An electronics merchant, during the conversation in our office, pulled out a dagger and placed it on the table. The managing director of a big publishing and distribution house from Delhi met me in Bengaluru and said that he would ‘throw babies in front my car’ when I was driving."

August 2013: "I was standing inside the Ulsoor police station on Cambridge Road in Bangalore. I waited to be interrogated by the inspector on a complaint filed personally against me by a merchant."

8 December 2013: "I had quit and I was not coming back. I had nothing to show for my efforts over fourteen years except for several court cases against me, social media abuse, being avoided like the plague by people I knew and being branded a failure."

At one point the author says, “Whenever I read about people taking their own lives due to financial troubles, I confess, I can understand and sympathize with a moment of madness.”

Building a “margin of safety” involves asking two questions: “What kind of catastrophic risk is there? And, can I live with it?” From the anecdotes above it looks as if the worst-case scenario was not difficult to imagine in 2009 itself. And yet no major action was taken to protect oneself against such a situation. Hence, I am beginning to wonder if creating a margin of safety could be the toughest of the 8 steps to innovation to master.

Catalign Quarterly March 2018

Catalign Quarterly March 2018 from vpdabholkar

Does “fail fast” contradict with “first time right”?

“Fail fast” is one of the principles I champion in my workshops on innovation and design thinking. “First time right” has been popularized by the quality movement, especially by the Six Sigma methodology. Hence, it is not uncommon to get the question: Does “fail fast” contradict with “first time right”?

To explore this question, it would help to understand “fail fast” and “first time right” better. Let’s start with “fail fast”. Does “fail fast” imply failing in any kind of way? No. To understand this better, let’s see the difference between a failure due to checklist-oversight and a negative result during hypothesis testing. Let’s borrow an example from Jeff Bezos of Amazon. In an interview, he said that if Amazon goofs up the opening of 19^th fulfillment center where an operational history exists, then that would be poor execution. Let’s call this checklist oversight failure. It means a prior learning has been consolidated into a checklist and the failure occurred because the checklist was not followed rigorously. “First time right” uses all the available past data in constructing the process to be followed for delivery of a solution.

In contrast, let’s look at the following hypothetical assumption: Amazon will be able to deliver a book size packet reliably on the terrace of a ten storied building in Bangalore via drone delivery. Let’s assume Amazon has experience of this kind of delivery in countries like the US but not in India. And if the first attempt at doing this delivery fails, then it would be a failure of the second kind – hypothesis test failure. Note that this failure would result in some learning which can be incorporated in the second attempt and so on. Depending upon the difficulty encountered, the cost of each experiment and the importance of this use-case for Amazon, more attempts would be made to learn more about this use-case.

When I say “fail fast” I mean fast testing of the assumptions associated with an idea. Now, we can see that “fail fast” is quite complementary with “first time right”. If Amazon were to launch the drone delivery on the terrace and get it right the first time, then it would help to do as many tests in different contexts – weather conditions, building locations, different building structures etc. Thus it would help to fail fast to get it right the first time you go live.

“Fail fast” assumes that there are certain unknowns / risks associated with achieving the goal. If all the steps in achieving the goal are well understood, then “fail fast” would not be required.

In short, “fail fast” helps you deliver “first time right”. The riskier your project, i.e. the higher the cost of getting it wrong the first time, the more important it becomes to “fail fast” in order to get it “first time right”. 

3 reasons why managers don’t throw their toughest challenge to their teams

In my innovation and design thinking workshops, we end up running a short challenge campaign. Participants get to experience what it means to identify a challenge area important to them, frame a challenge around it, generate ideas, build low-cost prototypes and validate them with a few people. Then I ask them, “Why don’t you throw your top challenges to your teams?” Why isn’t it common to see top challenges displayed in organizations? Here are top three reasons I have gathered from them:

   1.      Lack of clarity: Managers are busy attending to a number of issues – some short term, some long term. In the process, they typically don’t get time to step back and reflect. As a result, they don’t have clarity on what could be their biggest challenge. I ask them a few questions like, “What is one pain point you would rather leave behind in the office rather than carrying it home?” Or “Which is one trend – technology or otherwise – that may make your business irrelevant in future?” This gives them a clue. Of course, many of them don’t need any clue. Time and space for reflection is enough to bring out their toughest challenge. Nevertheless, if you don’t have clarity on what’s your toughest challenge is then there is no confidence to take further action.

2.      Fear of perceived incompetence: Managers feel, “I am being paid to solve problems. How can I communicate that I can’t solve them?” There is a feeling that if I throw my challenge to my team, my boss and perhaps even my team may feel that I am not competent to do my job. Of course, what isn’t realized by managers is that taking a position on a challenge is an important element of their job. The focus it brings makes a huge difference in aligning the creative energies of the people around you.

3.      Solver’s bias: What is more important – defining the right problem? Or finding the right solution? Most of us carry a bias for the right solution. We like to say – It was my idea. Of course, idea could have meant the challenge. But mostly idea refers to the solution. Mathematics is an area where problems are known for the people like Fermat or Riemann who defined them first. In most other areas, solver is perhaps more famous than seeker. As a result, some of the key issues remain pending or stuck. When a manager throws a challenge to his team, it is quite possible that the team ends up making initial prototypes for free. Because they feel it is their idea. When a manager asks a team member to build a prototype of his or her idea, the outcome may not be that enthusiastic. Who wants to work on boss’ idea?

To summarize, Managers don’t open up their top challenges because of (1) lack of clarity (2) fear of perceived incompetence and (3) solver’s bias. And I feel they have a lot to gain if they can establish clarity on their topmost challenge and seek solutions from their team or even outside in solving it.

Three characteristics of a good challenge book

Step-2 of our “8-steps to innovation” book is “Create a challenge book”. When I visit any organization, one of the first things I look for is a challenge book. Unfortunately, very few places that I have visited were able to articulate top few challenges clearly. Some of them discover in a leadership meeting that they don’t know what the top challenges are or at least there is no consensus on what the top challenges are. Without this clarity, it is difficult to focus innovation efforts. What are the characteristics of a good challenge book? Here are three:

    1.      Current-ness: What’s the point in making a list of challenges in an off-site and not updating it until the next one? In fact, a challenge book should be current like an airline arrival/departure list. OK, perhaps not that current. But it should be current within at least a few weeks. It is possible that the few top challenges may not change within weeks. It also helps to retire challenges which are not relevant any more.

     2.      Prioritization: Earlier this week, I got an opportunity to meet the manager of a new product development team of one of the largest e-commerce companies in India. The product manager was candid enough to articulate some of the tough challenges he and his team is focusing on. However, his team is relatively small – ten people. I asked him how he prioritizes his challenges. He said that he is still learning. The team used to change the priorities every day a few years back. Then they learnt to hold the priorities for at least a month. Now, the team is learning to set priorities for a quarter. Without prioritization or with rapidly changing prioritizing the team can be lost. Besides the toughest challenges don't vanish in a day or two, perhaps even in a quarter or two.

3.      Championing: This characteristic marks the difference between action and inaction on the challenge. A challenge either finds a champion or gets buried. A challenge is unaddressed doesn’t mean it dies. If a restaurant doesn’t respond to “home delivery through mobile app” trend, it may suffer its consequences eventually. But every tough challenge would need a person in a leadership position to champion it. This means she would put her weight behind the challenge, put a few resources together to study the issues and experimentation on the topic. A challenge book should indicate who the challenge is championed by in case it indeed has a champion. Number of challenges championed by people with senior positions is one of the important health indicators of its innovation initiative.

In short, current-ness, prioritization and championing are the three characteristics of a good challenge book. Hope this helps you to improve your challenge book.

Sustaining participation in innovation initiatives

I got an opportunity to write an article on “Sustaining participation in innovation initiatives” which appeared in NHRD Journal, Oct 2015 issue. This was a special issue on “HR in innovative organizations” edited by my friend Rishikesha Krishnan. In this article, I would like to summarize the paper in brief. You can read the full paper here.

Key hindrances: One of the key challenges that organizations face in running innovation programs is sustaining participation. People participate enthusiastically in the beginning. However, the energy is slowly dissipated and is replaced by apathy or cynicism. What are the key hindrances in sustaining participation? The paper presents 3: (1) Big bets only approach – i.e. organizations insisting that innovation is only about big bets. This limits the scope of innovation to a few people and most others feel “It’s not for me”.  (2) Lack of help for idea authors: Idea authors especially novices need help while taking their idea from a crude form to an attractive business proposal. The help could be being a sounding board, suggestions regarding prototyping, finding a collaborator, in preparing a business plan etc. If no such helps is available then idea authors may feel frustrated. (3) Absence of dashboard: A simple dashboard can communicate a lot about the progress of innovation activity. On the other hand, a lack of dashboard leaves people clueless. This includes those people who are running the innovation initiative in the first place.

What to do? Core elements: The paper suggests that any innovation initiative should have two core elements: (1) A program management function and (2) A focus on spotting and scaling “bright spots”. Program manager (full time or part-time) would maintain a roadmap and run various interventions which may include running a challenge campaign, training workshops, hackathon event, blogging contest, publishing a newsletter, calendarize reviews etc. “Bright spots” are evidences where things may be working in pockets. Program managers should be constantly on the lookout for such bright spots and see if they can be scaled.

More elements for “Continuous improvement”: If the primary focus is building creative confidence, then it helps to define what an acceptable idea is. Keeping the bar very high will be demotivating and keeping the bar too low will not generate an interest. Moreover, the review of small ideas needs to happen as low in the hierarchy as possible. Otherwise it can become a bottleneck leading to long turnaround cycles.

More elements for “Incubation process”: If the primary focus is on the incubation process, then it helps to run campaigns focused on specific business challenges.  Also management needs to give attention in the form of regular reviews and maintaining a rigor for the reviews.

Hope you find it useful and I would love to hear your comments / suggestions.

Rewarding innovation: process vs outcome

Tata Nano is one of the several stories that figure in our book “8 steps to innovation”. Nano received various innovation awards including the prestigious Edison award in 2010. Unfortunately, it hasn’t seen the market success yet. Does it mean that the decision to give innovation awards to Nano was poor or incorrect? Let’s explore this question in this article using Daniel Kahneman's lecture titled "The science of decision" delivered to the Pentagon (see the video above). 

Let’s begin with what Kahneman calls the key feature of decision making under uncertainty. “The key feature”, Kahneman says, “is that there is no perfect correlation between the quality of decisions and quality of outcomes. You could make a good decision and fail and you could make a bad decision and succeed.” (12:18) But then why are we so outcome obsessed?

Well, because we can’t help seeing it that way (15:15). We intuitively feel that if something ended well, it was done well. And if something ended badly, somebody must have goofed. The fundamental bias in operation here is called “hindsight bias”. Once the outcome appears e.g. that Tata Nano hasn’t had a success in the market, our model of the world changes. It starts looking obvious that the “cheap car” publicity was doomed to fail. We tend to evaluate the decision such as the Tata Motors’ decision to invest in Nano based on our current model of the world. And we find it extremely hard to evaluate the decision with the model of the world that existed before the decision.

Hindsight bias leads to another bias called “outcome bias” (19:30). This means we judge a decision on whether the outcome was a success or failure. This has significant implications. When we give innovation awards, we may be promoting daredevil gamblers rather than smart decision makers. More importantly, we may be losing good people because we may be punishing them for the failed outcomes in spite of their good decisions. So what should we do?

Kahneman suggests that we should focus on the process and not on the outcome when we judge a decision (13:28). Was the right process followed at the time of the decision? For example, was the investment decision based on a robust set of questions like Real-Win-Worth-it. An idea which comes out as the most promising idea through the scrutiny of such a process may eventually fail for factors not known at the time of the decision. That shouldn’t change the quality of the decision.

Kahneman also suggests that a good process of making decisions should ideally involve de-biasing steps, sort of corrective steps. For example, we can look at cost or time overruns for a similar project in estimating cost or duration (30:20). Or we could perform a project pre-mortem and bring out various reasons why this idea may not work etc (31:45).

Some companies recognize this and give rewards for smart failures. For example, P&G has in the past given “President’s fail-forward award” and Tata Group gives “Dare to try” award for smart failures from which significant learning has come out.

So, was the decision to award Tata Nano poor? Not necessarily. It depends whether the award was given after evaluating the process of making key decisions in developing Tata Nano. Not on whether Tata Nano succeeds in the market or not.

4 staminas of an innovator

Each of us has unique staminas – some of them come more naturally and some are built with rigorous practice. Examples are: running, weight lifting, reading etc. Some people can spend an entire day on WhatsApp without getting bored. I carry a view that innovativeness can also be built as stamina.  In an earlier article I wrote about “How to build curiosity stamina?” In this article I want to present the four staminas that I consider crucial for innovators: curiosity, experimentation, communication and collaboration.

Curiosity stamina: Being curious is easy, staying curious is not. Ask yourself this question “How long do you stay curious around one challenge area?” By now I have asked this question to several participants in my workshops. Most of them say, “Not more than a couple of days”.  That’s almost like being a couch potato or may be walking to your car every day while what you need is a running stamina of 5K or 10K. In a hyper active world where every day begins with its own set of problems, staying curious about one challenge area is difficult. That’s where one needs some discipline, perhaps of keeping a curiosity diary so that we don’t lose track of some of the interesting questions we ask ourselves. When do I decide to act on a challenge? One criterion I use is to check if I am still curious about it after a few months. Sometimes, I experiment around a challenge on the very same day!

Experimentation stamina: “But, sir, will my idea work?” Every now and then, I meet a guy who doesn’t like to share his idea openly in the class because someone may steal it. And he meets me after the class, tells his idea and wants to know if that idea will work. I usually tell him, “I don’t know. Why don’t you prototype and test it?” “That I will do, but I want to know if you think it will work.” That’s an example of low experimentation stamina. Similarly, I also meet technology enthusiasts who are busy perfecting their technology before they are ready to show it to customers. Many people just don’t get it that it is the speed of experimentation and the number of iterations that they can do that matters most. One of the participants in my workshop wanted to check if people in Bangalore would be interested in ordering filter coffee online in their office. He put out a web-page and got 30+ responses within a few days. He delivered the coffee himself to all the initial respondents. That’s an example of a low-cost experiment. How many experiments do you carry out every month? The answer could be a good indicator of your experimentation stamina.

Communication stamina: “I am not good at selling” our 12^th grader son tells us. That’s not very different from how I used to think about myself perhaps a decade ago. Having done a PhD, I have been trained to represent things abstractly. It took really long time for me to realize that abstractions are not useful when it comes to communicating your idea. One way you can measure your communication stamina is by answering the following question – How many times do you present your idea before giving up on it?  Personally, reading the book “Made to stick” by Chip & Dan Heath was a turning point as far communication stamina was concerned. It provides a simple checklist to ask you for improving the communication. Is your message concrete? Is it credible? Does it contain a curiosity flow? Are you telling an appropriate story? These questions can lead to improving the design of your presentation. It is no surprise that I find movies like “A beautiful mind”, “Twelve angry men”, “The matrix” useful in explaining my ideas.

Collaboration stamina:  I feel that this is the toughest stamina to build. Why? Because it not only involves you remaining curious about a topic but also needs at least one more person to be with you in the explorative journey. Moreover, it adds new dimensions like – who gets the credit? How do you resolve things when you don’t agree? I have been fortunate enough to be part of a collaborative effort with my friend Prof Rishikesha Krishnan which lasted four-five years and resulted in our book “8 steps to innovation”. How long has been a particular collaboration? This is a good indicator of collaboration stamina. Where do you start to build collaboration stamina? I don’t know. But perhaps a good place to start could be listening and appreciating others’ work which could be related and yet different from your work. It would helpful to have a collaborator who agrees with you on a few core assumptions (beliefs) at least as a starting point. The ultimate test of collaboration stamina is the ability to collaborate with someone who holds views exactly opposite that of yours. This is known as adversarial collaboration. Nobel Laureate Daniel Kahneman talks about his experience with adversarial collaboration here.

Well, this is my list of 4 staminas useful for an innovator. Perhaps yours may be different. Happy to hear from you. Who knows? It may lead to collaboration!