The Uncertainty Axiom of Product Management
6 September 2020 ‧ 5 min read
Developing digital products is inherently risky. Big software projects face delays, explode in cost, or get canceled all the time. Not just the effort involved in building the product is uncertain, though. More fundamentally, even the value itself is questionable. Consider the example of Quibi, the short form, mobile focused video subscription service launched to much fanfare in April, only to see disappointing download and engagement figures. Clearly, consumers did not find the offering particularly compelling and valuable, despite a lot of up front planning and big investments into the product.
This risk is also not limited to new products. Changes to existing products are affected as well. It frequently happens that product teams launch what they consider improvements to the product, only to see lack of user adoption, reduced engagement, or even backlash. What is happening here?
The fact of the matter is that digital product development is inseparable from this uncertainty. There are certain risks inherent to product development that can't be removed, they have to be managed. At the heart of these risks lies the fundamental fact that it's impossible to know in advance which of our ideas are good and which ones are bad—and the bad ideas outnumber the good ones. Marty Cagan, partner at Silicon Valley Product Group and author of the product management book “Inspired” calls this the inconvenient truth of product:
[A]t least half of our ideas are just not going to work [and] even with the ideas that do prove to be valuable, usable and feasible, it typically takes several iterations to get the implementation of this idea to the point where it actually delivers the expected business value.
There are many reasons why a product idea or product improvement idea may fail to deliver the hypothesized value to the customer or to the business. The most fundamental possible reason is that the problem that the product set out to solve simply doesn't exist, or that the market isn't big enough (which means that there might be some people with the problem the product is solving, but not enough). A variant of this is that there is a market, but you can't acquire customers profitably. If it costs you more to acquire customers than they bring in in lifetime value, the business isn't sustainable. Of course, it's okay to lose money on customers for some time while you figure out how to increase lifetime value, but eventually, that balance has to tip in the other direction.
Even if the problem was correctly identified and the market is big enough, you might have picked the wrong solution. This can happen especially if you just listen to customers telling you “I need feature X”, instead of drilling deeper to the underlying problem. Even if you understand the problem, though, you might have made some incorrect assumptions in developing the solution.
A related issue is that you might have had the right solution idea in general, but the implementation wasn't spot on. It might be that usability or quality issues prevent customers from realizing the value. It might be that users didn't discover or understand your solution. In any case, these issues can make a good solution idea fail on the first try, and it's usually hard to determine from looking at data alone whether the idea was bad or the execution was botched.
Lastly, you might have the right problem and a working solution without issues—but switching costs or inertia might be too high. If people are only mildly inconvenienced by the problem in the first place, or if they are locked into the current solution by contracts or simply by having previously sunk investments into it, they likely won't start using your better solution because the benefits don't outweigh the cost and inconvenience caused by switching.
So there are many ways that product and product improvement ideas can fail, and experience across organizations shows that most ideas do end up failing. Let me be clear about one thing, then: it's impossible to tell in advance which ideas will work. The ideas that end up failing are often ideas that sounded good in theory, but broke down in practice.
It's also not a factor of where the idea originated. Ideas can work or fail regardless of whether they originally came from the product team, from leadership, from stakeholders in the organization, from customer feature requests, or from original user research. It's not even purely a matter of skill or experience. Even the best product teams in the world have more ideas that don't work than those that do. While exact numbers are hard to come by, anecdotes state that even in top companies like Google, around two third of product experiments fail to deliver the hypothesized value.
This is why this is an axiom, a fundamental truth: regardless of your initial confidence in product ideas, most of them won't deliver the hypothesized value (to the customer, to the business) you hoped for. Moreover, you won't be able to identify the bad ideas until you have invested substantial effort in exploring them. The difference between a great team and a mediocre team is understanding this fact and trying to weed out the bad ideas instead of simply shipping them.
If you don't accept this axiom, it can be stressful and wear you down. No one innately likes being proven wrong again and again. If you believe that there should be a way to only have good ideas, or at least to identify the good ones early on, you will get frustrated when that doesn't happen. The truth is that you will pursue a number of ideas down the path only to later realize they won't work. Of course, the goal should be to identify the failing ideas as early as possible, but doing so without some real work in research, discovery, design, and perhaps even engineering is often an illusion. A prioritization framework where you put all the ideas in a spreadsheet and only the valuable ones fall out doesn't exist.
This axiom is one of the main reasons why you should move away from feature-based roadmaps and stop attempting to prioritize features top down by their “value”. Instead, prioritize strategic themes and problems to solve that will help you achieve your vision, and then assign them to empowered product teams with the mandate to try out different solution ideas until the problem is solved. Many of these ideas will fail, that's the axiom, but a team owning the problem will be able to identify the failing ideas and iterate until they've found one that works.
Once you accept that most ideas won't pan out, you will also embrace the need to validate all solutions and to prioritize actual outcomes over output in the form of shipping features. In a world where most ideas don't work, you can't be precious about your ideas and must embrace flexibility.
Lastly, the best way to find the ideas that work among the many that don't is to try as many ideas as possible. The way to do that is to make sure that ideas that don't work are invalidated as quickly as possible—“fail early, fail often.” This requires a culture and processes of iterative product discovery, in which many ideas are generated, sketched out, prototyped, validated, and often immediately discarded. The only way to find great ideas that deliver value is to test and discard many not so great ones.
The best product leaders, product managers, and product teams have embraced this axiom and are organizing their work around it. This is the only way for them to both develop great products and stay motivated in the face of many ideas failing for only few succeeding ones.
I hope you found this article useful. If you did, feel free to follow me on Twitter where I share thoughts and articles on product management and leadership.