When I worked on the technical side of software development, I talked a great deal about #NoEstimates. I’m now on the business side of software development (in charge of all products and data for a global SaaS provider). I thought it would be a good time to revisit estimates and put them in context using the Phase Precision Premise.
The conclusion to this article is that I still find #NoEstimates works, because my organisation is firmly in the Vast Phase (see below). If that upsets you, there are other places you can go to scratch your itch.
So, let’s get on with it. What is the Phase Precision Premise?
Phase Precision Premise
The Phase Precision Premise is a guide to when different approaches to estimation are appropriate for a given software product. The phase is determined by the distance between the actual cost and benefit for each idea implemented within the product. The further apart they are, the less precision you need to make a business decision (such as whether you implement an idea, or in what order).
There are three phases.
- Vast – the benefit is vastly beyond any likely cost
- Narrow – the benefit is within an order of magnitude of the cost
- Crash – it is likely the cost will outweigh any potential benefits
The Vast phase is most likely to occur early on, when there is a “killer idea”. This is typically close to the intial launch of a product idea, or following a pivot that gains advantage through a particular product strategy (i.e. you see an opportunity to differentiate, target a niche, or if you find a way to save large number of people their time, money, or effort). The phase is tracked using acutal numbers, not estimates. As long your payoffs are, on average, massive; you are in the Vast phase. When things begin to converge, you can anticipate the transtion to the next phase. You might be using an unsophisticated method to determine the value of an idea in order to implement them in the right order.
The Narrow phase comes next. Once you have an established product you’ll find that your ideas begin to have less impact, or that they impact fewer users. The gap between your cost and the payoff narrows to the point where you are within an order of magnitude (i.e. you spend $100 and the return is less than $1,000). At this point, you need to be on the watch for the next phase as failing to notice the next transition can be fatal. In the Narrow phase, you are likely to introduce some up-front number crunching to ensure you don’t pursue ideas with low potential. You are more likely to estimate, for example using software estimation techniques from Steve McConnell or Mike Cohn. You should still track the actuals as they will let you know when the sponge has been squeezed dry, which leads us to the final phase.
The Crash phase is where your ideas are low value or have a low probability of success. It is no longer worth attempting any of them. That doesn’t mean you won’t come up with an inspirational idea that will reignite your product and launch you back into one of the other phases – but until this happens you need to limit your spending and concentrate on other products. You might introduce sustaining innovations or simply shut the product, depending on whether there is an existing user base that can keep the product afloat.
A given product may transition many times, in any direction, and at surprisingly different speeds.
The main product I manage is in the Vast phase. This is commonly the case because it makes sense for a good portion of a product manager’s attention to be on products in this phase (although some organisations do very well without any products in this phase… more on this later). There are hundreds of competing ideas for this product, but I don’t need much precision to make a decision.
I don’t request any estimates for the implementation of ideas in this product, because I know the cost is minuscule compared to the massive benefit. Even if the actual implementation time is reasonably variable (one day, one month) I know that the pessimistic end of that range is still tiny compared to the value I’m getting out of most ideas.
To help stack rank the ideas, I have created a simple algorithm that can generate relative anticipated value. We are experimenting with this, especially in terms of how sophisticated we want to make it. This exercise provides some insight that can be used to decide the order in which we implement the ideas.
When you are firmly within the Vast phase the anticipated payback is orders of magnitude greater than the anticipated cost. You don’t need to perform a sophisticated cost/benefit analysis and you don’t need to estimate the implementation cost at all. You do need to track the actual cost and you do need to ensure that can capture the real payback; because these help you spot the phase transition.
This is very important, so I’ll include this short version of the Phase Precision Premise.
When the benefit is massive and the cost is minuscule, you don’t need more precision than the words massive and minuscule
The key to the Vast phase is tracking real numbers so you can detect when you are leaving the phase. When you exit, you need to change your planning strategy.
Phase Per Product
This concept of phases applies per-product. Your organisation is never in a single phase, but each product is. It is pretty common to have a number of products spanning different phases; especially if you are disrupting yourself before someone else does. As you generate a product or replacement product that enters the Vast phase, you redirect effort away from the Narrow phase and Cruch phase products. The portfolio I manage contains products that are in, or transitioning between, all of these phases.
What is common at the organisation level is phase preference. There are organisations that prefer to be in the Vast phase and others that are great at managing Narrow phase products. Startups are typically seeking the Vast phase whereas there is a big market in acquiring companies whose valuation has been impacted by poor management of the Narrow phase and squeezing out the value that has lain dormant within.
The most common mistake when it comes to Phase Precision Premise is lack of tracking. Not knowing which phase you are in results in missed opportunities for investment, or over-investing in a product that won’t pay you back. If you don’t see the margin narrowing on a product, you are likely to continue investing at the same (wrong) level. If your product degrades to a lower-margin phase and you don’t realise it, you spend too much on ideas that aren’t worth the money. If you come up with an innovation that improves the product substantially, you may miss good investment opportunities.
Real Number Tracking
You have to track the real numbers to determine which phase you are in. You can’t use estimates to do this; it has to be the real numbers. You need a thick green line of real value and thick red line of actual cost. The distance between them determines the phase. The phase detemines the appropriate amount of planning. If you can’t get these numbers, your first job is to make it possible to get this numbers.
The clash over #NoEstimates probably comes from differences in how various organisations exploit the Phase Precision Premise.
As I mentioned earlier, some organisations will discard a product as soon as it exits the Vast phase and direct all of their attention elsewhere. When they manage to keep a product in the Vast phase over the long term, they are likely to be the market leader. When an organisation prefers the Vast phase, the only thing to prevent the switch away from a Narrow or Crash product will be the lack of any alternative offerings in the Vast phase. Where the organisation is fixated on Vast phase products and fails to keep the product in the zone or find their next one, they are ideal targets for competitors to disrupt, or for our next kind of organisation to acquire.
The organisations that are experts at Narrow phase products understand how to sustain or retire products profitably. They will introduce sustaining innovations where they can lengthen the life of the product, or reduce spending where they know the end is near. They may be less likely to reignite a product, but they are solid custodians while there is still a viable user base to service.
There is room in this world for all these organisations. The universe, though, will intervene when an organisation doesn’t know which phase its products are in as they will, eventually, make a catastrophic investment decision.
For the organisations that discard products as they exit the Vast phase, it is possible to run a successful development effort without ever estimating. For the organisations that are tackling Narrow phase products, the inclination to estimate will be stronger. When the margin is narrower, it becomes more important to win each time and to limit the loss when it occasionally materializes.