In this article I will explain my step-by-step approach for creation of a trustworthy forecast of how much a feature/set-of user stories would take to do. It is is something that worked out for me and hopefully you find some inspiration that would help you.
The problem with forecasting/estimates and the Iron triangle
In an ideal agile environment the resources and the time is fixed in the iron triangle. You can say how much of the scope you can do with fixed amount of resources and in a given time:
Another problem is that the backlog always grows. If it would not grow I would be very suspicious. What is happening with the product? Most probably work is hidden under some existing backlog items if there are no new ones. This fact is at the hearth of agile. As you go, you discover new solutions, new ideas, improvements and there will be some things that you have not taken in to account, even things that were not possible to expect. You will discover new problems during the development. If you would base your forecast based on a fixed scope you would do a waterfall approach.
"If I were to define a failed project, one of my criteria would certainly be a project on which no one came up with any better ideas than what was on the initial list of requirements.”- Mike Cohn, Agile Estimating and Planning
Build trust by transparent forecast methods
In any relation, also in business, trust is the keystone of good relations. Same applies when you try to answer then question of when a certain scope will be done. Be careful! Never give a specific date! That’s the most important point in this article. By this you would recant the most fundamental characteristic of products in complex environments. A business environment is like waters of the sea upon which you try to navigate the ship of your product. The weather is constantly changing, there are strong currents, other ships, shallow water, etc… In complex environments you cannot take in to account everything and you have to change your course as the situation dictates. This is also the core of Agile, to react dynamically to the environment. By giving an exact date you drop an anchor, that can pin you down. Rather then giving a specific date you can give a forecast of what could be possibly done in a given interval of time.
Build trust with forecasts that are clear, understandable, rational, data-driven and ultimately trustworthy. Don’t give a specific date with 100% confidence and then chase it. That is a recipe for disaster. You need to find a way how to explain this to the stakeholders and find indicators (KPIs) that shows if you are on course. With an understood forecasting method and progress indication the stakeholders will believe in the project and even make decisions based on it.
Don’t be in conviction that nothing can change. Suddenly you can find an unforeseen landmine (even with a very good risk management) and you have to be prepared to change the course of the ship and with it the method should tell you how the future delivery dates can change.
Define the scope
To be able to say when it will be done you need to ask what should be exactly done. For this you need to define the scope. Either you already have a defined set of features or you need to create it. If it is not already created I always start with user story mapping where we break down everything. Ideally you should break it down, so the user stories are mostly estimable, so they are not giant or not-estimable at all. If you have the time you can slice the requests to smaller pieces, but that is not always possible because of lack of time for slicing or because there are just unknowns.
I always use user story mapping because, it defines who are the personas that will engage with the product, it gives a chronological user journey, it defines priority of the user stories and last, but not least it defines the requests as user stories. With the user story you know for who you are developing, what you are developing and for what business reason. Once I made a mistake and accepted requirement defined in a free form and it complicated the estimation very much. Just don’t do my mistake, never accept a requirement in a free form. For me the best is a user story. This format might not be known to your stakeholders. You can teach them how to write them. In one case I did it with a whole group of stakeholders during a design sprint.
Try not to forget each and every user story to capture at least 90% of the scope. Especially make sure you capture difficult, unclear or risky requirements, otherwise the preciseness of your forecast can suffer.
Estimate each individual user story
I take the whole list of the scope issues and I play scrum/planning poker one by one with subject-matter experts, in my case software engineers and architects. More brains can think of more variations and can evaluate more problems, please do not ask just one person. This method promotes dialogue about the user story and makes the user story clearer. Immeasurable times I hear the sentence “wow, I never thought of this” during refinement meetings discussing the estimate for a user story.
I always set an alarm during the explanation and discussion of the user story. We don’t discuss details just topics that have impact on the estimate. This you have to explain the team very clearly. It is very easy to go down the rabbit whole and discuss deeply. I set the alarm for 5 minutes, after it expires, we have to show our estimates. After the first round we have few words if the estimates are very different and if the team still cannot reach consensus I take the higher estimate (in a reasonable way – e.g. 5 and 50 is really for discussion, but 5,8,8,13,13 will be 13). So I’m not taking the average, as normally, but the higher estimate to be on the safe side.
The timebox is needed because the limited time, but also because of the law of diminishing returns. After a certain equilibrium there is no point in continuing in refinement discussions as it does not improves the estimates.
I use the Fibonacci sequence unchanged for the estimates – 0, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, ?. The scrum poker uses an alternated sequence (after 13 goes 20, 40 etc…). It reflects that the team has just a very rough idea about the user story and with less information the complexity could be higher then anticipated.
Vasco Durantes NoEstimates is a great book to suggests to not do estimates and base your estimates on how much backlog items the team is able to do. I was testing this and came to conclusion that we were unable to press the velocity volatility to an acceptable measure. I want to experiment with this approach in the future and find a way how to do it reliably, because I believe this can be a very powerful tool.
Apply the cone of uncertainty
After I have a list of user stories that are all estimated I apply the “cone of uncertainty” known from PMI. This tries to suggest a quantification of the effect of the unknown on how much an estimated feature would cost or how much time it will take by adding or decreasing the estimate by a variance. There are many variances, but I use for initial estimate the -25% to +75% rule. In other words you generate an interval of time during which the features could be done by decreasing the total of estimated time by 25% (in case you would be faster) and add 75% to indicate when it should be done at latest. If the estimate is still too broad, you can do another round of estimates, that would more in details, then I go for -15% – +25%. From my experience my software engineer team usually estimated in the realm of +/- 25%.
Buffering
If you are using average team velocity in SP to calculate your capacity to do something, then you don’t need any buffering, so again one step less for the forecasting. But if you care using PDs then you have to take in to account also the fact that bugs will occur. If you track how much time you spend on bug fixing then you can find out an average amount of PDs spent per iteration on bug fixing and count with that. I usually add 30% for cushioning for possible bug fixing, so I decrease the capacity by it and use 70% of the calculated capacity. The best is to use empirical data if available. After above is done calculate an interval of project delivery.
Calculate the time frame based on team velocity or capacity
For a project I was doing the regular estimates (User story refinement) in person days (PSs), so naturally the forecasting was also in PDs. It’s understandable that if you are not basing you estimate on team velocity you need to calculate the capacity of the team in PDs.
No one is using 100% of his time working, that’s just unrealistic, so I was inspired by what Mike Cohn, who writes, that one study calculated that people use their time about 55% to 70% of their time on project activities. Most effective were engineers in Toyota that are able to spend 80% of their time on the project. I use the 70%, so I reduce the total capacity by 30%.
If you are using story points it’s easier. On another project I could easily use our average story point (SP) per sprint velocity and apply it on the what we had estimated. This includes the real effort spent on the project. You also don’t need to calculate capacity for team members that are not 100% on the project and it all can complicate the process of capacity calculation.
Now after estimating all the user stories, technical requirements, running it through the cone of uncertainty adding the buffer (if needed) you should have the final interval how much sprints or PDs the implementation of the user stories/features would take. You can now say that the feature will take e.g. 5-8 sprints or e.g 5-7 months.
Suggest next steps if needed
As a last part of my forecast I always include a slide about what could be the next steps.
Sometimes the forecast can seem too much and for that the stakeholders needs to understand that the change of the plan does not need to mean also change in release date.
You can go in to deeper analysis of requirements that seems to be unclear and thus clarify what needs to be done. By analyzing it you can find, that nothing changes or that it would cost less (ideally) or it would be even more complicated. Either way you learn something.
Also, you can look at the scope an decide to drop something that has low priority. Especially in the MVP case, please make sure that it is really the MVP or MMP. No bells or whistles. Be very strict about it.
Alternatively you can add team members. In one case we decide to add additional members to the software engineering and we also dropped some features.