Skip to main content

Software Estimates Part 1: The problem with software effort estimation

If you’ve found yourself in any new development team, you probably agree that estimating the effort required to complete a task can be a headache. Can we make them right or, at least, plan to minimize their impact? Let's find out together.

If you’ve found yourself in any new development team, you probably agree that estimating the effort required to complete a task can be a headache… And, sometimes, even when you spend time to make your estimates carefully , your product just can’t ship on time with all promised features! So, one may wonder… what is the problem with estimates in software development? Can we make them right or, at least, plan to minimize their impact? Read on to find out.

The problem with software estimates#

First, taking a look at what data tell us, we may not be surprised to find out that estimating software tasks is hard. According to a study conducted by McKinsey , 1 out of 3 software projects falls behind schedule! And you can even double this statistic (to reach a devastating 66%) if you count the projects delivered on time, yet cost more than initially estimated. Ok, you say, this should be true in all industries, right? Guess again; only 3.6% of non-software projects in the broader IT domain fall behind schedule!

So, either we - as software practitioners - are truly bad at estimating effort or we have to admit that the task is tough itself (if only to make ourselves feel better). Let’s try to delve into the problem one challenge at a time and discuss potential resolutions.

performance

Software requirements are not set in stone#

If you’ve already communicated your products to customers and stakeholders, this should not come as a surprise. In an ever-changing landscape, people demand an ever-changing product. What’s worse is that they too experience information overload and sometimes they are not sure what they want. New feature requests may come at any time and disrupt your current development plan.

One could argue that this is actually one of the reasons for adopting an agile methodology in the first place. And that’s correct, but with a word of caution: being agile does not mean developing anything anytime. It means having a clear yet flexible plan. For instance, in Scrum, a set of the next features to be developed must be added to a sprint and estimated beforehand. Messing with user stories during a sprint is dangerous even for the Scrum Master.

Requirements are often not well-defined#

This is related to the previous point and is a multifold challenge. First things first, people might have a broad idea of what they want and lack the patience to describe it in full. And even when they do, miscommunication may be a bottleneck (let’s not forget the famous tree swing cartoon). On a not-entirely unrelated note, software teams often do not invest enough in requirements elicitation. It is also unfortunately common in agile (why should we invest in fully defining a user story, when it’s going to change next sprint?).

If you’re going to take home one piece of advice from this blog post, let it be this: time spent in defining requirements right is time earned in the long term. Make sure that everybody is on the same page before starting your development sprint, or you may end up in major refactoring setbacks later on in the project.

Estimates are just that, estimates#

Ok, this seems intuitive; however, believe me when I say it’s one of the most common problems when it comes to software task estimation. When you decide as a team that a task is going to take three days, this is not a deadline; it’s an estimate! Thus, if it takes five days instead, the problem is not with the velocity of your team, but with your estimation skills.

So, when it comes down to finding good estimates, there is no simple trick (a note effectively conveyed by xkcd). There are just some rules to abide b; keep your work well-defined, break it into pieces (smaller tasks are estimated easier), and, last but not least, expect them to prove wrong!

estimates Source: https://xkcd.com/1658/

Wait, so, what can we do after all?#

OK, I admit that this post may seem somewhat disappointing. However, the truth is that software effort estimation is a skill acquired by experience. One cannot learn to make good estimates without first trying to. In this sense, it’s like programming. Anybody can claim to have learned how to program from a course, however actually mastering programming is a skill acquired only by writing production-ready code.

What you can do is to think about the main guidelines outlined in this post before your next estimate. When estimating the effort for a task, ask yourself and your team these questions: "is this task clear and well-defined?", "is this what the customer really wants?", and of course (when all else fails) "what is the cost of altering the task later on?". The answers you’ll get should keep you safe from at least some trouble the next time you discuss the next sprint with your team.

At this point, make sure to subscribe to our newsletter, so that you don’t miss my follow-up post (Software Estimates Part 2), where I’ll give some useful tips and refer to common pitfalls to avoid during effort estimation.

Till next time folks!