Estimating the time for software projects is notoriously difficult. This is because the majority of the work in most software projects is “discovered” work. Work that was not obvious while planning that is discovered when implementing. If you’ve done any serious programming work, you’ll know what I’m talking about. “Turns out this 3rd party API has limitations we didn’t know about,” or, “Turns out, the update we planned will break another related feature, so we’ll need to do some re-factoring there as well.”
Something I’ve found quite useful in planning new projects is differentiating between uphill and downhill work. Downhill work is when you know, with a good degree of certainty, how much work is required to finish; when you can see the end (the bottom of the hill) and how to get there. Uphill work is when you don’t know what the tasks will be how or much work they will involve. There is a moment in every project where you cross the peak. The problem and solution become well understood and you can see the bottom – uphill work turns into downhill work.
I am a self-taught programmer so I have a lot of knowledge gaps. Therefore, uphill work tends to define the majority of the programming I do. But, regardless of expertise, uphill work is unavoidable. Could be that the solution requires a new library or tool, or a part of the code-base you’re unfamiliar with, or is just a generally hard problem. No programmer has tackled every problem, so even the experienced coder will face uphill work often.
Instead of trying to predict and plan all the work involved before starting, accept that uphill work is an unavoidable aspect of any software project. Most of the tasks will be discovered once the project starts. It may take a few days of exploration before any concrete tasks can be defined.
At the outset, try and identify what parts will be uphill work and downhill work. As the person who will be doing the programming, it’s important to be honest about your knowledge. Don’t try and save face by always claiming, “Yes, I can do that.” Confidence in the face of uncertainty is a virtue, but It helps everyone when you can be honest. “This could be an easy task, but I’ve never used this API before, so it’s mostly uphill work.”
Uphill work is difficult to estimate, because you don’t know how much work will be required or what rabbit holes might arise. Good planning should acknowledge this. It’s a mistake to hold a programmer accountable for their estimates when there is uphill work involved.
Something I like a lot better than estimating time required is defining appetites (a concept I’m borrowing from Shape Up). Instead of asking “How long will this take?” ask “How much time are we willing to spend on this?” The latter is a much easier question to answer and will lead to more effective prioritization.