Developers at War with Uncertainty

There’s a secret every seasoned project manager, product owner, and developer knows. It’s not the coding part that is hard. What’s hard is coming up with a realistic estimate and meeting the deadline.

There’s a secret every seasoned project manager, product owner, and developer knows. It’s not the coding part that is hard. What’s hard is coming up with a realistic estimate and meeting the deadline. It’s hard for small tasks, and it’s exponentially hard for big projects as a whole. Still, we do it all the time.

Do you have a story when implementing a simple feature like login took weeks instead of a few hours because one sentence in the spec was missing? You started being all optimistic, finally working on something predictable, this was done many times already.

After some time, you begin to realize that something isn’t right. It’s all too simple and suspicious, especially that 3rd party library you have to use to support login through ear gestures.

Finally, you realize that that library will require you to upgrade the other library. And a new version of it isn’t supported by half of the other open-source projects you are using. Sounds familiar?

If you’re new to the software development, it might sound like the unusual thing, but this is the bread and butter of what we’re doing.

We’re an established agency, more than twenty developers in the team. We’ve been working together for years, built all types of apps possible, helped dozens of clients, yet we have these problems every other day.

“The enemy is a very good teacher. “— the Dalai Lama

That’s a quote I found in the book “War of Art” by Steven Pressfield. He describes resistance as the main enemy of a writer. What keeps us, the developers, from knowing what it takes to implement X? The same thing that prevents Uber drivers from knowing how much time it takes to drive from A to B. Or what makes a stock market broker loose money of his clients in a few seconds.

Our biggest enemy is uncertainty, and we can learn a lot from it. We should because it’s so powerful that it can kill your project. It can cost you a client, or it can make your team members quit. It causes a lot of stress for everyone. It’s invisible. You can’t hunt it down and eliminate it because it’s everywhere and it’s a moving target.

You can’t beat uncertainty. But you can learn how to live with it and keep things at bay. We have been pulling it off successfully for years, just as lots of other companies in the industry, producing great products on time, and you can do it too.

About calculated estimates

You can’t provide an accurate estimate yet you have to do it anyway. Consultants have to do it. Just as developers. Managers and companies have to do it too because of business needs.

Jeff Attwood, the co-founder of StackOverflow, has my favorite example of estimations. If you were asked to estimate the temperature of the Sun, what would you say? Your estimate can be narrow, but probably inaccurate. Or it will be too wide and thus not very useful.

This type of estimation above is purely judgemental because it’s based only on gut feeling. A few years ago, I would tell you to avoid such guesstimates as much as possible and use historical data to calculate it instead.

But as much as I like to read books on project management and agile software development, I must admit that it doesn’t work.

Over five years, we were meticulously tracking every task, and every hour spent by developers. We also track an estimation for every task and user story, plus an estimation is given to the client by the manager. We have hundreds of thousands of data points.

So what we do when we try to predict how much time it will take for a new user story? We multiply the estimation from developers by a typical margin of error. That’s essentially the same as multiplying your gut feeling by two or by three.

Uncertainty and unpredictability

Have you ever heard of chaos theory? It’s a branch of mathematics focused on dynamic systems. I like its description from the book In the Wake of Chaos: Unpredictable Order in Dynamical Systems.”

“Small differences in initial conditions, such as those due to rounding errors in numerical computation, yield widely diverging outcomes for such dynamical systems, rendering long-term prediction of their behavior impossible in general.”

We are never doing implementing the same feature twice. Even small differences in the task at hand make predicting how much time work will take nearly impossible.

In reality, the only thing we can predict is that something unpredicted will happen. And if we acknowledge this and adjust our business goals to this fact, we will have a way to stay sane and actually complete our projects on time.

Estimate, prioritize, and cut off the scope.

That’s what I advise to our clients all the time. Business needs estimates to plan the budget and operations. To provide estimates, you need to write down the scope of the project. You don’t have to spend months on that, scoping can be done in a very lean way, and I’ll write about it a bit later. But an estimate is just an estimate — it’s a guess when we’re talking about software.

According to the project management triangle to create a quality product, you have to adjust three variables:

Scope, cost, and time.

Typically cost and time is more or less fixed. If it’s not — make it fixed, or you will end up with feature creep and burn a lot of money. The best way to do it is to define releases with a set deadline. We do it all the time, and we’re typically working on projects that take years. Yet, we deliver value every month or two, depending on the scope of the release.

When you have fixed releases, the only thing you can adjust is the scope, and that’s what you should do. I will write about the best way to prioritize and cut the scope next time!

P.S. Uncertainty is at its highest level at the beginning of every task or project. Feel free to check out this video about the cone of uncertainty to learn how it changes over time.


Originally published at The Startup: Build something awesome