Nader K. Rad

That's not a mistake!


You do something, and the result is bad, really bad. Did you make a mistake? Not necessarily.

Decision vs. outcome

Imagine there's a random number generator with outputs between 1 and 5, and you have to bet on whether the next number will be odd or even. What would you do?

Obviously, you would bet on odd numbers because there are 3 possible odd numbers and only 2 possible even numbers.

After you place your bet, the next number is generated, and it's an even number!

Did you make a mistake by betting on odd numbers instead of even numbers? Of course not. In other words, if you have to bet again, it's better to bet on odd numbers again.

Outcome bias

Our naive tendency to judge individual decisions based on their outcome is called outcome bias, and we'd do well to be aware of it and try to control it.

To practice, you can play a board game that mixes skill and chance, such as backgammon, while being aware of the outcome bias. It's also fun :)

Lessons learned

Why did I think I should talk about it?

It's because I've noticed that some people think that lessons learned in projects are all about mistakes. This is not correct because of two reasons:

When done properly, less than 10% of the lessons would be about mistakes.

The reason

The reason we shouldn't use the outcome of a single event to judge the decision is that there are many uncertainties and unknowns, and we have to make our decision based on the limited information we have.

Now, there's a complication: Some uncertainty is inevitable (think of the uncertainty principle in physics), but that's not all, and it's mostly about our lack of information, which we can change by spending more time and money.

For each decision, we can continue to spend more and more effort gathering additional information and then make the decision with a near-minimum level of uncertainty. But this has to be justified: If you have to choose between two brands of paint for the fences, you're not going to stop working for a few weeks, spend hundreds of thousands of dollars, and run all kinds of tests in a lab to see which paint is better, because the expected value of the negative outcome doesn't justify it. It's better to just pick a paint with the information you have and move on. If it turns out to be a bad paint, that's fine, we can repaint it with the other paint by spending a little time and money.

The framework

So, when we get an unexpected/important/sensitive outcome (either negative or positive), we need to ask ourselves two questions, one to address the correct concern and one to address its meta-concern:

If you do this correctly, you may find actual mistakes that led to positive outcomes, cases where you realize you spent too much effort on a decision that wasn't worth it, etc.

Why does it matter?

We shouldn't do anything without a clear purpose (NUP5). The purpose of lessons learned in projects is not to find mistakes but to make better decisions in the future. What's discussed in this article is one of the things that can help you make better decisions and, therefore, would be reflected in your implicit or explicit lessons learned system.

— the end —