Sorry I'm Late, Again
Why we remain optimistic about finishing work quickly
Posted Sep 28, 2012
I remain hopelessly optimistic about how much work I can accomplish. The time frame doesn’t matter — I have optimistic predictions for how much I will finish today, this week, this academic term, and really in my life. I’ve been known to bring home ridiculous amounts of work for weekends — many more student papers than I can possibly read. I make what I think are reasonable estimates for when I will finish a project. Much, much later, I discover that my estimate was wildly optimistic and that it took me three times as long to actually finish the work.
I remember some advice I received once on how to make appropriate predictions of the amount of time a project would require. First, make your best estimate, then double that, and finally increase your unit of measurement. If your estimate is one hour, then make it two hours, and finally two days. Why are we so atrocious at making estimates?
This failure is so well known that it has a name — the planning fallacy. These sorts of planning failures can also be incredibly troublesome and expensive. The really cool thing is that we all know that we have been wrong in the past. I know how little work I accomplished last weekend, yet I nonetheless will bring home too much this weekend. I know how long all of my papers have taken me to write, but I make what turn out to be silly projections of my future writing time. I can remember the past, but I am doomed to repeat it anyway.
My favorite research on the planning fallacy is a set of studies by Beuhler, Griffins, and Ross (1994). They asked college students about their plans for finishing a large and important task — their honors thesis. The students gave their best estimate of when they would finish and then gave a pessimistic estimate (one assuming that everything that could go wrong would go wrong). The students’ best estimates were that they would finish in 34 days on average and their pessimistic estimates averaged 48 days. In reality, the students finished in 56 days. Even the pessimistic estimates were optimistic! I guess everything that could go wrong went wrong and then a few more things went wrong as well.
Beuhler, Griffins and Ross replicated this on a shorter time period, asking students to estimate when they would finish both academic (papers, assignments) and personal (clean apartment, fix bike) tasks. The students were asked to choose things that they thought they would finish within one week. When the researchers called back the next week, they couldn’t finish the study because the students hadn’t actually finished about half the tasks. On average they thought they would finish the tasks within five days and ended up taking 10 days. Wildly optimistic estimates.
In other research, Beuhler, Griffins, and Ross (1995) asked people to estimate when they would finish and submit their taxes. For some people, this would be a good thing to finish because they anticipated getting refunds! People who anticipated refunds estimated that they would finish before people who didn’t estimate refunds. (No kidding, why do it if you have to pay?) But of course those expecting refunds did not finish any sooner. They were hopelessly optimistic even when money was sitting on the table.
Why are people so optimistic about finishing tasks? The problem hinges on what information people use to make the judgment. People typically focus on the future. They think about their plans: how much work there is, the various parts of the work, when they have time to work, and how important the work is. People rarely think about future problems. They think about past failures to meet deadlines even less frequently. So the first problem is that they may know they haven’t finished on time before, but not use that information in making a new estimate.
But even if people do think about the past, they may dismiss their previous failures to meet goals. Yes, something went wrong the last time, but that won’t happen again. A college student may know that he didn’t work last weekend because he went to John’s party, but he also knows that John won’t have a party this weekend (of course, Jim might have a party, but that doesn’t come to mind). I know that my previous manuscript writing was disrupted by other demands. But I also know that those demands were one-time things and won’t happen again. Of course, other one-time demands are bound to occur, but I don’t realize that connection. We see past failures as unusual and unrelated to the current prediction. To some extent we are right. The same problem is unlikely to occur again. Our failure is not realizing that some problem is bound to happen.
So I want to use this as an opportunity to apologize to all my co-authors for all the times I’ve failed to meet my own estimates of when I would finish. I am completely sure that this time, my prediction is much better.