The heuristics and biases literature claims that the substitution bias occurs when we replace a difficult judgment task with an easier one. According to Kahneman and Frederick (2002), we don’t even notice when our intuitive System 1 mode of thinking makes this shift, which underlies many biases and illusions. To reduce this bias we need to energize our reflective and analytical System 2 mode of thinking; where feasible we should rely on algorithms and artificial intelligence (AI) systems rather than on our heuristics and intuitions.
One example of the substitution bias is the bat-and-ball problem: A bat and ball together cost $1.10. The bat costs $1.00 more than the ball. How much does the bat cost?” Our immediate impulse is that the bat costs $1.00. But our intuition has betrayed us. (The bat costs $1.05.)
I am not enthusiastic about judgment biases, but recently I noticed some clear examples of substitution bias.
However, the examples aren’t about how ordinary people fall prey to substitution bias. Rather, the examples show how the analytical community falls into this substitution trap.
Example 1: The gamble metaphor for choices. Many academic decision researchers assume that decisions that are made in the face of uncertainty can be seen as gambles. Beach (2019) disagrees. Gamblers (e.g., roulette bettors) passively await the results of their choices but the rest of us work hard to make our choices succeed. "If a gambler worked to win his bet, he or she would be cheating. If a business person did not do it, he or she would be fired." (pp. 102-103).
Therefore, the analytical decision researchers have replaced a difficult topic—how we make choices—with an easier one and less interesting one for which they have analytical methods for conducting experiments: gambles in well-structured conditions. They are exhibiting the substitution bias.
Example 2: The computational metaphor for decision making. Decision analysts have suggested structured methods for choosing between options, such as multi-attribute utility analysis. These methods allow us to compute our preferences. However, there’s no evidence that using these methods actually helps people.
The methods treat choices as consisting of attributes that can be pre-defined and contrasted, but the methods don’t take context and expertise into account. The proponents take a very difficult issue—how people make decisions—and substitute a much easier one: how can people take a variety of features into account. An example would be cost-quality trade-offs, like pondering whether to pay for a sun-roof on a new car. This easier task can be readily calculated and studied and systematized.
Example 3: Detecting anomalies. In a previous post, I showed that the analytical community treats anomalies as outliers and offers up various statistical methods and AI approaches for highlighting these outliers. However, anomalies are not simply outliers. They are violations of our expectancies. We need expertise to generate expectancies within a context. The analytical community has substituted “statistical outliers” (which can be computed) for “cases of violated expectancies,” which is a much messier issue.
Example 4: Using Bayesian statistics to identify situations. Bayesian statistics let us update our beliefs based on the evidence while also considering how likely the base rate of different outcomes are. Bayesian statistics let us make judgments about what is going on as we receive new information. However, our assessment of a situation is more than updating beliefs. My view is that we build stories about how things came about and how they are likely to transform, and that we judge the plausibility of the transitions from one story-state to another using our mental models. (To be fair, Bayesian models can be used to represent stories, just as stories can be represented using language, semantic networks, and other structures.) From this perspective, using Bayesian statistics is a substitution for story-building and understanding other less formal ways people make plausibility judgments.
These examples show why I am claiming that the substitution bias is a real problem, not for individual decisionmakers but for the analytical community. I see this community as ignoring, devaluing, and distorting cognitive phenomena, and substituting formulations that are easier to calculate, the classical dodge of the substitution bias.
I am not claiming that the analytical researchers are making the substitutions intentionally. I think they are gripped by what D. Klein et al. (2018) have referred to as Rationalist Fever Dreams, so the use of algorithms and calculative methods seems reasonable to them.
The prime takeaways from this essay are (a) to be alert for the substitution bias on the part of modelers and analysts and AI developers; (b) to scrutinize their work to see if they are distorting the cognitive phenomena they claim to be addressing; and (c) to use that scrutiny to gain a deeper understanding of these phenomena.
References
Kahneman, D., & Frederick, S. Representativeness revisited: Attribute substitution in intuitive judgment. In t. Gilovich, D. Griffin, & D. Kahneman (2002). Heuristics and biases: The psychology of intuitive judgment. Cambridge University Press.
Klein, D., Woods, D., Klein, G., & Perry, S. (2018). EBM: Rationalist fever dreams. Journal of Cognitive Engineering and Decision Making, 12(3), 227-230.
Beach, L.R. (2019). The structure of conscious experience. Cambridge Scholars Publishing.
