The Ecology of Rationality
Irrationality often lies outside of the mind.
Posted September 23, 2021 | Reviewed by Vanessa Lancaster
- Rational choice depends more on the person’s context than on the person. If the input is poor, even a rational mind will make poor judgments.
- Babies, bats, and baboons can detect statistical regularities in their environment and master challenging tasks.
- Statements such as "We are all biased" lack meaning. Such declarations are little more than expressions of the fundamental attribution error.
by Joachim Krueger and David Grüning
Human beings, viewed as behaving systems, are quite simple. The apparent complexity of our behavior over time is largely a reflection of the complexity of the environment in which we find ourselves. – Herbert Simon (1969, p. 53)
Discussions of human rationality are often discussions of irrationality. If we understand the limits of mind so this view, we can learn how to make better judgments and decisions, and in time we will find ourselves in a happier and more humane company. Those who advocate this theory – e.g., to wealthy businesses – can get a good return by convincing their clients of their lack of rationality and then offering solutions.
Slowly but surely, the field of Judgment and Decision-Making (JDM) psychology has moved away from the flawed-mind theory. Two quiet revolutions have moved part of the burden of irrationality away from the mind and onto the structure of the environment in which the mind operates. The argument, in short, is that if the input is poor, even a rational mind will make poor judgments. This shift in emphasis, from the psychological to the ecological, has come with two critical distinctions. One distinction is about which kind of data is available for learning (and which is missing); the other is about the format in which these data become available. Let’s consider each distinction in turn and see if we can combine them to get the complete picture.
Kind vs. Wicked Learning Environments
The first distinction is between a kind and wicked learning environment. Hogarth et al. (2018) explain that in a kind environment, data are diagnostic and unbiased. Once learned, the lessons extracted from these data can be applied to out-of-sample predictions. When predictions are made, feedback is swift, accurate, and cheap so that learning can continue. Alas, data can be degraded into wickedness in several ways, and various types of selection bias can be the result.
Of these biases, the survivorship bias is most instructive. First noted – as best as we know – by Diagoras of Melos, the survivorship bias pops everywhere once we understand it. Diagoras, when proudly shown the gifts of gratitude seamen offered to the gods after having survived terrible storms, asked to see the gifts offered by those who had drowned. For his irreverence, Diagoras entered the history of Western thought as the first outed atheist.
Many predictors bet on the future in a wicked environment of our time – such as the neighborhood south of Wall Street; some will repeatedly hit the mark out of sheer luck. If others regard them as experts and see themselves as such, they will get rich, although those who bet on them will not.
The psychological challenge is a meta-perceptual one, that is, to know whether an environment is wicked. The damage is most significant when judges or decision-makers operate in a wicked environment without realizing it. This suggests two types of remedy; one is to improve the environment (which is hard to do). The other is to see through one’s metacognitive myopia and look elsewhere for profitable courses of action (which is even harder).
Judgment From Experience or Description
In a parallel development, researchers noticed the relevance of the format of the judgment or the decision task. Looking back into the field's history, they noticed that JDM psychology had inherited critical methodologies from the psychology of learning. Learning requires time, patience, and repetition. With these, organisms, including babies, bats, and baboons, can detect statistical regularities in their environment and master challenging tasks. They learn well from experience.
Beginning in the early 1970s, a new school of thought took hold. This school viewed JDM as a matter of direct perception, and the fun lay in finding out where perceptions turn into illusions. Kahneman (2011) reviewed this research tradition and laid the charge of irrationality at the mind’s doorstep. “What you see is all there is,” he famously declared, implying that there is always more to discover but that people are too dense or too lazy to look. On this view, irrational judgment is a matter of focalism. Single-minded attention to what is in clear view ignores the possibility that some focal stimuli may be irrelevant, while other, nonfocal, stimuli are essential.
Hertwig et al. (2018) note that the old school allowed respondents to learn from the data, whereas research in the new school presented respondents with narrative images to be perceived and judged. In short, whereas the old school emphasized the respondents' interactive experience with data, the new school emphasized the direct (and fallible) perception of described (i.e., summarized and stylized) data. For illustrations of this approach, Kahneman’s (2011) magisterial book is instructive. To remedy (ir)rational judgment, the experience-description distinction suggests a return to sequential learning. Such learning liberates respondents from making biased off-the-cuff judgments. Alas, experiential learning is expensive and time-consuming. Moreover, some decision tasks arise as perceptual challenges in our natural ecology and cannot be reframed as learning opportunities.
Intersecting the Two Distinctions
It seems now that we want a kind environment where we can experience and learn what we need to know. Let us consider the crossing of the two distinctions (see figure below). The research literature suggests that most studies on the distinction between kind and wicked environments allow experiential learning. The possibility of misguided learning (in wicked environments) disarms the conclusion that learning from experience is always better than learning from the description. In turn, most studies on the distinction between experience and description confound this distinction with the environmental distinction, such that most descriptions of judgment tasks contain wicked elements. In contrast, most experience-based studies provide kind learning environments.
As the untangling of the two distinctions remains a work in progress, some interesting results can already be noted. It has become clear, for example, that when judging from descriptions, people can extract conditional probabilities from natural frequencies more easily than from sets of conditional and unconditional probabilities (McDowell & Jacobs, 2017). People can, in other words, count very well and compute ratios, but they have a hard time mentally working through Bayes’s Theorem. We may now conclude that descriptions judgments can be sound if the environment is kind but that wicked environments degrade judgment regardless of experiential learning or perception and calculation.
These developments in the study of rationality have added much-needed nuance to understanding our mental strengths and shortcomings, and they have provided insights into design issues. They inform us how to configure judgment or choice tasks in a way that gives rationality its best shot.
Beyond Judgmental Psychology
We should be wary of broad declarations of the “We are all biased” or “Intelligence predicts rational judgments (and that’s all you need to know)” variety. Such declarations are little more than expressions of the fundamental attribution error – an error which, as we can now see, is itself modulated by presentation and environment (Krueger, 2009). To realize that our bad judgments are not entirely our fault is to reclaim an ounce of freedom.
Hertwig, R., Hogarth, R. M., & Lejarraga, T. (2018). Experience and description: Exploring two paths to knowledge. Current Directions in Psychological Science, 27(2), 123–128.
Hogarth, R. M., Lejarraga, T., & Soyer, E. (2015). The two settings of kind and wicked learning environments. Current Directions in Psychological Science, 24(5), 379–385.
Kahneman, D. (2011). Thinking, fast and slow. Farrar, Straus & Giroux.
Krueger, J. I. (2009). A componential model of situation effects, person effects and situation-by-person interaction effects on social behavior. Journal of Research in Personality, 43, 127-136.
McDowell, M., & Jacobs, P. (2017). Meta-analysis of the effect of natural frequencies on Bayesian reasoning. Psychological Bulletin, 143(12), 1273–1312.
Simon, H. A. (1969). The sciences of the artificial. MIT Press.