Verified by 
Kurt Lewin Did you hear about the university professor who got fired because he totally made up his theory?
Kurt Lewin
Did you hear about the university professor who got fired because he totally made up his theory?
Johan Galtung (1981) once described four great cultures of science. The Saxonic culture prizes observation and data; it is skeptical of grand theory. The Teutonic culture has the opposite preferences. It wants theory and logical discipline. The Gallic culture is concerned with élegance. A good theory pleases aesthetic sensibilities. Finally, the Nipponic culture seeks social harmony above all else. Sure, Galtung’s description was an idealization, or an act of gross stereotyping if you will. Nonetheless, there’s more than a grain of truth here, as visitors of international science conferences can confirm. Galtung himself invited us to imagine such a conference. A speaker presents a piece of scholarship and four colleagues respond. The Saxon asks “Do the data support your hypotheses?” The Teuton wants to know “Do your hypotheses follow from your theory?” The Gaul is interested in whether “On peut dire ça en bon français?” And the scholar from Nippon inquires “Who is your master?” (see also post culture shock for nerds)
The culture of contemporary social psychology is Saxonic. It is heavily empiricist and thin on theory. This was not always so. The Mid-20th Century was an era of theory, some of which grand, and a few theories were highly formalized. Kurt Lewin’s vision of a theoretically grounded social psychology had a Teutonic aspect, not surprisingly perhaps, since Lewin’s views on science were shaped on the continent. Over the years, data came to dominate, and the flight from theory accelerated. Lewin’s field theory is all but forgotten.
One trend in the move toward empiricism is the adoration of “Big Data.” Thanks to technological progress, quantitative limitations are falling away. As data files grow, the vision of being able to attain a complete picture of reality gains credibility. Big Data says that samples will go the way of the mammoth, and the skills necessary to take them will die with those who still practice them today. When populations replace samples, all we need to do is look. That, at least, is the idea. When scientific work reduces to observation, no one needs a theory. Theory, after all, only models reality and reduces it to digestibility. When the whole thing can be swallowed, theoretical reduction becomes an anachronism, or worse, an obstruction.
But, Big Data misunderstands theory. The word theory stems from the Greek idea that looking is an active process. ‘Regarding’ might be a better term. Observation is not a passive taking in of information. Theory tells us what to look for, how to look at it, and how to make of what we see. This is a fundamental psychological point. Research psychologists themselves champion this view when describing how visual perception works. Data, no matter how big, cannot speak for themselves.
Big Data has not supplanted “little data.” Indeed, there seems to be more little-datering as ever. Little data is about sampling and making inferences. Inferences are crucial because the whole is not observed directly. The critical maneuver performed on the data is statistical significance testing. A minimal experiment has one condition with an intervention (e.g., the delivery of a persuasive message) and a control condition without. Significance testing computes a probability of the observed difference on a dependent measure (e.g., a measure of attitude) between conditions under the assumption that there is, in fact, no difference. The no-difference assumption is the so-called null hypothesis. It has no theoretical pretension. It is a foil. If the null is rejected because the computed probability is low, researchers conclude that “there is not nothing” (to use Robyn Dawes’s phrase, which aptly captures the double-negative logic of null testing).
Despite the propaganda for Big Data, little data dominate the literature. Many studies yield probability values just low enough to support the claim that there is not nothing, and it is not without irony that the claim that there is a suspicious peak in just-low-enough probabilities itself rests on a probability just low enough to make this claim. Like Big Data, little data is spiritually atheoretical. Big Data science is unabashedly so, whereas little data science still clothes its empiricism in the mantle of theory. Hypotheses are, after all, being tested.
Little data science is vulnerable to the temptation of telling stories to explain the data after the fact. If one accepts the assumption that observation is all that matters, there is nothing wrong with post-hoc narratives. Only the view that science ought to be able to predict (and thereby explain) phenomena on theoretical grounds implies that story-telling creates an illusion of understanding. Theory-based science requires a priori assumptions from which testable hypotheses can be derived. That, at least, is the Teuton’s way of looking at it.
Meanwhile, the little data paradigm sees its greatest threat in the so-called replication crisis. Faith in the reality-revealing power of the p value (offered by significance testing) has been eroding. When a null hypothesis has been rejected (p < .05), there remains great uncertainty about whether it would be rejected again if the experiment were repeated. Empirical attempts and computer simulations suggest pessimism in this regard. Many believe that the only way out of the crisis is to run many replications until the true effects stand out beyond questioning. In other words, the little data paradigm wants to get into the business of Big Data. With that, the role of theory shrinks further.
When theory is exorcized, what is left is magic. In 1967, Berkowitz & LePage published a seminal paper on the weapons effect. They showed that individuals who had been provoked responded with greater aggression if a rifle lay in plain view (as opposed to a badminton racket). Their work was embedded in a complex and evolving theoretical framework, which Berkowitz summarized in 1990 as a cognitive-neoassociationistic approach to aggression. Berkowitz’s theory was broad, rich, and subtle. It built a bridge from the early frustration-aggression hypothesis to the present-day priming paradigm. Bargh and colleagues (1996) ramped theory down by claiming that a stimulus directly activates behavior. Theoretical psychology was reduced to cognitive entities (ideas) that are semantically related to both the stimulus and the behavior. When, after being exposed to words related to old age, participants walk more slowly, it must be because the idea of an old-age shuffle was activated and that this idea then controlled behavior. The proposed cognitive mediation does not seem to add much beyond the re-description of the observed stimulus-response association, though. Yet, it seems necessary. If one were to claim a truly direct effect of the stimulus on the response – without an intervening mental construct similar to both – the magicalness of the result would be unbearable.
Rigorous reliance on data, to the detriment of theory, makes fraud more likely. Mr. Stapel had his co-authors and the gatekeepers at Science Magazine convinced that meat eaters are more selfish than vegetarians. He had, he claimed, the data to prove it, and how can you argue with the data – as long as you believe they are legitimate? Theory was no help because Stapel committed his fraud (this one and many others) in a theoretical wasteland. He did not, however, operate in a no man’s land of folk psychology and story telling. Not having read the paper, since it was thankfully never published, I don’t know the tale Stapel told, but here’s my own version of what it might have been: Wanting meat requires a willingness to hunt and kill, or at least a willingness to let someone else kill on one’s behalf. Hunting and killing require aggression (if not the red-hot kind, then the more cerebral instrumental kind). Aggression in turn implies self-regard and selfishness. So the dots are connected. It was easy to think up this chain of associations, which suggests that Stapel wrote for an audience that was ready to perceive as true what he claimed was true.
Theory cannot be faked, but it can – and should – help keep empiricism honest. An often overlooked quality of good theory is that it tells us what cannot happen (see here for an application to parapsychology). Good theories rule out as much as (or more than) they allow as possible. If you fake data in a domain ruled by strong theories, the result (e.g., cold fusion, mind-bent spoons) is recognized as false before the author confesses. Unfortunately, much of current social psychological work is done is a detheorized environment. Any significant effect can demand attention as long as it is ‘interesting’ and delivered with a good story. A series of failed replications, published in journals dedicated to failed replications, will be slow in making a dent in the public acceptance of the ‘finding.’
Bargh, J. A., Chen, M., & Burrows, L. (1996). Automaticity of social behavior: Direct effects of trait construct and stereotype activation on action. Journal of Personality and Social Psychology, 71, 230-244.
Berkowitz, L. (1990). On the formation and regulation of anger and aggression: A cognitive-neoassociationistic analysis. American Psychologist, 45, 494-503.
Berkowitz, L., & LePage, A. (1967). Weapons as aggression-eliciting stimuli. Journal of Personality and Social Psychology, 7, 202-207.
Galtung, J. (1981). Structure, culture, and intellectual style: An essay comparing Saxonic, Teutonic, Gallic and Nipponic approaches. Social Science Information, 20, 817-856.
