Why Run-of-the-Mill Rewards Might Lose Their Appeal

Source: Tumisu/Pixabay

New research sheds light on that "jackpot feeling" one gets after unexpectedly winning a rare prize, and, conversely, on the relatively blasé feeling of receiving a predictable reward that is nothing to write home about. This paper, "Rare Rewards Amplify Dopamine Response," appears in the April 2021 issue of Nature Neuroscience.

Because dopamine neurons play a pivotal role in reward-based learning but relatively little is known about how the rareness of a reward affects individual dopamine neurons, the University of Pittsburgh researchers investigated "how the frequency of rewards and reward prediction errors influence dopamine signals" in the midbrain.

"We were curious to know how dopamine neurons would respond if the range was constant, but the relative probability of rewards within that range changed," senior author William Stauffer said in an April 2 Medical Xpress article. "Accordingly, the main objective of our study was to learn whether dopamine neurons were sensitive to the shapes of probability distributions."

"Reward prediction errors are crucial to animal and machine learning," he added. "However, in classical animal and machine learning theories, the 'predicted rewards' part of the equation is simply the average value of past outcomes. Although these predictions are useful, it would be much more useful to predict average values, as well as more complex statistics that reflect uncertainty."

Rare Juice Rewards Amplify Dopamine and Get Monkeys' Juices Going

For this study, Stauffer and colleagues at the University of Pittsburgh's Center for Neuroscience and Pitt's Brain Institute set up an experiment in monkeys that paired visual cues with different "reward probability distributions" based on small, medium, and large drops of a fruit juice reward.

As an example, one reward probability distribution consistently gave each monkey medium-sized juice drops; however, on rare occasions, a monkey might receive a smaller or larger juice drop than expected. In another distribution pattern, the monkeys predictably received an equal number of small, medium, and large juice drops with equal probability.

Then, the neuroscientists used electrodes to record dopamine responses in reward centers of the brain as monkeys viewed visual cues paired with different reward probability distribution patterns.

Blasé Rewards Don't Amplify Dopamine

Notably, Stauffer and co-authors found that the juice drop rewards that seemed rare amplified dopamine response in the brain. On the flip side, highly probable rewards of the same size that were commonplace and predictable evoked a weaker dopamine response.

As the authors summed it up: "Rare rewards amplified dopamine responses, even when conventional prediction errors were identical, indicating a mechanism for learning the complexities of real-world incentives."

"This study was a first step towards understanding how subjective reward probability distributions are coded in the brain, and what form these beliefs can take," Stauffer concluded. "With these results at hand, we will now get back to studying choices. Nevertheless, I suspect these results will have broader implications and also be important for biological and artificial intelligence-based learning systems."