Exploration Versus Reward in Toddlers

A new study examines the decision-making patterns of toddlers versus adults.

Posted Aug 31, 2020

One of the first things any parent notices about their growing infants and toddlers is their enhanced ability to explore. But what is it that motivates young minds to do so? Is it simply that kids want to reduce the uncertainty in their environment by learning more about it? And is their exploratory behavior similar to more mature exploration exhibited by adults? A recent study published in the journal Developmental Science examined these questions and more.

The authors of the study, Nathaniel Blanco and Vladimir Sloutsky, describe in their paper the conflict that adults and children alike face between “exploration” and “exploitation.” Successful decision making in many aspects of life requires a careful balance between gaining new knowledge (exploration) or utilizing existing knowledge to gain rewards (exploitation). Of course, the choice between one or the other depends on several factors, including how much the individual already knows, and how uncertain or variable the environment is.

Given that toddlers and children are just building up their knowledge of the world, it is not surprising that exploration is of prime importance to them. Studies have shown that even infants show signs of exploration, interacting with or looking longer at a stimulus that was unexpected. 

Blanco and Sloutsky, in this study, set out not just to explore how toddlers decide between exploration and exploitation when faced with a series of decisions, but to try and gain a deeper understanding of their choices. Current theories of decision making propose that children’s exploratory behavior is largely unsystematic since “systematic” exploration requires the involvement of brain regions in the prefrontal cortex of the brain that is typically not fully developed until adolescence. This study, however, produces results that show that children as young as 4 years old not only engage in more extensive exploration than adults, but that this exploration is indeed systematic and driven at least partially by uncertainty.

What exactly is systematic exploration, though? Researchers describe it as an exploration that is directed towards the parts of the environment that are highest in uncertainty. With time, uncertainty increases in areas that have not recently been explored. The individual must go back and explore those areas again. This generates a “systematic,” sequential structure to people’s choices.

In this study, 32 4-year-olds and 34 adults were recruited to participate in a virtual experiment designed as a game, in which the goal was to collect as much candy as possible from characters displayed on a screen. There were four characters displayed on the screen in each trial, and each of these four characters would always give the same amount of candy, although the participants were not aware of that fact at the start of the experiment. Characters A, B, C, and D gave one, two, three, and 10 candies, respectively.

Trial one would go something like this—a screen would pop up with four characters on it. The participants would go ahead and select one of these, and a screen would then show them how many candies they earned from this trial.

Since the number of candies produced by each of these characters was fixed, adults very quickly learned which character was giving them the most candy and, as would be expected, went on to keep selecting them in subsequent trials to gain the most candy (only very occasionally exploring once they had learned the reward values). Interestingly, although the toddlers learned as well as the adults about the reward values, they exploited the best option much less than the adults, choosing to continually explore all the available options instead.

The authors write that “these findings suggest that children’s choices are not motivated by achieving maximum reward to the extent that adults are.”

Also, more interestingly, the study found that the children’s exploration was “far from random.” The authors analyzed the patterns of the choices that the children were making using modeling analyses and found that these were consistent with systematic exploration.

Finally, to examine the effect of uncertainty on the choices the toddlers were making, the authors of the study designed another experiment. Here again, the aim was to select one option per trial and collect as much candy from these as possible across multiple trials.

But in contrast to experiment one, where the participants did not know which of the four characters gave how much candy at the start of each trial, in this instance, the “candy value” of three of the options was visible to the participants, with just one being hidden. Which of the four options would be hidden was randomized—in each of the trials, a different option would be hidden. But the “candy value” of all four of the options never changed, even when it was the hidden one.

The design of experiment two essentially “concentrated” the uncertainty of the experiment to just that one hidden choice. If the children stopped systematically exploring like experiment one and centered their choices around the hidden (uncertain) option, it would mean that uncertainty does play a role in their exploratory behavior.

And that is exactly what happened. While adults ignored the hidden option, except in times when it was the highest value option, children overall interacted with the hidden option much more.  

But while toddlers as a group explored the hidden option more than the adults, a closer look at just the toddler group revealed that some children avoided the hidden option altogether, while others were drawn strongly towards it. More research is needed into what causes this inter-individual difference, but the authors suggest that it could be that the ideal level of uncertainty (beyond which children start avoiding high-uncertainty targets) could vary between children. Also, some children might find the visible rewards in experiment two too appealing to bother about exploring the hidden option.

Finally, the authors address how they think systematic exploration might be working in toddlers, given that they don’t yet have the “top-down” prefrontal cortex machinery developed enough to achieve it in a traditional manner. Other research has shown that children tend to allocate their attention broadly—unless a very pressing stimulus captures their attention, they are not as “focused” on a task as adults may be. Since attending to a stimulus in a focused and selective manner is important for the exploitation of the stimulus for reward, the authors suggest that the tendency that toddlers have to distribute their attention in a more broad manner could be an important mechanism supporting their exploratory behavior.

Although it might seem that young children’s exploration of their environment is random, scientists are now learning that it is indeed quite systematic and driven at least partly by uncertainty.

References

Blanco, NJ, Sloutsky, VM. Systematic exploration and uncertainty dominate young children's choices. Dev Sci. 2020; 00:e13026. https://doi.org/10.1111/desc.13026

Sim, Z.L. and Xu, F. (2017), Infants preferentially approach and explore the unexpected. Br J Dev Psychol, 35: 596-608. doi:10.1111/bjdp.12198