Can Cognitive Testing Really Measure Our Brain Function?

In 1960, an ingenious experiment was designed to test the effects of high altitude and fatigue on cognition. The participants were all part of the ‘Silver Hut Expedition’ – a Himalayan excursion led by the legendary Everest climber Sir Edmund Hillary. While sitting in their flimsy tents at 5800M, a group of climbers were challenged to sort cards into categories, according to their shapes, colors and so on. The results showed that accurate work was possible at high altitude, but it just took longer.

In the sixty years or so that have passed since the Silver Hut Expedition, many of the ways that we think about assessing cognitive function remain unchanged. We cling faithfully to tests that were designed in the 1950s and 60s to assess aspects of performance, long before we knew very much at all about the relationship between the brain and behavior. Many of these tests are based on outdated concepts like ‘IQ’ – I say ‘outdated’ because they were developed before (and therefore take no account of), the revolution in neuroscientific understanding that has occurred over the last 25 years. But then, they were never designed with the brain in mind. The Himalayan experiment was motivated by the eminent Physiologist and Biochemist, Sir Joseph Barcroft who had noted a certain amount of “bumbling at high altitude” during an expedition to Cerro de Pasco in Peru in 1920–21. In 1960, card sorting was how Psychologists operationalized and measured “bumbling”, but that’s all it was; a more formal measurement of performance.

In the late 1980s, I was part of a team at the University of Cambridge, U.K., that developed and tested the first touch-screen based, computerized battery of cognitive assessment tools, designed specifically to test human brain function. The tests were based in large part on emerging scientific literature from neuropsychological studies of patients with damage to different parts of their brains. Those studies were beginning to reveal that the functioning (and ‘dysfunctioning’), of specific brain regions, including those in the frontal cortex and the temporal lobes, could be assessed directly with carefully controlled tests of memory, attention, problem-solving, reasoning and planning – all so-called ‘higher cognitive functions’ that are relatively impervious to level of education, yet characterise many of the world’s most successful people. These computerized touch-screen tools, while faster and more accurate than traditional tests of performance, were also a whole lot more powerful in terms of the conclusions that could be drawn about an individual and, more importantly, about their brain. Put simply, because they measured how well particular regions of the brain were functioning, rather than just how good a person was at any one test, the results were predictive of performance across a variety of everyday situations requiring those parts of the brain. It was the appliance of neuroscience to cognitive assessment.

In the mid-1990s, I spent three years at the Montreal Neurological Institute (MNI) in Canada, which is arguably the birthplace of human Neuropsychology and its modern-day incarnation, Cognitive Neuroscience. Functional neuroimaging techniques like positron emission tomography (PET) and functional magnetic resonance imaging (fMRI) were just beginning to take off, allowing us, for the first time, to probe the inner workings of the healthy human brain while it went about its business. I scanned the brains of hundreds of volunteers while they took our computerized tests showing, for example, that performance on any task requiring ‘working memory’ is the result of a fine interplay between different regions of the frontal lobe. Working memory is a special kind of memory that we only need to hold on to for a limited period of time until that information is no longer needed; for example, where we parked our car this morning. Crucially, however, these types of studies also showed that the frontal lobes aren’t just involved in tests of working memory; they contribute to many aspects of daily living, including attention, problem-solving, planning and decision making.

The fact that we were now able to measure how different parts of the brain – like the frontal lobes - contribute to each of these general cognitive functions, made it possible to start to make predictions about how individuals are likely to perform in many different day-to-day situations that require those parts of the brain, rather than relying solely on performance on any one task, like card sorting.

A new generation of online cognitive tools are able to measure how specific regions of our brain are functioning, rather than just how good we are at any one test.

Source: Adrian Owen

But the true potential of these emerging technologies is only being fullyrealizedd now that the World Wide Web is such an integral part of all of our daily lives. In the late 2000s, back in Cambridge, U.K., we reconfigured our tests of cognitive assessment for delivery over the internet. Although faster, slicker, and more sensitive than the originals, they retained all of the essential neuroscientific ingredients that 25 years of data collection had taught us were essential for understanding differences in human brain function. In 2010, we were approached by the BBC's 'Bang Goes The Theory' who asked whether this approach could be used to test the claims being made about ‘brain training’. Over 6 weeks, 11,700 members of the public kept up a regular regime of brain training using versions of some of the most popular commercial games on the market, and we tested their brains, both before and afterwards. The results, published in the journal Nature, were unequivocal. While brain training improved performance on every test that was trained, there was no overall improvement in cognitive function. In short, practise improves performance, as it does in every aspect of life, but it doesn’t make you smarter. In 2012, we took on the concept of IQ. This time, 44,600 participants took our tests to see whether it really is true that some people are smarter than others and that this ‘individual difference’ can be sensibly reduced to a single number – your so-called ‘IQ’. The results, published in the journal Neuron, showed quite clearly that human intelligence is not supported by a single neural system.

Not that any of this was surprising; after 30 years of functional neuroimaging, if there was an ‘IQ spot’ in the brain, then someone would have found it by now.

These tests, based on three decades of neuroscientific investigation, have now been taken more than 8 million times and are finding new applications in healthcare assessment, pharmaceutical testing, clinical assessment and education. What are the downsides? Some fear that testing brain function is a step too far, an invasion of one’s personal privacy that might reveal information best left undisclosed. But brain function is just another measurement, like height, heart rate, or performance on those traditional psychometric tests that many have relied on for so long.

If Hillary's climbers were sitting in their tents today, they would not be sorting cards. They would be logged on to the internet while their brains were monitored in real time and compared to the brains of millions of others at sea level, for signs that altitude and fatigue were affecting their cognitive function. We’ve come a long way in 60 years.