Mind Games

Examining the intersection of psychology and digital entertainment

Can The Xbox One's Kinect Read Your Mind?

Microsoft's new hardware might lead to new advances in game design.

Well, no. Of course not. That's a silly question. Why would you even ask it?

The updated supercamera on the Kinect 2.0 has been shown to be capable of some pretty amazing things, though. Microsoft demonstrated how it can tell where you're looking, estimate your heart rate from the color of your skin, and even infer your mood from your facial expressions. Finally, it has a sophisticated voice recognition system and the ability to see in the dark, which will come in handy when it wants to sneak into your bedroom at night and listen to your breathe for hours on end.

And though it hasn't been discussed, I wonder if the Kinect 's high definition camera could be programmed to measure one other important biometric: pupil dilation. This would be both awesome and worrisome, because while not exactly a mirror into our souls, the eyes can reveal a lot about what goes on in our minds.

Daniel Kahneman wrote in his book, Thinking Fast and Slow, about how pupil dilation is a darn good proxy for mental effort. In a series of experiments he asked people to take a multidigit numer, then increment each digit in the number by one to form a new number. So 348 would become 459. The would then do the same to a new number, using a metronome to do one new number/sum every two seconds.

Find a Therapist

Search for a mental health professional near you.

Try it yourself and you'll see that the task gets very difficult pretty quickly.If not, try adding 2, 3 or even 4 to each digit instead of 1, Brainiac. And if you had someone eyeball your eyeballs they would clearly notice that your pupils would steadily grow larger and larger as the mental machinery behind them started to work harder and harder --right up until the point where you gave up, when they would snap back to normal size.

In what sounds less like science and more like an exhibit you'd make a face at while visiting the museum of contemporary art, Kahneman and his colleagues would train a camera on subjects during these experiments and broadcast an enormous image of their eye onto a television in the hallway so that the pupils were about a foot wide and dilation was easy to measure. The results were pretty consistent: the more mentally taxed we are, the bigger our pupils get.

If the Kinect (or any camera) could detect pupil size, it would open up a whole new level of scaling game difficulty. A puzzle game could be made more and more difficult until you're taxed just the right amount to get you in the zone --something psychologists call "psychological flow." Imagine playing a rhythm game like Guitar Hero and having the game adjust the speed of the note highway until you're pushed just to the brink of your abilities based on how hard you're concentrating on the task.

Or what about knowing when to offer you a helping hand? If the Kinect can tell the point at which you've given up on a puzzle or sequence because your pupils shrink back to normal, it might offer you a hint. Possibly in a condescending tone.

Another, more unsettling implication would be that if the Kinect could tell when you are stressed and mentally taxed, it could use that opportunity to sell you stuff. Willpower is like a muscle that can be exhausted by any mental activity, and when it's depleted we're more likely to do dumb stuff like make impulse purchases or, one might imagine, place an impromptu Skype call to an ex-boyfrined when we really should know better.

Jamie Madigan, Ph.D. is an industrial-organizational psychologist, writer, and life-long video game enthusiast.

more...

Subscribe to Mind Games

Current Issue

Just Say It

When and how should we open up to loved ones?