If you are at this page reading or listening to this post, you used some kind of interface with your computer to get here. Pretty much all of us have used a computer mouse. Or a keyboard. Or a track point interface. And, frankly, it really isn't that much fun. It is like we're constantly trying to use a bizarre translation device in order to do something that should be natural. The computer is a kind of tool; a tool we're trying to use to make our lives more efficient and easy. (If you aren't laughing at how often it really seems like we might be less efficient and doing it the hard way, you should be!)
The whole idea of using a mouse and keyboard or any kind of interface with a computer is to get it to do what we want it to do-allow us to communicate for example. And yet think of how you normally talk to people. You move your hands around, you gesture, if you want to point out something you literally use your hand to point at the thing you're talking about. There is a direct physical link between the action you're producing and the thing you're talking about. As extensions of these movements, simple tools like a pointer, a pencil or even a hammer are intuitively easy to use.
Even so, using a hand-held tool is more difficult than just using your hand and arm. Anybody (like me) who has ever thought "why don't I just pick up the golf ball and throw it at the hole?" while staring in frustration at the (my) obviously faulty golf club and then wistfully at their (my) ball way off in the trees, has realised this. I have always found this issue of tool use to be really interesting. I noticed something early on in my training when using Okinawan martial arts weapons like the long staff (bo) or the side-handle baton (tonfa). Learning and remembering motor skills with these tools was even more difficult than without tools. This means empty hand martial arts like karate are easier to remember than similar activities with weapons. This is probably because the tools that we use are not "calibrated" as parts of our bodies.
Even though you aren't aware of it a continuous calibration of your body parts has been occurring in your brain throughout your life. This process results in a kind of mapping that goes on in the sensory and motor parts of your brain. You have had those maps of the skin areas on your body and the muscles in your limbs in your brain since you were a baby. They have been calibrated and tuned over the years along with your changing body size and the things you've done. Except in cases of tragic accidents where a limb may have been lost or amputated, your body has always been there with you 24/7.
Tools that we use, though, aren't with us all the time. We typically use tools only when we need them (we always need our bodies). It turns out that the sensory maps of our bodies can be reshaped to include parts of the way we use tools. But it seems these changes in the maps are likely weak compared with those for our actual body parts. They need much more "maintenance" activity than do the maps that represent our actual body parts. The maintenance activity, or practice, is the stimulus to incorporate the functional properties of the tools into our body maps. This kind of melding with the tool is termed "embodiment" and reflects the plastic changes that your nervous system undergoes to keep you as functional as possible. This process is heavily influenced by the sensation of moving the tools and the visual input that you get from seeing yourself using the tools.
Back in 2009 a team of French and Italian scientists headed up by Lucilla Cardinali, confirmed this plasticity in a really simple but clever study. They developed a long hand-held extension of the arm that had a "grabbing" piece at the end. It was very similar to the kind of tool that is used to clean up trash from parks and streets without having to bend down. By squeezing one end the tool grabbed items at the other end. The researchers had participants practice using the grabber to pick up and move things around. It turns out that practice in using the grabber changed the arm movements performed even when they didn't use the tool! It even led to changes in pointing movements they made and in how long they perceived their arms to be. They thought their arms were longer, presumably because the tool allowed them to reach further. In fact, from a functional perspective within the brain, their arms were longer since they could reach further with the grabber.
The explanation for this plasticity is related to body maps in the brain changing as a result of using the tool. The tool gives you some different abilities, like reaching further in this example, and this change in function pushes the plastic changes in the brain. How long those changes last and how strong they can be is uncertain. Could the changes become durable enough to become real memories for a new map? We know that limb amputation can lead to changes in the maps. It leads to emptying some territory in those maps and taking over of territory by brain cells for other regions. The opposite perspective, that is, what happens when you add something to a map that is already complete, isn't well understood. A few studies looking at monkeys practicing to use neural prosthetics controlled by the brain show very strong changes over only a few weeks. The changes are in features of long-term memories. In fact, the term "prosthetic motor memory" is used to describe this.
I had been thinking about these ideas for quite some time but it wasn't until I wrote Becoming Batman and then Inventing Iron Man that I really got interested in getting answers. During that time I acquired (and continue to have) extremely painful tendonitis in my elbows. It is made much worse by typing (like I'm doing right now of course.) A few times I have tried to use voice-recognition software so I can speak what I want to write. This software takes a bit of learning and works pretty well overall, but there is also a fair bit of frustration associated with them. You need to "train the software" and even then things don't always go smoothly. I tried using voice recognition software for part of this post. Mostly it was okay. But in this very paragraph I wanted to delete the word "that" in a previous sentence. When I gave the verbal command "delete "that"" my computer just kept re-writing "delete that". Over and over. Of course I also tried using the method relied on by tourists the world over. You know, if someone speaks a different language and cannot understand what you are saying, simply say it louder! After that predictable failure, I just went back to typing.
The bottom line is I felt like I was using my voice interfacing with my computer as a tool to write. That got me to thinking wouldn't it be better to just write with commands from my brain? Normally when I want to type I produce activity in my brain that would activate neurons in my spinal cord which then activate my muscles to produce the words I want to type. Let's cut out the "middle man" and go directly from the activity in my brain to the computer. That's an example of a brain-machine interface and is really taking tool use to extremes.
The thing I really began to puzzle over when writing my recent books, is what happens when you take tool use to such an extreme? What would happen in the brain if the tool is a representation of the body? That was something that occupied a lot of my attention when thinking about the Marvel Comics character Iron Man and his robotic suit of armor as a metaphor for brain-machine interface. To be fully functional I suggested it would have to be a tool connected to and controlled by the brain. In other words, the ultimate brain-machine interface. What would happen to the body maps if we increase the representation of the body in the brain without first taking something away? Would the neural plasticity associated with this affect the connection between your brain and your real body? If you keep going on this line, it takes you to the point to suggest that tools (like technology) that are connected to the nervous system could affect control of the body. And the integrity of the body and brain. And maybe even what it really means to be human.
So here we are, back to where we started with the current clunky state of how we interface with technology. Yes it isn't great, but if we take it so far that it is seamless, are we really ready for what may lie ahead? For now I'm resigned to sticking with my keyboard. And mouse...and occasionally talking to my computer.
© E. Paul Zehr (2011)