New AI Trains Robots Like Dogs

Johns Hopkins University develops framework to accelerate AI machine learning.

Posted Oct 28, 2020

ClaireDiaz/Pixabay
Source: ClaireDiaz/Pixabay

Can robots be trained with positive reinforcement just like dogs? Reinforcement learning is an artificial intelligence (AI) concept inspired by behavioral psychology. Researchers at Johns Hopkins University have developed a new method to train robots that efficiently improves upon state-of-the-art reinforcement learning artificial intelligence algorithms, and published their study in this month’s IEEE Robotics and Automation Letters, a peer-reviewed journal.

The research team of Andrew Hundt, Benjamin Killeen, Nicholas Greene, Hongtao Wu, Heeyeon Kwon, and Gregory Hager from Johns Hopkins University, along with Chris Paxton of NVIDIA, created a new approach to AI reinforcement learning, called the SPOT (Schedule for Positive Task) Framework, a model aptly named given dog training served as the inspiration.

The SPOT framework greatly reduces the amount of training required in robotics. According to the study’s lead author Andrew Hundt in a YouTube video, what would normally take a month of training could be reduced to two days.

Robots are as smart as their AI training algorithms, but AI lacks common sense. So how to train a robot efficiently? The SPOT framework provides common-sense constraints that speed learning and robotic task efficiency. For any actions that reverse progress, the system does not provide reward nor punishment.

“We have demonstrated that the SPOT framework is effective for training long-horizon tasks,” the researchers reported. “To our knowledge, this is the first instance of reinforcement learning with  successful sim to real transfer applied to long-term multi-step tasks such as block-stacking and creating rows with consideration of progress reversal.”

In behavior psychology, a reinforcement is a resulting consequence. “The consequences of behavior determine the probability that the behavior will occur again,” said B.F. Skinner (1904-1990), an American psychologist, author, inventor, professor, and pioneering behaviorist. For example, patting a dog’s head affectionately and saying “good dog” in a pleasing tone of voice can be used as a positive reward in reinforcing desired behaviors.

“Consider the goal of training a dog, 'Spot,' to ignore an object or event she finds particularly interesting on command,” the researchers wrote. “Spot is rewarded with treats whenever partial compliance with the desired end behavior is shown, and simply removed from regressive situations with zero treats (reward).”

The results were favorable. “The SPOT framework successfully completes simulated trials of a variety of tasks, improving a baseline trial success rate from 13% to 100% when stacking 4 cubes, from 13% to 99% when creating rows of 4 cubes, and from 84% to 95% when clearing toys arranged in adversarial patterns,” reported the researchers.

Now with the SPOT framework, multi-step robot tasks can be trained more efficiently than existing reinforcement learning algorithms alone. The intersection of behavioral psychology and artificial intelligence is producing novel approaches in machine learning algorithms with improved results.

Copyright © 2020 Cami Rosso All rights reserved.