Google and Facebook AI Make New Linguistics Discovery

AI used to understand the emergent phenomena of natural language

Posted Feb 03, 2019

Source: geralt/pixabay

At the intellectual crossroads of science and humanities is linguistics, the scientific study of language. The structure of linguistics touches across the disciplines of psychology, neuroscience, biology, and philosophy. Language is one the most fundamental defining characteristics of humans, and yet its origin remains a scientific mystery not only to linguists, but also psychologists, neuroscientists, anthropologists, biologists, and archaeologists. How did human language emerge and evolve? The challenge to solving this mystery is largely due to the scantiness of empirical evidence. The other roadblock is time—it could take many years, even centuries, to observe and understand the patterns of emergence and evolution of natural language. Recently, researchers from Google AI, Facebook AI, and New York University deployed AI deep learning to simulate and understand the emergent phenomena of language, and released their findings in arXiv in January 2019.

The research team of Laura Graesser, Kyunghyun Cho, and Douwe Kiela used the latest in artificial intelligence (AI) techniques to create a computational framework “in which agents equipped with communication capabilities simultaneously play a series of referential games, to study the properties of natural language.” According to the team, theirs is a novel framework because they used the latest-generation deep reinforcement learning that could process rich sensory input.

The multi-agent computational framework uses neural networks that can exchange messages about their perceptual input. The components of the computational multi-agent model consists of agents, learning algorithm, environment, and reward mechanism. The agent used ranged from the simple to complex, and included difference equations, a “CPU-like architecture with an instruction set and registers,” a “co-occurrence matrix between objects and symbols,” a single-layer neural network, and a deep neural network. The learning algorithm used was either a gradient-based optimization or a variation of evolutionary algorithms.  

The researchers found that “the success rates between self-play and paired-play are indistinguishable from each other, strongly implying that a common, shared language emerges as a social convention if and only if we have more than two language users,” and all “that is needed in order for a common language to emerge is a minimum number of agents.”

Next the team ran simulations at the community level. They wanted to understand what would happen if two different communities with different languages come in contact. The team discovered that inter- and intra-group connectivity are important factors in determining the level of language convergence. With enough inter-group connectivity, languages become mutually understandable through contact, regardless of whether or not the agents have been exposed to the other language.

The team learned that with linguistic contact over time results in the dominant majority protocol taking over and the other language disappearing. If the communities are balanced, a new “creole” protocol that is simpler than the original languages emerges. Neighboring languages are more mutually understandable, and communicability decreases as the distance between communities increases. The researchers discovered that “intricate properties of language evolution need not depend on complex evolved linguistic capabilities, but can emerge from simple social exchanges between perceptually-enabled agents playing communication games.”

Now scientists have a sophisticated tool to study the evolution and emergent characteristics of natural language. The research findings could potentially impact theories on the origin of language and provide a greater understanding on one of the defining characteristics that make humans unique.

Copyright © 2019 Cami Rosso All rights reserved.


Graesser, Laura, Cho, Kyunghyun, Kiela, Douwe. ”Emergent Linguistic Phenomena in Multi-Agent Communication Games.” arXiv. 25 Jan 2019.