A GPS for the Past Can Be Found in Language

Recently, exciting news appeared that the secrets of the Dead Sea Scrolls may be revealed by using artificial intelligence technologies.

Using deep learning techniques and artificial neural networks, researchers trained algorithms to analyze images of the 2,000-year-old scroll that preserved 66 chapters of the Hebrew Bible’s Book of Isaiah. Pattern recognition on the microlevel of handwriting was then used to identify writing styles, demonstrating that multiple scribes carefully copied each other's writing style and could work together on one particular manuscript.

Artificial Intelligence offers new avenues for historical text analyses to recover information from the past. It provides a great opportunity to better understand historical texts—and not only when it comes to author identification. These techniques might even be helpful for getting access to the contents of a historical text. This goes as far as estimating the longitude and latitude of locations on the basis of which they are described in the (historical) text.

Let me try to illustrate using a modern example.

Imagine that we were to count the number of times that Boston is mentioned in the same sentence as Chicago, Minneapolis, Los Angeles, and Miami, and we were to do this for all these four cities. The result is a matrix that consists of 5 x 5 frequencies. By using a dimensionality reduction technique called multi-dimensional scaling, we can now compute the loadings of these five cities on two dimensions based on their frequencies of co-occurrences.

The end result is fascinating. The loadings on two dimensions correlate with the actual longitude and latitude of these cities. To make a long story short: Cities that are located together are mentioned together, and based on cities that are mentioned together, we can estimate where they are located in relation to one another.

You may wonder what the psychological implications are. Bear with me. First, let me give the evidence. In one study, we took a year of newspaper articles—from the New York Times, the Wall Street Journal, and the Los Angeles Times—and marked the number of times a city name co-occurred with another city name for the 50 largest cities in the United States. Doing this manually might not be the most efficient way of spending your time. A computer can of course do this much faster and more efficiently, and can do it at more complex levels. But let’s keep the story simple. Based on the frequencies and the dimensionality reduction technique, the loadings of the cities correlated with the longitude and latitude of the actual cities! And not for just one newspaper, but for all three!

We put this to the test further by estimating the location of cities in China using Chinese texts, and the location of cities in the Middle East using Arabic texts. And for those of us who like to escape into fictional worlds, we took Lord of the Rings, threw it in the computational meat grinder, and out came the longitude and latitude of locations in Middle Earth.

The exact same technique can come to the rescue of archaeologists. In yet another study that used the same technique, the Indus Script was analyzed. That study investigated whether the geographic origin of artifacts from the Indus Valley Civilization, the earliest urban society on the Indian subcontinent, could be estimated by applying the co-occurrence technique to the Indus Script. Results showed that the relative locations of archaeological sites could be estimated based on the artifacts of known provenance. Moreover, the most probable excavation sites of four sealings of unknown provenance could be predicted.

In the analyses of these studies, we found something else. The frequency with which these cities were mentioned in the text correlated with the population size of the cities. If a city is populated more, it is mentioned more. In fact, the computational estimates were better than human estimates.

So if have a historical text and would like to extract geographical information from it—whether it concerns excavation sites, location, or population size—a simple technique of looking at the context of words could come to the rescue.

One may argue that artificial intelligence works in mysterious ways. Deep learning seldom allows us to look closely at the mechanisms behind its computations. But when it comes to predicting the longitude and latitude of cities using language, the magic does not come from the computational techniques that are used. Instead, the magic comes from the language system itself.

The way language is organized is that cities that are located together are most likely mentioned together. Even though we can choose any topic to talk about and have complete freedom in the way we use words, there is a structure to it such that the geographical world gets encoded in language.

What is stated here about artificial intelligence actually says something about human intelligence. If the perceptual world has been encoded in language, this has major psychological implications for why humans are so good at language processing—and why a child anywhere in the world can acquire language without any form of formal instruction.

Most literature in the psychology of language has argued that the explanations for language acquisition are that we hear language from our parents, that we have specialized brains for language processing, that we have mighty neural networks of the non-artificial kind, or even that we have incredible memory skills to keep so many words in mind. What distributional techniques show us is that in addition to these explanations for language processing, another one may have been overlooked—and that explanation lies in the language system itself.

Language has encoded perceptual information in such a way that it provides a convenient cognitive shortcut for language processing. We don’t have to keep everything in mind. We can offload cognitive effort to the organization of language. That would suggest that language creates meaning.