MetaNet: A Multilingual Metaphor Repository

A large team of researchers from ICSI, UC San Diego, University of Southern California, Stanford, and UC Merced is building a computer system capable of understanding metaphors used in English, Persian, Russian, and Spanish. The work will span the fields of computer science, linguistics, psychology, and cognitive science. MetaNet, funded by Intelligence Advanced Research Projects Activity (IARPA) under its Metaphor Program, is led by Professor Srini Narayanan, ICSI’s AI Group leader.

Metaphors describe one thing in terms of another; they’re mappings from a source term to a target term. The target, which is abstract, is understood in terms of the source, which is concrete. Shakespeare’s “bank and shoal of time,” for example, maps abstract concepts of time and life to more concrete images having to do with bodies of water.

But metaphors are not just literary devices; they pervade language and affect, not just the way we speak about the world, but also the way we think about the world. “Many of our abstract concepts are learned through bodily experience and are understood by projecting to bodily experience,” said Narayanan, the principal investigator of the project. Conceptual metaphors – such as the idea that time is money or life is a journey – draw on these bodily experiences to express complex, abstract ideas.

Conceptual metaphors, the researchers say, structure how we see the world. UC Berkeley Professor George Lakoff pointed to the example Love Is A Journey, a complex metaphor built on more basic metaphors that underlie English speakers’ cognitive processes, such as Action Is Motion and Purposes Are Destinations. These metaphors give rise to a conceptualization of love as a journey and romantic problems as obstacles along that journey. This ultimately leads to linguistic metaphors – phrases and expressions that rely on and evoke conceptual metaphors, such as “Their marriage is on the rocks.”

Theoretical Foundations

The AI Group has a long and productive history of combining computational modeling with cognitive linguistic theories of metaphor. Over the last two decades, researchers from the Neural Theory of Language (NTL) project, including Professor Jerry Feldman, Lakoff, Narayanan, and several students, have combined biologically based computational models with cognitive linguistic analyses and experimental techniques (both behavioral and imaging) to demonstrate the ubiquity of metaphor, its connection to fundamental cognitive processes, and its use in everyday reasoning and in specialized discourse.

Narayanan, who in addition to leading the AI Group is a cognitive scientist on the UC Berkeley faculty, developed the computational theory and an early interpretation system that demonstrated the utility of conceptual metaphor for reasoning about abstract concepts and domains. The neurally motivated model and results have led to new insights on the nature of inference, called the “simulation semantics” hypothesis, which asserts that understanding is imaginative simulation. Simulation semantics makes detailed predictions on the manner, content, and timing of metaphoric inference. These predictions continue to be confirmed by experimental results in laboratories around the world. Narayanan’s more recent work demonstrates through computer simulation how metaphors emerge as a natural consequence of the functional architecture and known information processing constraints of the human brain.

“The knowledge model and representation techniques in the existing system will significantly inform the design and implementation of the metaphor repository and its use in applications of interest to IARPA in the program,” Narayanan said. “This work enables us to hit the ground running.”

The team’s linguistic analysis lead, George Lakoff, is a linguist, cognitive scientist, and co-founder of the NTL project at ICSI and UC Berkeley. Over the last three decades, Lakoff and his students have pioneered much of the modern research on metaphor. This research, reported in more than a dozen books, shows that metaphoric linguistic expressions are surface manifestations of metaphorical thought processes, and that a large class of conceptual metaphors arise from bodily experiences — which began the research tradition of what is now known as “embodied cognition.” Lakoff and Feldman founded the NTL project, an integral part of the AI Group since 1988.

In recent analyses of metaphors used in political discourse, Lakoff has uncovered basic parenting frames and foundational metaphor systems that underlie deeply held beliefs and worldviews. These metaphor systems are of core interest and will inform the linguistic research in the project.

A Multi-Lingual Repository

The goal of IARPA’s Metaphor Program is to build a system that extracts linguistic manifestations of metaphor (words and phrases that are based on on metaphor) from text and interprets them automatically in four different languages. Researchers in ICSI’s MetaNet project will do this by building a multi-lingual metaphor repository that represents the network of conceptual metaphors and includes links to linguistic realizations. Users will be able to browse, navigate, annotate, and modify the repository, which will also provide programmatic access for metaphor extraction, analysis, and inference.

The work is being done in American English, Iranian Persian, Russian as spoken in Russia, and Mexican Spanish. While different languages use different metaphors, the use of metaphor “appears to be cross-linguistic,” Narayanan said. “You and I, from different backgrounds, share this.”

For example, speakers from all cultures studied for use of metaphor discuss abstract ideas, such as political action, in terms of spatial motion, such as physical movement. The metaphor underlying this phenomenon – the event structure metaphor – is so basic and intuitive that it shapes the way we think about abstract actions without our knowing. UC Merced Professor Teenie Matlock, an experimental cognitive linguist involved in the validation portion of the project, says the way we speak about time is an example of this. English speakers think of time in spatial terms: April is after March. This metaphor influences not just the words and phrases we use to discuss time, but also the way we conceive of time. Matlock said, “We often go to the physical sense of time to help us think about it.”

IARPA, a research arm of the Office of the Director of National Intelligence (ODNI), hopes to use the results of the Metaphor Program to understand the role metaphor plays in how people from different cultural backgrounds make judgments and decisions. The system developed during the MetaNet project will help by automatically analyzing metaphor in the four languages under investigation. The project, said Narayanan, will “try to understand how different cultures have different beliefs and worldviews as a result of these metaphors.”

Automatic text analyzers have difficulty dealing with metaphor. One reason is that the metaphorical mapping from a concrete source to an abstract target is only partial. For example, while information, an abstract target, can be discussed in terms of a liquid – it may be contained or leaked – the metaphor does not extend to every aspect of a liquid.

Another challenge is that complex metaphors arise from more basic metaphors. As Lakoff pointed out, the metaphors Action Is Motion and Goals Are Destinations, when combined with others, eventually lead to the complex idea that Love Is A Journey. A computer system must understand the relationships among these metaphors before it can understand that a marriage “on the rocks” is a troubled one.

IARPA is providing the team with a series of target concepts to be analyzed by the system in the first years of the project; later, the system will be asked to respond to test cases and scenarios determined by IARPA.

Frame Semantics

The analysis of target concepts and cases will rely in part on frame semantics work done by ICSI’s FrameNet Project. FrameNet, led by Collin Baker and Professor Charles Fillmore, is building a lexical database, usable by both machines and humans, that shows how words are used in texts. In a sentence annotated by FrameNet, a word is understood through the semantic frame it evokes and the roles it plays in that frame. A semantic frame is a description of a type of event, relation, or entity, and comprises frame elements that may include, for example, the person carrying out an action and the object on which it is carried out.

FrameNet annotations go beyond grammatical parsing. The agent of a frame – the one carrying out an action – may not necessarily be the subject of a sentence that evokes the frame. This is common in commands (“Leave that alone”), in which the agent you is implied, but is also true, for example, in the sentence “Graham’s back arched,” where the agent is Graham even though the grammatical subject is Graham’s back.

This method of annotation will be used to describe the relationship between a linguistic metaphor’s literal meaning and its metaphorical meaning. For a word that evokes a metaphor, cognitive linguists will determine both the target and the source. In the sentence “That flat tire cost me an hour,” the annotation of the word cost will show its abstract target (time) and its concrete source (money). This will provide the repository with information about how metaphorical frames and frame elements work together, forming a network that shows the relationships between metaphors and their literal meanings. The repository will also show the relationships among different metaphors.

These mappings will be used as base information by a system that extracts metaphors using supervised machine learning. The system will use what it has learned about the relationships between the frame elements of conceptual metaphors to find new metaphors in text.

Automatic Extraction

The team is also working toward unsupervised machine learning of metaphor mappings, which uses less initial information than supervised learning and so will be particularly helpful for extracting metaphors in languages, such as Persian, with fewer resources. While a graduate student at the University of Cambridge, Ekaterina Shutova, now an ICSI postdoctoral fellow, developed a system able to extract metaphors using a small set of annotated metaphors. Shutova clustered words associated with these metaphors, and the system used these word clusters to find new metaphors that used the same words. Shutova points to the metaphor Relationship Is A Mechanism as an example. If her system were seeded with this metaphor and a cluster that grouped the words relationship and function, it would be able to identify the metaphor Democracy Is A Mechanism, since democracy can be seen in similar linguistic contexts in the data. While the system operates with high precision, it is dependent on its seed metaphors, which currently results in low recall. Shutova’s next goal is to expand on this work and build a more robust, potentially fully unsupervised system. With these extraction methods and the representation techniques, the researchers will fill a repository of metaphors and their frames, mappings, and elements.

Evaluating the System

In conjunction with this, a group of cognitive linguists and neuroscientists will test how metaphor affects thinking and emotion in order to evaluate the effectiveness of the metaphor repository and methodologies developed by the rest of the team. The validation team is led by Lera Boroditsky of Stanford and includes Lisa Aziz-Zadeh of the University of Southern California, Ben Bergen of UC San Diego, and Teenie Matlock of UC Merced

Matlock, founder of the Cognitive and Information Science program at UC Merced, says this part of the project will “test the underlying psychological reality of metaphor.”

“We want to know what are the conceptual underpinnings of metaphor in actual conversation,” she said.

Although the validation work is in the early stages of planning, the team hopes to use, among other techniques, eye-tracking tests, in which participants look at a blank screen and listen to an audio track while a device records the movement of their eyes. Such tests reveal changes in participants’ internal state. There will be similar analyses of participants’ gestures and posture while they engage in conversation.

Matlock specializes in conducting tests using natural discourse, in which participants watch a video or read a paragraph and either answer questions about it or discuss it in pairs. Researchers analyze the transcribed recording of the discussions, tracking what words are used, in what order they are used, and how frequently they are used.

The team will also evaluate how effectively the system extracts metaphors by asking study participants to rate the metaphors on a variety of measures, including how useful and familiar they are.

The team is also interested in “seeing whether people have implicit emotional reactions to the various metaphors,” said Bergen, a behavioral researcher at UC San Diego. He will be running implicit association tests (IATs) on the metaphors extracted by the system. IATs measure unconscious reactions by measuring how long it takes participants to categorize terms along a set of axes, one of which has a value judgment associated with it. IATs are often used to measure racial bias: participants are asked to categorize faces that appear on a screen as one of two races, which are paired with value judgments (such as “good” and “bad”). Participants who quickly categorize minority faces as the correct race both when the value judgment is positive and when it is negative are said to have low racial bias. Bergen will adopt this experimental design, substituting metaphors for faces.

The team will also look at biophysical responses to the use of metaphor. Bergen will use electromyographic recordings to measure subtle facial expressions that may be so slight that they are not visible. Aziz-Zadeh of the University of Southern California will track what parts of the brain are activated by metaphor using functional Magnetic Resonance Imaging.

The results of these tests will give researchers an idea of how metaphors affect speakers’ brain activity and emotion and how they influence decision-making and judgment. The results will also inform the work of the rest of the team as they develop the metaphor extraction system and methodology.

Baker, the FrameNet and MetaNet project manager, says that the combination of disciplines makes the project unique. “Our approach is particularly challenging because it requires bridging vastly different fields, from cognitive linguistic analysis to machine learning techniques to brain scans,” he said. “But if we can do it, it will be particularly rewarding.”

The Implications of Metaphor

Previous brain imaging studies have shown that talking about metaphorically grasping an idea uses the same parts of the brain as physically grasping an object. This strong connection between metaphor and bodily experience, Narayanan said, has implications for political and social discourse.

Narayanan pointed out that images of strangulation are often used in political discourse (“our economy is being strangled by socialist policies”). Narayanan said, “You’re using a word that has a very deep connection to our experience as humans to talk about government.”

“Language understanding is imaginative simulation,” he said.

Boroditsky, the Stanford professor leading the experimental validation portion of the project, and her student Paul Thibodeau found in previous studies that changing which metaphors are used to discuss a crime wave affects how people think it should be solved. When crime was described as a beast “lurking in neighborhoods,” participants in a study were more likely to suggest increased enforcement – more police officers and harsher punishments – as a solution. When the same crime wave was described as an illness “plaguing” neighborhoods, participants were more likely to suggest social reform.

Narayanan said this raises several questions, such as, “Why these mappings and not others? Are there different sets of mappings across cultures? Can we intervene and suggest different mappings to change the way people think about crime?”

Matlock has done similar work showing that the way something is framed affects how people think about it. In one test, some participants read a paragraph about a political candidate who “was having an affair”; others read about a candidate who “had an affair.” The former believed that the candidate was less likely to be elected than the latter. Such findings, said Matlock, suggest that our thinking is influenced, not just by the use of words and metaphors, but by the form of the linguistic construction as well.

The MetaNet project is supported by the Intelligence Advanced Research Projects Activity (IARPA) via Department of Defense US Army Research Laboratory contract number W911NF-12-C-0022. Disclaimer: The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of IARPA, DoD/ARL, or the U.S. Government.

All research involving human subjects, to include use of biological specimens and human data, selected for funding must comply with the federal regulations for human subject protection, namely 45CFR Part 46, Protection of Human Subjects (www.hhs.gov/ohrp/humansubjects/guidence/45cfr46.htm) and 32CFR Part 219, Protection of Human Subjects (www.dtic.mil/biosys/downloads/32cfr219.pdf). All human subjects research proposed for the MetaNet project has been subjected to IRB review and has received IRB approvals from UCSD, USC, UC Merced, and Stanford University.