ICSI to Collaborate with Microsoft on Conversational Human-Machine Interaction

Elizabeth Shriberg

Andreas Stolcke

^{July 24, 2012}
Researchers at ICSI will work with Microsoft Corp. to advance the state of the art in human-computer interaction relying on speech and other modalities. The collaboration takes advantage of ICSI’s history of excellence in speech processing research and Microsoft’s wealth of data, technology, and experience in deploying natural speech interfaces in its services and applications.

"ICSI looks forward to working with Microsoft to improve how computers and devices interact with humans in a natural way," said Roberto Pieraccini, director of ICSI. "This work is particularly important now, as the popularity of devices that understand and produce speech grows more quickly than ever before."

Elizabeth Shriberg and Andreas Stolcke, principal scientists with the Conversational Systems Laboratory at Microsoft and ICSI external fellows, will lead the effort. The Conversational Systems Lab (CSL) is an applied research group within Microsoft’s Online Services Division based at the Microsoft Silicon Valley campus in Sunnyvale, California. CSL is exploring novel ways to interact naturally with computer systems and services using speech, natural language text, and gesture. Its aim is to enable conversational understanding of users’ inputs and intentions across a range of devices, from mobile phones to Xbox consoles in the living room. CSL conducts research spanning a range of scientific disciplines from acoustic to semantic and affective language processing.

In one of the first projects under this collaboration, researchers will use information conveyed by the melody and rhythm of speech, known as prosody, to improve automatic speech understanding. "Patterns of timing and intonation in spoken language encode information far beyond that conveyed by words alone. This information is important for achieving natural and efficient conversational interactions with machines," said Shriberg. "We expect to accelerate progress on human-computer dialog systems that better understand and use cues in human-human spoken communication that we often take for granted."

"We are excited about this collaboration with ICSI, which has been a venue for cutting-edge research in speech processing for more than 20 years," said Stolcke. "This project combines the expertise and creativity at ICSI with Microsoft’s experience deploying increasingly more powerful and natural speech interfaces to its services and applications."

The research will be conducted by senior ICSI and Microsoft researchers, as well as postdoctoral researchers and students at ICSI. The collaboration has the potential to extend beyond the prosodic aspects of speech to other research topics in human-computer interaction.