Knowledge-Aided Interface for Big Data Streams

In this collaborative project with Mod9 Technologies, researchers from ICSI's Audio and Multimedia group and ICSI's FrameNet project seek to demonstrate real­time monitoring of broadcast news streams to support a tactical operations center (TOC). A primary focus of this effort is to exploit multimedia – audio­visual data containing speech, images, and metadata such as geo-location and personal identification – and integrate it into an intuitive and informative visualization for a TOC’s use. The researchers will use specialized algorithms and infrastructure that go beyond traditional text-based processing of data, which is demanded by the complexity and scale of multimedia. They will demonstrate how automatic speech recognition and machine translation can better monitor simultaneous foreign-language sources than human analysts and show how the FrameNet database may be used to improve searches for related stories effective through query expansion. They will also show how multimedia processing techniques can filter social media content as it is posted to determine the locations and identities of key participants.