Featured Alum: Jeff Bilmes

Professor Jeff Bilmes, of the University of Washington, is our featured alumnus for this issue.  Bilmes was one of ICSI's very first employees, joining our ranks while a computer science undergraduate at U.C. Berkeley in the late 80's.  Bilmes started out working as a programmer on spatial data–base access methods for Oliver Günther, but as staff members began to realize his unusually broad set of talents and interests, he was offered a research position in the Realization Group (now the Speech Group).  He worked with them for a year before heading to MIT for his Master's degree and then returning to ICSI and UCB to complete his Ph.D. in Computer Science.

While at ICSI, Bilmes worked in a wide range of disciplines.  His work with Prof. Nelson Morgan introduced entirely new methods to existing speech recognition algorithms.  His early pioneering work on optimized numerical library auto–tuning, along with Krste Asanovic (another ICSI–to–MIT and back–to–ICSI researcher), generated the PHiPAC project, a method that could automatically generate code that would specifically tailor numerical programs to run at near–peak performance. This work spawned the entire field of auto–tuning in high–performance computing.

In 1999, Bilmes accepted a faculty position at the University of Washington, Seattle, where he received an NSF Career award, a CRA digital government fellowship, and a NAE Gilbreth Lectureship award. He has continued in his tradition of making breakthroughs in a number of different fields.  At UW, he developed the Graphical Models Toolkit (GMTK), a software system that researchers can use to express and compute with an unlimited number of statistical models over sequential data, such as speech, language, and biological signals. GMTK allows computation with more sophisticated structures than simple Hidden Markov Models. GMTK is currently the most widely used graphical–model toolkit for sequential data, and is in heavy use by laboratories all over the world. Bilmes has published in a diverse set of areas, including speech recognition, natural language processing, human–computer interaction, bio–informatics, social networks, computer vision, active learning, semi–supervised learning, and submodularity in machine learning.

One particular project of Bilmes's that has enjoyed considerable media attention is the Vocal Joystick.  Named one of the 25 leading–edge IT research projects in 2008 by Network World, the Vocal Joystick's aim is to enable individuals with motor impairments to control on–screen devices (like mouse pointers, scroll bars, and general computer interfaces) and electro–mechanical devices as fluently as any non–impaired user might do.  With the Vocal Joystick system, a user uses simple and easy–to–learn continuous non–verbal vocalizations to control a mouse cursor (or a robotic arm). For example, instead of saying things like "go, up, left, stop", the Vocal Joystick uses these continuous vocalizations to cause a sweep from "up" to "left" at a manner and rate as fast or slow as the user desires.  Applications of the Vocal Joystick range from basic mouse control, to sophisticated cad–design and/or photo touch–up applications, to art, and even to video games (as entertainment too should be made accessible to the motor impaired). Studies have shown that the Vocal Joystick greatly outperforms other voice–control methods in terms of output, experience, efficiency, and user satisfaction.

Bilmes is currently on a year–long sabbatical at the Max Planck Institute for Biological Cybernetics in Tübingen, Germany.  He's in Prof. Dr. Bernard Schölkopf's group working on various writing and other projects related to machine learning, robotics, and sequential graphical models.

Of his time at ICSI, Director Morgan says Bilmes "was in permanent overdrive.  Science, engineering programming, music, you name it; he was always at what the baseball guys would call ‘110%'."  Prof. Bilmes is a great example of the kind of researcher that ICSI tries to foster.  He is still collaborating internationally, and is just as comfortable crossing disciplines as walking down the hall.