The ICSI Vision Group investigates core computer vision research problems as well as integrative research problems from multiple modalities and research areas including human vision, computer vision, and robotic vision. We are interested in segmentation, object matching, visual attention, the perception of human expression for multi-modal interaction, the recognition of object and scene categories, and applications ranging from mobile phone interfaces to interactive robotics.
We study learning mid-level representation from natural non-curated data to achieve efficient and generalizing performance on downstream visual tasks such as recognition, segmentation, and detection. We exploit instance discrimination, instance grouping, model bias and variance analysis, pixel-to-segment contrastive learning, and visual memory to handle open-set recognition, long-tail distribution, open compound domain adaptation, unsupervised or weakly supervised recognition and segmentation.
Complex-valued data is ubiquitous in physics and signal processing applications, and complex-valued representations in deep learning have appealing theoretical properties. While these aspects have long been recognized, complex-valued deep learning lags far behind its real-valued counterpart. Existing methods ignore the rich geometry of complex-valued data, instead opting to use the same techniques and architectures as real-valued data, with undesirable consequences such as decreased robustness, larger model sizes, and poor generalization.
Sound carries complementary information to vision and can help scene understanding and navigation. We train a model to tell individual sounds apart without using labels, which can be used to accelerate subsequent training on supervised sound event classification, and to explain how song birds such as zebra finch can develop communication without any external supervision. We also demonstrate with a low-cost real system that learns echolocation and generates depth images only from sound.