Shimon Ullman: Visual Understanding: State of the World, Future Directions

Topics: Overview of visual understanding; object categorization and variability in appearance within categories; recognizing individuals; identifying object parts; learning categories from examples by combining different features (simple to complex) and classifiers; visual classes as similar configurations of image components; finding optimal features that maximize mutual information for class vs. non-class distinction (Ullman et al., Nature Neuroscience 2002); SIFT and HoG features; Hmax model; state-of-the-art systems from the Pascal challenge vs. human performance; deep learning and convolutional neural nets (e.g. ImageNet); unsupervised learning methods; fMRI and EEG studies indicating high correlation between informativeness of image patches and activity in higher-level visual “object” areas (e.g. LOC); recognition of object parts with hierarchies of sub-fragments at multiple scales (Ullman et al., PNAS 2008); object segmentation (e.g. Malik et al.; Brandt, Sharon, Basri, Nature 2006) using top-down semantic information to enhance segmentation; future challenges include recognizing what people are doing, interactions between agents, task-dependent image analysis e.g. answering queries, visual routines, and using vision to learning conceptual knowledge in a new domain