Jul 18, 2019 - 4:00 pm Venue:  MIT 46-5165 Address:  43 Vassar Street, Bldg 46-5165, Cambridge MA 02139 Speaker/s:  Gemma Roig

Title:

Task-specific Vision DNN Models and Their Relation for Explaining Different Areas of the Visual Cortex
 

Abstract:

Deep Neural Networks (DNNs) are state-of-the-art models for many vision tasks. We propose an approach to assess the relationship between visual tasks and their task-specific models. Our method uses Representation Similarity Analysis (RSA), which is commonly used to find a correlation between neuronal responses from brain data and models. With RSA we obtain a similarity score among tasks by computing correlations between models trained on different tasks. We demonstrate the effectiveness and efficiency of our method to generating task taxonomy on the Taskonomy dataset. Also, results on PASCAL VOC suggest that initializing the models trained on tasks with higher similarity score show higher transfer learning performance. With the same approach, we propose a novel algorithmic method to select the branching location from a given pre-trained model for a new task. With RSA we compare the representations at different layers of the pre-trained encoder with the representations of a set of  DNNs trained on different vision tasks. We use our method to design a multi-task model composed of one shared core network, and several task-specific functional blocks. We also incorporate task-specific batch normalization and parametric ReLU parameters, which allow to incorporate modulations into the core network according to the task. Finally, we explore the power of DNNs trained on 2D, 3D, and semantic visual tasks as a tool to gain insights into functions of visual brain areas (early visual cortex (EVC), OPA and PPA). We find that EVC representation is more similar to early layers of all DNNs and deeper layers of 2D-task DNNs. OPA representation is more similar to deeper layers of 3D DNNs, whereas PPA representation to deeper layers of semantic DNNs. We extend our study to performing searchlight analysis using such task specific DNN representations to generate task-specificity maps of the visual cortex. Our findings suggest that DNNs trained on a diverse set of visual tasks can be used to gain insights into functions of the visual cortex. This method has the potential to be applied beyond visual brain areas.

Organizer:  Hector Penagos Frederico Azevedo Organizer Email:  cbmm-contact@mit.edu