Nov 12, 2019 - 4:00 pm Venue:  MIT 46-5165 Address:  43 Vassar St, MIT bldg 46, Cambridge MA 02139 Speaker/s:  Katharina Dobs(Kanwisher Lab)

Using task-optimized neural networks to understand why brains have specialized processing for faces

Previous research has identified multiple functionally specialized regions of the human visual cortex and has started to characterize the precise function of these regions. But why do brains have functional specialization in the first place, and why do we have the particular specializations we do (e.g., for faces or scenes, but apparently not for food or cars)? Here, we address these questions using the well-studied case of face selectivity. Specifically, we used deep convolutional neural networks (CNNs) to test whether face-specific regions are segregated from object cortex in the primate visual system because the optimal feature spaces for face and object perception differ from each other.
    We trained two separate CNNs with the AlexNet architecture to categorize either faces or objects. The face-trained CNN performed worse on object categorization than the object-trained CNN and vice versa, demonstrating that the learned features differ for the two tasks. To determine whether a CNN could learn a common feature space, we trained CNNs on both tasks with a branched architecture, varying the number of layers that were shared across tasks (Kell et al., 2018). The fully-shared network performed worse than the separate CNNs suggesting a cost for sharing both tasks in one system. However, like in the primate visual system, early layers could be shared without impairing performance.

    Do these results generalize to architectures with larger capacity? We trained three networks with a VGG16 architecture: one on faces, one on objects, and one on both. In contrast to the AlexNet results, the dual-task CNN performed as well as the separate networks. To test whether this network discovered covert task segregation, we performed lesion experiments and found that lesioning face-specific features selectively impaired face performance. This result suggests that functional specialization for faces emerges spontaneously in networks optimized for face and object tasks.

Organizer:  Hector Penagos Frederico Azevedo Organizer Email:  cbmm-contact@mit.edu

Oct 1, 2019 - 4:00 pm Venue:  MIT 46-5165 Address:  MIT Building 46, 43 Vassar Street, Cambridge MA 02139 Speaker/s:  Andrzej Banburski, Poggio Lab Title: Biologically-inspired defenses against adversarial attacks   Abstract: Adversarial examples are a broad weakness of neural networks and represent a crucial problem that needs to be solved to ensure tampering-resistant deployments of neural network-based AI systems. They also offer a unique opportunity to advance research at the intersection of the science and engineering of intelligence. We propose a novel approach, based on the hypothesis that the primate visual system has a similar architecture to deep networks but seems to be immune to today's adversarial attacks. What are then aspects of visual cortex that are not captured by present models? We focus on the eccentricity dependent sampling array of the retina and on the existence of a set of spatial frequencies channels at each eccentricity. Our proposal will test whether systems based on these properties may be robust against adversarial examples. Organizer:  Frederico Azevedo Hector Penagos Organizer Email:  cbmm-contact@mit.edu

Jul 18, 2019 - 4:00 pm Venue:  MIT 46-5165 Address:  43 Vassar Street, Bldg 46-5165, Cambridge MA 02139 Speaker/s:  Gemma Roig

Title:

Task-specific Vision DNN Models and Their Relation for Explaining Different Areas of the Visual Cortex
 

Abstract:

Deep Neural Networks (DNNs) are state-of-the-art models for many vision tasks. We propose an approach to assess the relationship between visual tasks and their task-specific models. Our method uses Representation Similarity Analysis (RSA), which is commonly used to find a correlation between neuronal responses from brain data and models. With RSA we obtain a similarity score among tasks by computing correlations between models trained on different tasks. We demonstrate the effectiveness and efficiency of our method to generating task taxonomy on the Taskonomy dataset. Also, results on PASCAL VOC suggest that initializing the models trained on tasks with higher similarity score show higher transfer learning performance. With the same approach, we propose a novel algorithmic method to select the branching location from a given pre-trained model for a new task. With RSA we compare the representations at different layers of the pre-trained encoder with the representations of a set of  DNNs trained on different vision tasks. We use our method to design a multi-task model composed of one shared core network, and several task-specific functional blocks. We also incorporate task-specific batch normalization and parametric ReLU parameters, which allow to incorporate modulations into the core network according to the task. Finally, we explore the power of DNNs trained on 2D, 3D, and semantic visual tasks as a tool to gain insights into functions of visual brain areas (early visual cortex (EVC), OPA and PPA). We find that EVC representation is more similar to early layers of all DNNs and deeper layers of 2D-task DNNs. OPA representation is more similar to deeper layers of 3D DNNs, whereas PPA representation to deeper layers of semantic DNNs. We extend our study to performing searchlight analysis using such task specific DNN representations to generate task-specificity maps of the visual cortex. Our findings suggest that DNNs trained on a diverse set of visual tasks can be used to gain insights into functions of the visual cortex. This method has the potential to be applied beyond visual brain areas.

Organizer:  Hector Penagos Frederico Azevedo Organizer Email:  cbmm-contact@mit.edu

May 14, 2019 - 4:00 pm Venue:  Harvard NW Building, Room 243 Address:  52 Oxford St, Cambridge, MA 02138 Speaker/s:  Duncan Stothers, Will Xiao, Nimrod Shaham

Duncan Stothers-

Title: Turing's Child Machine: A Deep Learning Model of Neural Development

Abstract:

Turing recognized development’s connection to intelligence when he proposed engineering a ‘child machine’ that becomes intelligent through a developmental process, instead of top-down hand-designing intelligence into an ‘adult machine’. We now know from neurobiology that the most important developmental process is the ‘critical period’ where the architecture (equivalently connectome or topology) expands in a random way and then prunes itself down based on activity. The computational role of this process is unknown, but we know it is connected to intelligence because deprivation during this period has permanent negative effects later in life. Further, the fact the connectome changes during this period through ‘architecture learning’ in addition to the synaptic weights changing through ‘synaptic weight learning’, set it apart from deep learning AI research where the architecture is hand designed and stays fixed during learning and only ‘synaptic weight learning’ takes place. To understand development’s connection to biological and artificial intelligence we model the critical period by adding random expansion and activity based pruning steps to deep neural network training. Results suggest the critical period is as an unsupervised architecture search process that finds exponentially small architectures that generalize well. Resultant architectures from this process also show similarities to hand-designed ones. 

 

Will Xiao-

Title: Uncovering preferred stimuli of visual neurons using generative neural networks

Abstract:

What information do neurons represent? This is a central question in neuroscience. Ever since Hubel and Wiesel discovered that neurons in primary visual cortex (V1) respond preferentially to bars of certain orientations, investigators have searched for preferred stimuli to reveal information encoded by neurons, leading to the discovery of cortical neurons that respond to specific motion directions (Hubel, 1959), color (Michael, 1978), binocular disparity (Barlow et al., 1967), curvature (Pasupathy & Connor, 1999), complex shapes such as hands or faces (Desimone et al., 1984; Gross et al., 1972), and even variations across faces (Chang & Tsao, 2017).

However, the classic approach for defining preferred stimuli depends on using a set of hand-picked stimuli, limiting possible answers to stimulus properties chosen by the investigator. Instead, we wanted to develop a method that is as general and free of investigator bias as possible. To that end, we used a generative deep neural network (Dosovitskiy & Brox, 2016) as a vast and diverse hypothesis space. A genetic algorithm guided by neuronal preferences searched this space for stimuli.

We evolved images to maximize firing rates of neurons in macaque inferior temporal cortex and V1. Evolved images often evoked higher firing rates than the best of thousands of natural images. Furthermore, evolved images revealed neuronal selective properties that were sometimes consistent with existing theories but sometimes also unexpected.

This generative evolutionary approach complements classical methods for defining neuronal selectivities, serving as an independent test and a hypothesis-generating tool. Moreover, the approach has the potential for uncovering internal representations in any modality that can be captured by generative neural networks.

 

Nimrod Shaham-

Title: Continual learning and replay in a sparse forgetful Hopfield model

Abstract:

The brain has a remarkable ability to deal with an endless, continuous stream of information, while storing new memories and learning to perform new tasks. This is done without losing previously learned knowledge, which can be stored to timescales of the order of the animal’s life. In contrast, current artificial neural network models suffer from limited capacity (associative memory network models) and acute loss of performance in previously learned tasks after learning new ones (deep neural networks). Overcoming this limitation, known as catastrophic interference, is one of the main challenges in machine learning and theoretical neuroscience.

Here, we study a recurrent neural network that continually learns and stores sparse patterns of activity, while forgetting old ones (a palimpsestic model). Time dependent forgetting is incorporated as a decay of old memories’ contributions to the weight matrix. We calculate the required forgetting rate in order to avoid catastrophic interference, and find the optimal decay rate that gives maximal number of retrievable memories. Then, we introduce replay to the system, in the form of reappearance of previously stored patterns, and calculate the enhancement of time for which a memory is retrievable due to different patterns of replays. Our model reveals in a tractable and illuminating way how a recurrent neural network can learn continuously and store selected information for lifelong timescales.     

Organizer:  Daniel Zysman Hector Penagos Frederico Azevedo Organizer Email:  cbmm-contact@mit.edu