Evidence that recurrent circuits are critical to the ventral stream’s execution of core object recognition behavior
Date Posted:
April 29, 2019
Date Recorded:
April 25, 2019
CBMM Speaker(s):
James DiCarlo ,
Kohitij Kar Speaker(s):
Diane Beck
All Captioned Videos Publication Releases
Description:
Center for Brains, Minds and Machines researchers and paper authors Jim DiCarlo and Kohitij Kar discuss their recent research findings as published in the journal Nature Neuroscience . Diane Beck of the University of Illinois, Beckman Institute provides an external perspective of the research results.
[MUSIC PLAYING] JAMES DICARLO: My lab works on trying to understand how each of us can understand our visual world, how we can identify objects and faces in the world. We know there are brain areas that do this called the ventral visual stream. And we study those brain areas measuring the neural properties within those brain areas and trying to build computational models that can emulate not only what the neurons are doing but our ultimate behavior and our perception of the world.
KOHITIJ KAR: In the DiCarlo lab, I work at the interface of experimental and compositional neuroscience. We here work in a closed loop where experiments are done on primates, humans, and monkeys, specifically to falsify current models and also to make measurements that provide constrains for building newer models. Once we provide this constrains, newer models are being built and then fed back to the experimental rig so that we can test these models. And thus we establish the loop and the loop goes on to build better and better models of human vision.
JAMES DICARLO: For a long period of time, we've modeled the ventral visual stream as a largely feed forward process. Even though we known that there are recurrent anatomical connections within the ventral stream, we can never really isolate those connections to study them independently before. But now since we started to have models that could approximate the feed forward processing quite well, that gave us a new opening to begin to understand perhaps what the recurrence is doing. So in particular, comparing the feed forward models to what the brain is doing, has given us a new opening to study that recurrence. And so we try to take advantage of that right now.
KOHITIJ KAR: We observe that there are specific challenging images for the models that are not challenging for humans and monkeys. So when we measured the solution of these images in the primate brain, we figured out that these specific challenge images were solved around 30 minutes seconds later in the primate brain.
DIANE BECK: What this paper has shown is that these feed back connections are very likely playing a role in object recognition. And the way they've demonstrated this is to show that those very images that these deep convolutional neural nets find difficult are the same images that take longer to unfold in the brain. And that longer latency is likely to be the feedback for recurrent activity.
KOHITIJ KAR: Surprisingly, the current very deep neural network models also have solutions for these challenged images. However, we think what the computer vision community has achieved by stacking more layers onto these computational neural networks, evolution has achieved it in a much smarter way by making the brain a recorded neural network. Record neural networks also come with other advantages like it is more energy efficient, and it can be more generalizable. So when we build these record neural network models of the brain, these advantages can be tapped into and further explored.
DIANE BECK: What I think is really nice about this paper is that it is a very good demonstration of the synergy between computer vision and biological vision. And so in fact, this paper I think is informing both fields. So it's telling us that yes, primate vision, biological vision, and in particular, object recognition needs these feedback recurrent connections. But it's also telling us something about the convelutional neuro net. And that is one way in which you can improve computer vision is perhaps to include recurrents.
JAMES DICARLO: So what's most exciting to me about this study is not just saying that recurrence is important but that we are comparing models on an image-by-image basis with what's going on in the primate brain. So that not only motivates the idea of recurrence, but it provides constraints on those future models to say if a model is correct, it should be able to predict the patterns that we've uncovered again on an image-by-image basis. So even beyond the statement of recurrence, the data that are provided as part of the study will for a long time act as guidance on future models of the ventral visual stream.
[MUSIC PLAYING]
Associated Research Module:
have an interactive transcript feature enabled, which appears below the video when playing. Viewers can search for keywords in the video or click on any word in the transcript to jump to that point in the video. When searching, a dark bar with white vertical lines appears below the video frame. Each white line is an occurrence of the searched term and can be clicked on to jump to that spot in the video.
