Neural Population Control via Deep Image Synthesis
Date Posted:
May 2, 2019
Date Recorded:
April 25, 2019
CBMM Speaker(s):
James DiCarlo ,
Pouya Bashivan ,
Kohitij Kar All Captioned Videos Publication Releases
Description:
Center for Brains, Minds and Machines researchers and paper authors Jim DiCarlo, Pouya Bashivan and Kohitij Kar discuss their recent research findings as published in the journal Science .
[MUSIC PLAYING] JAMES DICARLO: Our research aims to understand how we perceive and understand the visual world. And to do that, we think we need to build computational models. So that means our overall goal of our research is to build models of the brain processing that intervenes between the image striking your eye and your perception of what's out there in the world. So success would be very accurate models of the ventral visual stream.
One way to judge whether you have an accurate model is to actually try to use the model to do something. And we figured using a model to try to control the neurons' activity within the brain was a very strong test of our current understanding of the ventral visual stream as embodied by those models.
POUYA BASHIVAN: My background is in control engineering. And the aim of this discipline of engineering is to design systems, called the controllers, to enable us to control the output of another system, usually referred to as the process. So what we did in this study was to bring in that viewpoint into neuroscience, and to look at the brain as a process, and see if we can design a controller now that could help us to control the output of this process that would be the neural activities.
KOHITIJ KAR: Experimentally, we first chronically implanted multi-electrode areas into the monkey's brain. In this case, we implanted them in area V4. One of the main advantages of having this procedure is that you can record from the same neurons over the course of a month, sometimes even a year. The objective of this particular project was to really put the current computational models of vision into a very strong test.
POUYA BASHIVAN: So what we did here was to use that model and try to control the activity of neurons. And we did that in two settings. In the first set of experiments, we set out to drive the activity of individual neurons as high as we could drive them. And we called this case "stretch." We were basically trying to stretch the neural responses beyond the nominal range of [? fine ?] rates.
KOHITIJ KAR: So once the images were synthesized, we brought the monkey back to the lab. And we played these images back to the monkeys, presumably to the same neurons that we were recording.
POUYA BASHIVAN: And when we did that, we found out that, for most of these cases, for most of the neurons, we were successfully driving them outside their nominal range, beyond the levels that could be reached by showing natural images to these animals.
KOHITIJ KAR: The second objective is that we wanted to see if we can drive a population of neuron to a certain state. And in this case, we tried to push one neuron to a very high state while keeping the other ones at a minimally active state.
POUYA BASHIVAN: What we found out was that, in most cases, it could successfully drive these population of neurons now to a one-hot state. That is, basically, target neuron being maximally activated, the other neurons not being activated-- much better than what we could have achieved by showing natural images.
JAMES DICARLO: One of the things that I find quite exciting about this science study is that, for several years, scientists had debated whether the models of the visual system that we were using really constituted an understanding of visual processing. In this study, rather than trying to answer that question directly, simply says, one of the fruits of understanding is the ability to do things that you couldn't do before.
And in this case, we show that we're able to control neurons in a way that is far more accurate than we were able to do before. And that has both scientific goal value and also applied goal value, potentially, clinically. And in that sense, the models are demonstrating that we have sufficient understanding embedded in these models to enable the things that we'd like to do with our science.
KOHITIJ KAR: We, through this project, brought back to life this closed-loop approach where model predictions and neural measurements go hand in hand. And I think this will be the more efficient way to go forward in model-building.
POUYA BASHIVAN: What I'm most excited about is the implications that this work could bring for neuroscience, that by formulating our knowledge of the brain into such a framework, we can now use it to reach inside the most complex structure that we know of, and that's the brain.
[MUSIC PLAYING]
Associated Research Module:
have an interactive transcript feature enabled, which appears below the video when playing. Viewers can search for keywords in the video or click on any word in the transcript to jump to that point in the video. When searching, a dark bar with white vertical lines appears below the video frame. Each white line is an occurrence of the searched term and can be clicked on to jump to that spot in the video.
