Learning to see late in life (1:17:28)
Date Posted:
July 7, 2020
Date Recorded:
June 29, 2020
Speaker(s):
Pawan Sinha
All Captioned Videos CBMM Summer Lecture Series
Description:
[The video is missing the first few minutes of the talk due to technical difficulties.]
Pawan Sinha, MIT
PAWAN SINHA: --point that's kind of relevant for all of us, given that all of us see ourselves as scientists with these dual goals that are near and dear to our hearts. One is the advancement of mankind's knowledge. And the other is helping humanity.
And we hope that by pursuing curiosity-driven research, at some point, the findings that we make will translate into tangible benefits for society. This translation does not always happen. And when it does, it might often take a long time. It might take several years, if not decades.
But what I want to share with you is the idea that there are some fortunate instances where these two enterprises of advancing knowledge, advancing basic knowledge, and helping humanity, these things can coincide in time. So you don't have to wait for many years to be able to have a tangible impact. The very pursuit of your research gets you to engage with societal problems.
And I'll describe this idea through the lens of a project that my lab has been pursuing over the past several years. This is-- we refer to it as Project Prakash. And you will see why that name and what it means.
So before talking about Project Prakash in particular, let me talk about the general domain of exploration that my lab engages in. And that has to do with neuroscience, but specifically, vision. And vision, as I'm sure all of you have already learned, occupies the lion's share of the sensory processing apparatus of the brain. By some estimates, about 30% to 40% of the primate brain is devoted to processing visual information.
And there are just a ton of interesting questions about visual processing that we would love to find answers to, but for which we have very little understanding so far. One of those problems is illustrated here. So this crazy-looking landscape of hills and valleys that you are seeing, this is actually an image that I'll show you in just a moment.
But I have re-represented the luminance information in the image by height to make it a little less familiar to you. But all of the information that the image has is present in this, just it's just represented in a strange way. And the point to take away from looking at this landscape is that it's very hard to tell whether there are any objects in here, if they are, what they are, there one begins and ends, what are the boundaries between objects. So there are very few clear cues about how to parse an image. And yet somehow we are able to do this.
Here is the image that this is a representation of. These two individuals, some of you might recognize them. The one on the left is Gandhi. And the one on the right is Nehru. Nehru was the first prime minister of India, of independent India.
And you can-- now that you see the luminance version of the image, you can perhaps begin to make some linkages between the patterns that you're seeing in the image and the patterns that you're seeing in this landscape. So for instance, Nehru's sleeves, which are quite wide, show up as these tall ridges. Similarly, his cap shows up as this tall peak, and so on. But it's quite amazing that the brain is able to take this very complex visual signal and is able to parse it into distinct objects.
So in effect, and to borrow words of a famous American psychologist William James, our brain is taking in a blooming, buzzing confusion. And it's creating an organized sensorium out of it. So how we do that, that is one of the biggest problems that we face as sensory neuroscientists.
So in order to understand what might be clues to this question, the temptation so far and the most natural experimental approach one can imagine would be to work with newborns, because these are the creatures who are actually experiencing this transformation from this blooming, buzzing confusion to an organized sensorium. So the hope would be that, by studying newborns, whether humans or non-humans, and examining how vision develops in these living creatures, we would be able to understand how this transformation from the confusion to organization happens.
But newborns, being newborns, they're very hard to work with. There are many developmental processes happening in the brain at the same time. It's hard to titrate out visual development from other kinds of developmental and maturational processes.
So the typical tendency on the part of developmental researchers is to wait for some time, months, if not years, until the baby, if we talk about human babies, until the baby is minimally able to comprehend what we want them to do in an experiment and is minimally able to report what they are experiencing. So you might wait for six or seven months and then work with that young six- or seven-month-old child, which is still a challenging thing to do.
But fundamentally, the problem with that approach is illustrated here. If you look at the top bar, these are the timelines of development in human infants. The top bar corresponds to visual development. And what you notice is that visual development is amongst the fastest-progressing developmental processes in the brain. By the time a child is seven or eight months old, they are already extremely sophisticated observers of the world. So if you have waited until that time to begin your experiments, then you essentially missed the boat for understanding how those visual proficiencies came to be.
So that then leads us back to this question. How do we study the very earliest stages of this transformation from the buzzing confusion to an organized sensorium? And I had been struggling with this question in my research career for a few years until serendipity and good fortune intervened and I became aware of a problem in the country I grew up in, in India, which actually gave us a way forward.
So let me take you through just a little bit of what this problem is. It's a societal problem that contains within it the seeds of an interesting research approach. So here is the societal problem encapsulated in one fact. About 1 in every 100 Indians is blind. Now, this is across all ages. It's a startling figure. I was not aware of this figure while I was growing up in India, but only became aware of it later in my life.
If you focus your attention just to children, it turns out that the rate at which children are being born blind in India, that incidence of childhood blindness, say, how many children for 1,000 life births are born blind in India versus in the US, that rate in India is about three times as high, conservatively estimating, about three times as high as it is in the US or in Europe. Now there are various causes of childhood blindness in India, and I would say in the rest of the developing world as well. You see some of the top ones here. So this corneal scarring, the child you see on the left is suffering from opaque corneas. Whereas you and I have these beautifully transparent corneas and lenses, children can either develop or even be born with opaque corneas.
The child on the right has cataracts. So you notice that the pupils of this little girl's eyes are white instead of being black. And that's because in both of her eyes, her lens has gone opaque. It's become white. So these optical opacities are preventing an image from being formed on the retina. And that's why these children are blind.
There are also various kinds of retinal dystrophies, so children being born prematurely, not getting the right kind of care, and developing retinal problems because of that, various kinds of infections that are common in the tropical world, things like trachoma, which can be treated, but if left untreated, they can lead to blindness, and the congenital rubella syndrome, which is a big one that afflicts India, in particular. So this refers to the fact that if a woman during the first trimester of pregnancy gets a rubella infection-- rubella is also referred to as German measles. So if the woman gets a rubella infection while she's pregnant, that greatly increases the chances that the child will be born with some kind of a challenge, whether a neurological challenge or a sensory challenge.
Blindness is one of the big consequences of congenital rubella syndrome. And India, unfortunately, did not have a program of Measles, Mumps, and Rubella-- the MMR vaccine that I'm sure all of us have received, it's still a rarity in the Indian countryside. So cases of maternal rubella are still disturbingly high.
So given these kinds of causes, it turns out that over 40% of all childhood blindness in India is either preventable or even treatable. But most of these children, because they're living out in remote villages far away from any kinds of hospitals, their parents might not even be aware that their child has a treatable condition. So for all of those reasons, these children very often stay with their blindness throughout their lives. They never receive treatment even though their condition is treatable. And that lack of treatment has devastating consequences on their lives.
These are some of the consequences that befall a blind child in India. Their lifespan is, on average, 15 years shorter than that of a sighted child. And that's for the lucky ones who are making it past the first few years alive.
So the WHO estimates that fewer than half of all children born blind live to see their fifth birthday. Less than 10% of the blind children get any kind of education. And a vanishingly small proportion are employed as adults. Most of them, even those who come from middle class families, they are reduced to a life of beggary just because of their blindness.
So here is clearly a humanitarian crisis. And it's clearly a race against time. If you do not provide treatment to a blind child, chances are very high that they are either going to die. And it's almost a certainty that they are going to live a very, very difficult life.
So there is a clear humanitarian need that we need to tackle. We ought to be identifying children like this boy who have treatable forms of blindness. We don't need to invent any new technology. We don't need to invent any new medical procedures. We already have these things. It's just a matter of having the will to go out into the villages, identify these children, bring them to the medical centers, provide them care. So that's the humanitarian mission.
But to link this to what I was saying a few minutes ago, embedded in this humanitarian mission is an incredible scientific opportunity. Doing this, bringing sight to a child who have been blind from birth, gives us a unique window into early visual development. So let's say you have a 10-year-old who has been blind until that point.
Through a half-hour surgery, you are able to transition this child from blindness to sight. From the very first moment that the bandages come off, you can study what the visual capabilities of this child are, what they are experiencing. And because they are old enough to speak, they can actually give you fairly detailed descriptions of their experiences.
So seeing this almost perfect synergy between the humanitarian need on the one hand and the scientific opportunity on the other, about 15 years ago, my lab launched this project that we named Project Prakash. "Prakash" is the Sanskrit word for light. And it refers to the twin missions of the project, to bring light into the lives of blind children, and in doing so to illuminate some deep questions about neuroscience.
And before proceeding further, I should say, please feel free to interrupt me at any point. I don't see all of you on my screen, but feel free to unmute and speak out if you have any questions.
So Project Prakash is organized operationally as a three-part effort. The first and logistically most complex part is outreach, just going out into the villages, finding these children. We obviously cannot sit in the hospital in New Delhi and wait for these children to come in, because as I was saying, the parents very often are unaware that their child's condition is treatable.
They have often been told-- in fact, this is what we have encountered repeatedly. The parents have often been told that their child's blindness is a consequence of the bad deeds that either the child or they did in a previous life. So it's seen as cosmic justice. And as soon as the condition is posed in those terms, then there is very little motivation on the part of the parents to go against cosmic justice and to upset the order of the gods to seek treatment for the children. So we have to proactively go out into the villages, identify children.
And here, if the video plays, is a little clip of an outreach camp.
[VIDEO PLAYBACK]
So this is the camp that we had organized in the [INAUDIBLE].
- [INAUDIBLE]
- [INAUDIBLE]
- Very good. [INAUDIBLE].
- [INAUDIBLE].
[NON-ENGLISH SPEECH]
[END PLAYBACK]
PAWAN SINHA: So somehow the video and the audio were misaligned. But what you didn't get to see-- the video cut out too early. What you didn't get to see was a child with cataracts who was identified as part of this outreach mission. So after these children are identified, we then have them come to a well-equipped hospital in New Delhi where we provide them the very best medical--
AUDIENCE: Excuse me. You're very quiet.
PAWAN SINHA: Oh, am I? How about now, because I didn't really change any settings. How about now, [INAUDIBLE]? Can you hear me?
AUDIENCE: Still a little quiet compared to where you were before.
PAWAN SINHA: Huh.
AUDIENCE: I can hear you really well. I don't know if it might be something else.
AUDIENCE: Yeah.
AUDIENCE: Yeah, [INAUDIBLE] it might be like--
AUDIENCE: I'll just try and [INAUDIBLE].
PAWAN SINHA: Sorry. Good, thank you. So we bring the children to New Delhi, where they are provided surgical care. And this is a very short clip of a pediatric cataract surgery.
[VIDEO PLAYBACK]
- [INAUDIBLE]
PAWAN SINHA: So this girl has just been given general anesthesia.
- [INAUDIBLE]
PAWAN SINHA: There is the cataract in the eye. And then the surgeon is placing an acrylic lens to replace the opaque biological one. And here is a boy for whom the first moment of bandage opening. His left eye has not been created at the time the video was made. We treat one eye at a time. The little bit of capillary bleeding is nothing to be concerned about.
[END PLAYBACK]
But this is, quote, unquote, "the magical moment" to understand what is the child seeing at that point. And even more importantly, how will this child's vision change as a function of time?
So that brings us to the third arm of Project Prakash. And that's research, understanding how vision develops after sight onset. But before we can get to the question of how vision develops, we have to seriously consider whether vision is going to develop at all.
So is the brain going to be plastic enough this late in life, say, you have an eight-year-old boy, or even a young adult, a 22-year-old that you see on the right, would the brain be plastic enough this late in life to adapt to this complex information that the eyes have now began providing it? And even if the brain changes, so let's say there is some plasticity in the brain, is this the right kind of plasticity? Would the child actually benefit from that change? Would the child be able to make use of the visual information for performing tasks with more proficiency?
So the reason we have to ask these questions and the reason, in fact, why we might even be a bit pessimistic about the answers to these questions is because of a rich body of work in neuroscience related to the notion of critical periods of development. And this is just a very small sampling of that work.
So many researchers, most notably David Hubel at Torsten Wiesel who won the Nobel Prize in 1981 for their beautiful work, they have looked at the consequences of visual deprivation early in life on later visual performance. And what they have found is that even short durations of early visual deprivation can be devastating. Here is a figure from Hubel and Wiesel's groundbreaking work.
So they worked with kittens. And say, in this case, as soon as the kitten was born, they sutured shut one eye. So the kitten is now walking around for maybe a few months viewing the world only with one eye.
After this period, Hubel and Weisel cut the sutures. So now the kitten can observe the world with both eyes. And as far as the physical structure of the eye is concerned, both eyes look perfectly normal. But if you look at the brain, some dramatic changes are apparent.
So in a kitten that has suffered monocular visual deprivation, almost all of the neural machinery in the primary visual cortex it turns out has been taken over by the dominant eye, by the open eye. And this change seems to be permanent. No matter how long the kitten now walks around with both eyes open, that initial period of three months or four months of visual deprivation seems to have affected the brain permanently. And it compromises vision in the closed eye permanently. And the kitten is effectively blind in that eye.
So extrapolating from this to the Prakash children, one would be worried that if a few months of visual deprivation can have such dramatic adverse effects and permanent adverse effects, then coming onto the scene a few years into that deprivation is way too late. And these children are not going to be able to do any useful vision. And we should not be exposing them to the risk of surgery, given that the benefits, the expected benefits are nearly zero.
But before we make that extrapolation, we have to keep in mind that these studies that I just talked about, they might not be directly linkable to human children. Firstly, most of these studies have looked at the consequences of monocular deprivation. And monocular deprivation might have very different effects than binocular deprivation, because with monocular deprivation, you're setting up a competitive interaction between the two eyes, whereas with balanced deprivation, maybe the neural resources for both eyes stay more balanced.
Most of these studies are with non-human subjects. So the kinds of studies that can be done probing different aspects of vision are much more constrained because these are non-verbal animal subjects. The durations of follow up with these animal subjects have been fairly short, a few months, so we don't know how vision develops over a matter of, say, years. And as I said, because these are animal studies, the assays that are being used are quite limited.
So the bottom line is that many fundamental questions about human brain plasticity still remain open. And to address these questions, we need data from human subjects. And Project Prakash has now begun to provide some beginnings of answers to these questions.
So I'll share with you some vignettes of the kinds of things we are finding. We have had the good fortune of having access to a neuroimaging facility in New Delhi. So we can work with these children prior to their surgery and then at multiple time points after surgery and examine both structural and functional aspects of brain organization as a function of time. And what we are finding is quite startling.
So you see here results from a 20-year-old participant, so blind for the first 20 years of life. And then you see him. These are resting-state functional connectivity patterns two days prior to surgery, t minus 2, a week after surgery, a month after surgery, and four months after surgery.
And even without getting into the details of the analysis that we have done, so this is-- just for those of you who are interested, this is Independent Component Analysis of resting-state functional connectivity data. So ICA is pulling out chunks of the visual cortex that are behaving similarly over time. So when one part of the visual cortex is active, if another part is also co-active, then it'll show up as being part of the same component. So what you notice is that prior to surgery and soon after surgery, much of the visual cortex, which is the back of the brain, much of the visual cortex is behaving monolithically. But then as time progresses, there seems to be this de-correlation of activity across the brain.
This is interesting from two fronts. One, at a very high level, it shows that there is preserved plasticity in the brain, even two decades out, that something about the brain, some functional connectivity patterns can undergo rapid and massive changes, even 20 years after birth. And second, the kind of change that we are experiencing, or that we are observing, this de-correlation, seems to be very intriguing, because an efficient information encoding system you would expect to be de-correlated.
You don't want all of your units in an encoding system to be doing the same thing. That's redundancy. That's wasteful. You would expect the visual cortex to be de-correlated. And that's what seems to be happening as a function of time in this individual and others like him. That's one.
Here is another example. Here we are looking at the development of face-specific responses. So I'm sure many of you have heard about face-specific responses in the temporal cortex of the normal brain. And our question here is, would the Prakash individuals exhibit face-specific responses? And if so, would they land up in the same locations that they occupy in the normal brain?
So here you have a 21-year-old. You're seeing him at two days post op. And quite interestingly, even just two days post op, you begin to see a face-related activation in roughly the location where it would be in the neurotypical brain. And then as time progresses, that activation gets progressively stronger until, by about a year post op, the activation is, at the resolution that we are imaging things, indistinguishable from normal activation.
AUDIENCE: So I just have a quick question.
PAWAN SINHA: Certainly, Will.
AUDIENCE: These participants who receive the operations, are they given specific trainings after operations to facilitate?
PAWAN SINHA: That's a great question. Yeah, so no training. And we have held off from training so far, firstly because in the initial stages of the project, we didn't know what training to give. We didn't know what the landscape of visual abilities was going to be like post surgery.
And secondly, we did not want to influence the normal course of learning. So we wanted to observe, what can the brain do just on its own through experience with the natural world? But now that we have about 15 years of experience working with these children, we now have a better sense of which abilities might be slow in developing or might even be permanently compromised. And those become the targets of our visual rehabilitation routines.
AUDIENCE: Thank you.
PAWAN SINHA: Great question. And we also-- this some work that we are writing up. We've also looked at whether there are structural changes. So the two changes that you saw in the previous slides, those were functional changes. We are also surprisingly finding that the anatomy of the brain, the white matter tracts themselves undergo changes post surgery in these individuals.
So on the first question, is the brain plastic enough to make use of information from the eyes later on in life? The accumulating evidence that we are getting seems to suggest that the answer is a yes. The brain does have the ability to change, even late in life.
But perhaps the more important question from the perspective of would a child benefit is the behavioral question. Would the child's behavioral skills in terms of being able to use visual information for performing tasks, does that register an uptick? And the best way to get a sense of that is just by looking at the difference between the child pre surgically and post surgically.
So here, again, if the video plays, you will see a young girl, 11 years old. Her name is Soumitra. And you will see her pre surgically and post surgically. And in both cases, we have asked Soumitra to try to find a box of candy that we have put by the side of the hospital corridor.
So pre surgically, her vision is just light perception. She can tell night from day, but nothing more. So she is, for all practical purposes, blind. And it seems almost cruel to be asking her to do this task knowing fully well that she would find it impossible, but we simply wanted to document the magnitude of her visual impairment. That's her father in the background.
Now you see her eight days following surgery. So it makes a world of a difference. And children benefit tremendously from the surgery. But that's just an anecdote. And we have many such anecdotes. But of course, as scientists, we need to be a lot more rigorous in documenting changes.
So over the past several years, my students and I have been systematically following the development of various different visual skills, including low-level visual abilities, things like resolving patterns of seeing patterns at different contrasts, so contrast sensitivity functions, acuity, and also higher-order things like detecting faces and images, imagining structures in your mind, perceiving visual illusions, and so on and so forth. And the story in most of these experiments is that these children, even though they might not reach normal level of proficiencies, there is a significant improvement from their pre-surgical state. So it's a story of residual plasticity and residual ability to learn even late in life. So to the second question, can a child blind for several years since birth benefit from optical correction of the eye, the bulk of our evidence seems to answer that question affirmatively.
So these are the inferences that we derive from the successes that we observe in the Prakash children. Can we derive any inferences from the failures of the children? And here is one instance of failure.
So I'm not sure if you can tell, but the person looking at this image is actually the same. So the person on the screen in the blue shirt is the same as the person on the right of your screen. He's looking at these people on the screen.
And I asked him, well, can you recognize who these people are? Now, he has interacted with us. The two people that you see here standing next to him are two graduate students from my lab.
And he has been interacting with us for several days. And yet, when asked this question, he is unable to recognize himself or the two grad students. And this seems to be a common refrain across the Prakash children. They perform poorly on face identification.
So when we assess their post-operative vision, we find that many dimensions of vision register an uptick, but face identification seems to be seemingly permanently compromised. And the question of course is, why would face identification in particular suffer this compromise? So why this failure?
This failure has been accounted for in the past by appealing to a critical period kind of an idea. And it seems like a very natural explanation to give. So here is a quote from Charles Nelson's paper. Chuck Nelson is a very eminent visual developmental scientist. He is based at the Children's Hospital here in Boston.
So he says, it seems likely that face recognition reflects an experience expectant process, whereby exposure to faces during a sensitive period of development likely leads to perceptual and cortical specialization. And the implication, of course, is that if during the sensitive period a child is deprived of exposure to faces, then they face-recognition machinery will not forevermore develop normally.
Now, this may well be right. But this assumes some specialness of face processing. So effectively what we are saying is, these other aspects of vision that do develop don't suffer from this critical period idea, but there is something unusual about face processing that makes it vulnerable to early deprivation.
And again, biology being the complex thing that it is, this may well be the case, that somehow there is something particularly vulnerable about the face processing machinery that makes it susceptible to deprivation. But as scientists, we have to always think about what are more parsimonious hypotheses before we resign ourselves to a very specific one. So that's what we started thinking about. Could there be a more broad hypothesis that does not depend upon any domain-specific assumptions about the specialness of faces?
And this led us to thinking about the development of acuity. And this is a very interesting story just in its own right, the development of acuity in a normally sighted children. So when we are born, our acuity is really terrible. We have about 20 over 800 vision. So normal vision we talk of as 20/20 vision.
And what that means is that a person, a normally sighted person can resolve at 20 feet the details that an average human can resolve at 20 feet. That's what 20/20 vision means. If one has worse than normal vision, so 20 over 800 would mean that, if I have 20 over 800 vision, then I need to be at 20 feet to resolve what an average human can resolve at 800 feet.
So my acuity is 40 times worse than normal acuity. So infants at the time of birth are starting out with 20 over 800 vision. This then steadily improves, especially over the first few months of life. And by about two to three years of age, babies acquire normal vision.
Do any of you have any guesses as to what is causing this initial acuity degradation? Why is it that, as babies, we have such poor acuity, which is worse than the criterion for legal blindness? So any guesses what might be limiting our acuity at birth?
AUDIENCE: Could it be the size of the eye in the eye socket?
PAWAN SINHA: So that's an interesting thought. So yes, the size of the eye is different during infancy. But one could imagine the lens changing its shape to compensate for the sight of the eye. So that seems not to be the primary culprit here. Any other thoughts?
AUDIENCE: Maybe the brain just hasn't developed enough to recognize, like, images.
PAWAN SINHA: Good thought. So it is certainly the case that the visual cortex and many other areas of the brain are still immature. The dendritic arborization of many of the neurons has not reached its full extent. So there is certainly cortical immaturity that causes some of this acuity loss, but it's not the bulk of the acuity loss. There is one factor that is primarily responsible for this acuity.
AUDIENCE: Does it have to deal with the fovea?
PAWAN SINHA: You are warm. So what about the fovea, do you think?
AUDIENCE: Like, they haven't fully developed enough, so that there isn't-- so like, the best acuity is at the fovea. And so I'm assuming that that area is still-- like, light needs to hit the eye enough in order for that specific spot to kind of develop.
PAWAN SINHA: Very good, so except for that last bit. So light does not need to-- does not play a critical role. But you are right that there is some immaturity in the foveal zone that is limiting. That's the primary cause for this acuity loss. That immaturity turns out to be this.
AUDIENCE: Can I ask what the fovea is?
PAWAN SINHA: Yes, absolutely. So the fovea is the section of the retina along the optical axis of the cornea and the lens. So if you were to draw a straight line to the center of the cornea and the lens, you would hit a spot of the retina in the back of your eye that corresponds to the fovea. So that's just the location of the fovea.
The fovea, if you examine it with an opthalmoscope, is a little bit of a depression. So there is a little indent or a little dimple in the retina. And it's jam packed with cone photoreceptors.
And the neural circuitry in the foveal zone is such that it gives us the very best acuity, as I think Ian was just mentioning. So our highest acuity is at the fovea. And as you move away from the fovea, our resolution degrades quite quickly.
In fact, on a tangent, it's very interesting to note that the extent of the fovea is just about two degrees. Two degrees is the width of your thumb at arm's length. So if you were to hold out your thumb at arm's length, the space that that occupies in your image, in your visual image, that is the full extent of the fovea. And that's the zone at which we have high-resolution vision.
Everywhere else, we have really crummy vision. But it's a testament to the brain's ability to fill in information that we have this illusion that we have high resolution everywhere, when in reality, we have high resolution just this tiny little zone. And everywhere else, it's crummy vision.
So the fovea is really important for high-resolution vision. And as I mentioned, it's jammed in the adult eye with very fine cone photoreceptors. Photoreceptors come in these two flavors, cones and rods. And the fovea is exclusively comprised of cones.
But in an infant, in a newborn, the cones that are sitting in the fovea are fat. Literally, they are much bigger than the cones in the adult eye. And that's what's shown schematically here.
This on the left is a newborn cone. And so you notice how big that photoreceptor is. And on the right, this thin pencil-like thing, that's an adult cone. So because the infant cones are so fat, their packing density, as you see in the newborn corn lattice schematic, is much less than the adult cone lattice. So it's as if you have a much lower-resolution image that the infant eye is able to supply to the brain than the adult eye, more sampling points in the adult retina than in the newborn retina.
This maturation of the cones, of the newborn cones to the adult cones, that, interestingly, is not dependent on light. This maturation keeps happening even if a child has opaque corneas or opaque lenses. And you can kind of see where this is headed.
So a normally developing child is going through this process of gradually thinning out his or her cones. The acuity improves. So after experiencing the world in this degraded manner, after a few months, the child begins to experience the world at higher resolution and eventually reach its 20/20 vision.
Now, in a child born with a cataract, this maturation of the retina is still happening, but the child is not getting any visual imagery. Now let's say we find this child. Say it's an eight-year-old child. We find him, provide him surgery. We remove the cataract lenses. The images that begin to fall on the retina, they are being picked up by a high-resolution sensor.
So because this retina has already matured, the cones have become fine. So this child will begin to experience the world right away at higher resolution than what a newborn child does. And that is the hypothesis that we want to build upon, that perhaps it's this unusual access to high-quality imagery right away without this initial period of degraded imagery that might in fact be causing problems.
Even though one might naively think that starting out with high acuity might be useful instead of having to contend with poor-quality imagery, we are proposing that that initial period of poor-quality vision might in fact be useful. So how might poor acuity at the outset help? And by extension, how might the Prakash children be suffering by having missed out on this initial period of poor acuity?
So here is the intuition. Don't worry about reading all of the text. The intuition is that if you have degraded imagery, a very blurred image, then looking at just any local section of that blurry image is not going to give you very much information about what that object is. You necessarily have to expand the amount of spatial integration you do. And you have to bring in information from the greater part of the image, so like that. You have to expand your integration field in order to be able to make any guesses about what this object is in a blurred image.
But if I were to show you a high-resolution image then even small sections of that image are information-rich. And I have essentially taken away the impetus on the part of your visual machinery to expand the integration fields. And just to complete this thought, anybody recognize who this person is?
AUDIENCE: JFK.
PAWAN SINHA: JFK, very good. So that's the intuition. So why that that newly sighted face identification is compromised? It may be the case, we hypothesize, that such individuals may have small integration fields, leading to compromised holistic processing and a bias towards local processing. So that's our tentative hypothesis. Is there any evidence other than ours to support this idea?
So here is some interesting corroborating evidence that comes from the lab of Daphne Maurer, an emeritus professor in Canada who has worked with children who were born in Canada with cataracts. I mean, on rare instances, children even in the US or in Canada, in the developed world, are born with cataracts, but because they are being born in a good medical system, their cataracts detected right away. And they get treatment within a few months of birth.
So what Daphne does is to follow up with these children several years after they have been treated. So from the medical records of the hospital, she finds out who were the children who were treated, say, 15 years ago at such and such hospital for cataracts. And then she contacts these children who are now teenagers and performs studies with them. And what she finds, remarkably enough, are some lingering deficits in these children.
So here is one report of such a deficit. So Daphne is testing these children on a face-discrimination task. So look at these two pairs, the upper pair, pair A, and the lower pair, pair B. These two pairs differ in terms of what's been done to make the two faces different in each case.
For the upper pair, the features, the local features are the same. The eyes, nose, and mouth are still the same, but the spatial configuration has been changed. So you notice that the mouth has been moved a little up in the image on the right relative to where it was in the image on the left. But the mouth itself is the same. The eyes are the same. But they have been moved a little further apart. So this, Daphne calls configural changes.
In the lower pair are faces that have local featural changes, so where the eyes themselves are different. They are in the same position, but they are different eyes. The mouth is different. I think the nose is the same. So this is a featural change.
And what Daphne finds is that these children who had suffered deprivation for, say, the first six or seven months of life, when--
AUDIENCE: (SINGING) You set me free.
PAWAN SINHA: Kayla, did you have a question?
AUDIENCE: No, sorry. I accidentally unmuted my microphone for a second.
PAWAN SINHA: All right, no worries. So what she finds is that, on this local featural processing task, the patients are performing as well as controls, but on the configural task, they are significantly worse. So a task that requires spatial analysis, spatial integration, these children end up being worse on, consistent with the hypothesis that we are formulating. So again, just to reiterate, my hypothesis is that the Prakash children's difficulty with face identification may arise in part from not having experienced the visual world in a degraded manner. The typical acuity trajectory provides that opportunity to most children.
And one other thing to point out is that this is not a face-specific hypothesis. To be a saying that any task that requires spatial integration would suffer. And indeed, when we have studied these children's performance on other spatial integration tasks, we find a similar story, that these children are worse than their normally sighted counterparts at those kinds of tasks, which may be completely unrelated to face recognition.
So the conventional explanation that was talking about sensitive periods of development, so that relied upon the notion of a reduced plasticity, that post sensitive period, the brain simply doesn't have enough plasticity to be able to acquire facial recognition skills. The proposal that we are making can be thought of as a distorted learning hypothesis, that the kinds of inputs that the brain is getting because of these kinds of maturational factors in the eye, they are presenting to the learning system unusual inputs that are causing this compromise.
So we asked, can we test this idea? It's fine as a hypothesis, but can we test this computationally? So we decided to do the testing with deep networks. Somehow or the other, deep networks get involved in our project.
So I'm very cautious in terms of talking about deep networks as proxies for the human visual system or the primate visual system, more broadly. But at least for the first few stages of deep networks, we can be fairly confident in saying that the kinds of receptive field structures that emerge in the convolutional layers of deep networks look quite similar to the kinds of receptive field properties that have been observed in V1 neurons, for instance. So without going deep into the deep network, can we say something about what are the attributes of the convolutional layers in these deep networks as a function of the kinds of training inputs that we provide to it?
So can we have a deep network trained in a biomimetic manner, so progressing from poor-quality images to high-quality images, and then observe what the receptive fields are in the convolutional layers to see whether there is any systematic difference in those receptive fields relative to the ones that the network would come up with if it were to be trained just with high-resolution images, as it's conventionally done in machine vision? So what happens to receptive field structures when training it with high-res images alone versus low-res images?
And this is what happens. So in these five columns, what you're seeing are the receptive fields in the convolutional layer of AlexNet after it has been trained with images of different resolution. And what's hopefully apparent to you is that the network that's been trained with high-resolution images on the far left is exhibiting these receptive fields that are small and are focusing on high-resolution structures, fine [INAUDIBLE].
On the far right is the network that's been trained with exclusively low-resolution images. And you notice that its receptive fields are large. And it's emphasizing coarse image structure.
So when I first saw this result, I thought that this, well, this is kind of obvious. If a network has been given low-resolution images, how surprising is it that it would come up with big and coarse receptive fields relative to one that's only seeing high-resolution images? But in thinking more about it, I realized that this is not entirely an obvious result. The network that is looking at high-resolution images, it has access not only to high-resolution images for the high-resolution content of the image, but also to the low-resolution structures in the image.
So it could, if it so decided, it could have discovered low-resolution receptive fields as well, because it has access to that information. But what ends up happening is that the network, to anthropomorphize it, falls prey to the temptation to just use high-resolution information because that suffices for the task of classification. So the only thing it discovers is these high-resolution receptive fields and no low-resolution ones. In order for the network to discover low-resolution receptive fields, you externally have to limit the information in the images. And that's being done by optical means. So that's one result that the receptive field structures differ.
Well and a couple other-- I'm not going to take you to the details of these plots. But let me give you the takeaways from these plots. So starting from the top left, the low to high progression, so instead of just showing the network low-resolution images, let's now follow the more complete biomimetic progression of going from low resolution to high resolution. And then also let's reverse the process, go from high resolution to low resolution, so that, in aggregate, the network would have seen the same set of images. It's just the temporal ordering of these images is different. And what we find is that the temporal order matters. So the low-to-high and high-to-low progressions are non-symmetric in their effect on receptive field sizes.
Second, even a fairly short period of low-resolution training seems to be sufficient for expanding the receptive field sizes. So let's say you just have that 20% of all images being in low res is already enough to get the full benefits of that initial degraded experience.
And finally, in terms of performance, the low-to-high progression, the biomimetic progression, is most effective for enhancing classification performance across multiple resolutions. So a network that has had this exposure to low- and then high-resolution images actually outperforms a network that has only seen high-resolution images or only seen low resolution, so it-- or has seen the high first and then low. So this biomimetic progression seems to be a good strategy if all that you cared about is performance of the network.
So the inferences from this fairly straightforward computational exploration are, one, face-processing difficulties in the newly sighted may be due in part to their unnaturally high initial acuity post-operatively. Thinking about what this says for normal visual development, this suggests that the poor initial acuity that we start out with as babies might not in fact be a limitation of biology, but rather a very clever way of turning that biological limitation into a feature. So our visual performance might actually be benefiting from that initial exposure to degraded imagery.
And from the pragmatic perspective of machine vision, perhaps machine vision systems can gain added robustness by adopting aspects of biological visual development. So instead of just throwing high-resolution images for training these deep networks, maybe following this biomimetic progression is a better idea. And in case you're interested in following up on this, here is a paper that we wrote about this not that long ago where we describe the results and also this general idea of initial degradations being adaptive.
So that's the story of visual acuity. But this idea, the adaptive initial degradation hypothesis, might actually extend to other aspects of vision as well. But before I do that-- oh, so we are already at the end of our time. So let me then not take you through this. Let me just end quickly.
But while I'm doing that, a similar story seems to hold for the perception of colors. Babies start out having very crummy color vision and then gradually improve their color perception. And computationally testing the idea reveals that this progression from poor color to good color actually gives the network added robustness to color change in images. And we are going even beyond vision entirely and looking at whether this notion of initial degradations being adaptive might also apply in other sensory modalities like audition. And we'll skip all of that.
So we call these effects butterfly effects, after [? Lorenzo's ?] description of this, of this effect of a butterfly in the Amazon rainforest flapping its wings and leading to a hurricane in the North Atlantic. Similarly, early changes in the developmental trajectory might have long-lasting consequences or important consequences for later on in life. And Project Prakash is very well-suited to examining such butterfly effects.
And this line of investigation has allowed us to bring together three threads that we are, the students and I are all interested in, our interest in understanding development after deprivation, our interest in understanding normal development, and our interest in machine learning. So both the successes as well as failures of the Prakash children are beginning to give us some insights into this big question of how we go from a blooming, buzzing confusion to an organized sensorium.
So Project Prakash has been this amazingly gratifying experience for all of us. It's given us some insights regarding brain plasticity and learning. Something that I didn't mention, it has guided our investigations of autism quite unexpectedly. And again, I'm happy to send you a paper describing that work if you're interested. It's guiding the design of our AI systems for autonomous visual learning. It's serving as a model of an alternative paradigm to combine basic science with societal service.
When I was getting my training in the sciences, I always thought of basic science as being a completely different kind of pursuit from engaging with society and immediate pressing needs of society. But what Project Prakash is telling me is that you don't need to have the dichotomy. You can find opportunities for merging both of these interests together. And of course, Project Prakash has helped alleviate, in some modest way, the problem of childhood blindness in India.
We have been profiled prominently in scientific and lay destinations. But the most gratifying part of Project Prakash is just witnessing the kinds of transformations children go through. You see a child coming in very helpless into the hospital.
And then you see him or her a few months later a completely transformed person, like this child a year after his surgery is now learning to ride a bicycle in his village, or these children who are learning to express their creative talents after gaining sight. We run a program called Unruly Art where we have newly sighted children engage, not just in appreciating art, but in actually creating art. And we have an exhibition of some of the art pieces at Princeton University not that long ago.
So so far we have screened over 40,000 children, provided surgical care to over 500 of them, and non-surgical care to many more. We've also very recently started an education initiative for these children, because most of these children have stayed out of the educational system because of their blindness. And after they gain sight, they're kind of in this odd position. They are somewhat older than the normal child who would be starting their education. So many schools are hesitant to take them on.
So we are starting our own educational program. At the moment, it's only for girls. And we are developing a special curriculum for them. And that explains the three-part-- the three petals of the Prakash logo. It's a merging of health care, scientific research, and education.
But before we begin to get too complacent about the journey so far, we have to keep in mind that we've only scratched the surface of the problem. So this gives you a sense of the scale of the problem. The green dot represents all of the children that we've treated so far. And the red dots represent all of the children who are still waiting to be identified and to be treated. So it's a massive, massive problem that we have to get ready for.
AUDIENCE: Just a quick question.
PAWAN SINHA: Yes, [INAUDIBLE]?
AUDIENCE: So would you say that a large part of the issue is actually getting families when you're doing outreach work to agree to let their children get the treatment? Like, I was wondering what kinds of cultural barriers do you all face when you're doing the outreach work?
PAWAN SINHA: Yeah, so when we were starting out, it was a big cultural barrier, because I would go there. And my Hindi, which has now become quite poor after having been in the US for over 25 years-- so even that halting Hindi that I would try to speak to the parents would be a barrier to them trusting me. So now we have increased the representation of local people in our outreach teams to convince the parents that there are no ulterior motives here. When we are offering to treat the child for free, it's not for any nefarious reason that we are doing this, but rather this is truly something that they can trust. And we are different from the kinds of quacks that they might have run into in the villages. Some deep-seated issues about, say, the religious beliefs of people believing that their child's blindness is because of bad deeds in a previous life or bad karma, those are harder to overcome.
AUDIENCE: Yeah, so I was wondering, also kind of to tag on, so I know there is a group at my school that does something similar but for mental illness in rural India. And I know that that exact issue is something that they have a lot of problems with, which is just deep-seated religious views when it comes to mental illness. And I was wondering if y'all have started kind of like educate-- I know that they had some success with kind of implementing earlier education initiatives in the communities, so like starting community classes about mental illness, or in your case, about childhood blindness, and that it is a very treatable thing to kind of get that as something in the back of their minds for when they do come and approach these families?
PAWAN SINHA: Yes, absolutely, [? Ian. ?] So that's a very important point. Every outreach camp that we hold, we also try to include awareness building. It's a very small intervention directed towards educating the local rural population about, say, the eye conditions and how they can be treated.
But I can't say that we have really cracked that nut completely. We are still running up against this problem. Although what we are finding is that our best ambassadors and the best spokespersons are the children whom we have treated.
So on occasion, we have taken a couple of the Prakash children who have already been treated to the outreach camps. And it's an amazingly convincing pitch to talk about how these children were also initially subject to the same kinds of beliefs from their parents. And now look at how their lives have changed. So that, I think in the long run, might end up proving to be the most effective thing to--
AUDIENCE: Right. And so as you get more and more patients, it just becomes an exponential thing.
PAWAN SINHA: Exactly, yeah.
AUDIENCE: Thank you.
PAWAN SINHA: Certainly. So just to conclude, because I know we are running way over time, so one of the things that we are hoping to do is to set up a dedicated facility for Prakash. At the moment, we are working with a medical partner.
And there are issues of scale. How many children can that hospital treat in a given month? We are hoping to set up our own pediatric hospital in close proximity of a neuroscience research center and a residential school so that we can physically bring together these three components of Prakash.
It's a huge undertaking. I've never done anything of this scale ever. So it's daunting. But whenever one is faced with such overwhelming challenges, it's good to remember Albert Einstein's famous words, those who have the privilege to know have the duty to act. So if we as scientists know of a problem in the world and we also know how that problem can be tackled, we have the solution for that problem, then we are morally obligated to do everything that we can to act upon that problem and to provide a solution to that problem.
And it's in that spirit that my students and I keep pursuing Project Prakash. Please feel free to visit us, ProjectPrakash.org. And feel free to write to me directly for any further information. And with that, thank you very much.
have an interactive transcript feature enabled, which appears below the video when playing. Viewers can search for keywords in the video or click on any word in the transcript to jump to that point in the video. When searching, a dark bar with white vertical lines appears below the video frame. Each white line is an occurrence of the searched term and can be clicked on to jump to that spot in the video.
LEARNING