Rebecca Saxe: Patterns of Minds: Decoding Features of ToM and MVPA
Date Posted:
June 6, 2014
Date Recorded:
June 6, 2014
CBMM Speaker(s):
Rebecca Saxe All Captioned Videos Brains, Minds and Machines Summer Course 2014 Description:
Topics: Theory of Mind (ToM): false belief task embodies a difference between reality and what a character thinks; 5 year olds can do false belief tasks while 3 year olds cannot; fMRI study reveals regions selectively involved in reasoning about mental states, in right/left temporal parietal junction, medial prefrontal cortex, and posterior cingulate; decoding features of ToM, e.g. what is the population code of neurons (voxels) that encode information about mental states; test case: morally relevant thoughts, e.g. distinguishing stories with very similar events but differing moral blame; scale from accidental vs. intentional harm is correlated with RTPJ activity; information about justification and source of information (e.g. visual vs. auditory) also encoded in RTPJ activity
REBECCA SAXE: Thanks so much for having me here. I keep hearing wonderful things about what's going on at this school. Sounds like a really fun time. Also the weather could hardly be more beautiful.
And so I'm going to tell you today about some of the work that's been going on in my lab recently, using techniques that I think you've heard about already in this summer school, through using fMRI decoding techniques and now applying them to social cognition, kind of as a connection between the "neuroscience and social cognition" thrust of this program. This is work in my lab with a bunch of people, including my current grad student, [? Jorie, ?] her post-doc Amy, and ex-post-docs Liane and Marina. They're all running their own labs.
So the overall topic that my lab has been working on for quite a while now is how can we use neuroscience to understand high-level human capacities and, in particular, theory of mind-- the ability that people have to think about other people's mental states. This is an exciting thing to be studying, in part, because it's a very abstract aspect of cognition. So it includes all of the hard problems of really high-level intelligence. And so it sets the bar for really hard problems in the eye as well. But it's possible to study with neuroscience, because of the existence of the distinctive brain regions involved in this process.
And so that's what I'll talk to you about. But just to set the stage, the beginning of theory of mind research as a problem in developmental psychology, cognitive psychology, and eventually cognitive neuroscience starts with a task called the false belief task, which was the original litmus test for studying kids' developing ability to think about other minds.
To give you a sense of that, I'm just going to show you a child being given a false belief task, so you get a sense of what we're talking about. This is a five-year-old.
OK so that's called "passing the false belief task." And the key idea here is that when the reality of the situation-- so which sandwich is really Ivan's-- is different from the reality of a character's mind, which one he thinks is his.
You can use action-prediction, like what do you think he's going to do, as a litmus test for whether or not the observer, in this case the child, can take into consideration the character's mind-- so predict that Ivan is going to take Joshua's sandwich, because he thinks it's his.
This task was developed originally for use in chimpanzees, who can't do this kind of task, and then subsequently became extremely famous as a test of the development of explicit reasoning about other minds in young children.
Part of why it became so famous is the discovery that while five-year-olds can do this task, three-year-olds look really different. So just to give you a sense of that, here's a three-year-old, just the very end of the same story.
OK. So that's called "failing a false belief task." And it has a number of different elements in it. So one is, of course, that when Ivan said, I want my cheese sandwich, and I asked which one is he going to take, the child predicts he will take his cheese sandwich.
So that's the kind of most basic level, that if he says he wants his cheese sandwich, the child doesn't take into consideration his false belief about which one is his. The second thing to note about this kid, which is totally characteristic, is that three-year-olds don't fail because they don't know, or they're guessing, or they experience a lack of confidence, or being a chance; three-year-olds pick the wrong answer 100% of the time and with great confidence.
If you train a three-year-old to bet, they bet 100% of their counters that you will take his cheese sandwich. And the third thing is, this doesn't only show up in action-prediction, if instead of asking them to predict what will happen, you show them what will happen-- so you show them Ivan taking the wrong sandwich and you ask them to explain it-- they characteristically confabulate other reasons, basically reasons why he wouldn't want his own cheese sandwich anymore, like his is dirty because it's fallen on the ground.
So that sort of constellation of phenomena has been studied in terms of showing that three-year-olds who do think about other people's actions in terms of their minds, think about minds as having a relatively limited set of ingredients, ingredients like desires and plans, but not beliefs. And then there's the development of many subsequent concepts and ideas that are even more complicated than that, that develop even after age five.
OK. So here's the basic observations. If you use an explicit task like this and ask to provide their predictions or explanations, you get predictions and explanations that refer to mental states like beliefs at age five, not at age three. This has been tested all over the world, not only in developed and Westernized countries, but also in hunter-gatherer societies; and this approximate age transition is the same across a massive range of cultures.
And, then, I think the reason why this task became the focus of so much research was the discovery very early on, that there's an exception to this relatively universal pattern, which is children with autism spectrum disorders.
So autism, which is a neurodevelopmental disease, doesn't harm cognitive functioning equally. It has a very disproportionate effect on social cognition. So relative to kids with autism, relative to their own IQ and other kinds of cognitive abilities, they struggle disproportionately when the task requires thinking about other people. And this task was such an example. So kids with autism failed this task way out of proportion to their ability to do other logical reasoning tasks, even ones [? that ?] matched difficulty.
OK. So that set of observations, as of really the '80s-- this is a very old idea-- led to the hypothesis that there might be a distinct brain system involved, and so that we could study the neural mechanisms underlying this human cognitive capacity; again, based on the idea that anything that develops relatively universally and can be disproportionately affected by a neurodevelopmental disease might be supported in typical function by a distinct mechanism.
Of course, there was no idea what kind of mechanism that would be. Because to the extent that this is unique to humans, it can only be studied in humans; until it was possible to study noninvasively, and that became possible with the advent of neuroimaging. So as early as in the mid '90s, when PET became available, and then certainly in the beginning of the 2000s, with fMRI, people started testing this hypothesis, that maybe theory of mind has a distinct neural basis.
In my lab, we do this with fMRI. And so I'll just give you a brief introduction to the fmRI methods that are kind of the basic foundation for how we do our research. And then I'll tell you more about the current research we're doing.
OK. So in our basic experiments, somebody-- [INAUDIBLE] lying in an fMRI machine. And while they're there, they're reading short, usually verbal narratives. I should say, by the way, I'm happy to take any questions at any time, on any topic that I mention and even some things I don't mention.
OK.So somebody in an fMRI machine, lying on their back, reading short, verbal narratives-- this is one thing you might wonder, why we use verbal narratives. Question for the reader. Anyway, so they're reading short, verbal narratives. And these narratives are designed to pose problems like a false belief task.
So this one says, the morning of the dance, Sarah placed her shoes under her dress and went shopping. That afternoon, her sister borrowed the shoes and put them under the bed. And so this sets up that same kind of difference between where the shoes really are and where Sarah thinks they are. We can then probe your understanding by asking you either where Sarah's shoes are or where she thinks they are. And as this goes along, you're also reading other stories that don't have any people in them and don't have any mental states in them.
So here we have a volcano that destroys an island. We have photographs of the island from before the eruption. And again we can ask you either about the current reality-- what does the island look like now-- or about the reality preserved in the photograph-- what does the island look like in the picture?
The idea here is that both of these kinds of stories require many of the same elements of cognition. They both require being able to read a sentence. They both require the syntax of putting the sentences together out of words, and the words into stories. They both require considering two alternative ideas of reality-- the real one and the one in the representation-- either in the mind or in the photograph, and then choosing between those two, based on the question that we ask you.
That's the kind of contrast that kind of traditional MRI analyses depends on. We're going to ask over and above the brain regions that are sufficient for you to solve a problem about a false photograph, such as see the stimulus, read it, understand it, think about two realities, choose one, and make a button press. Over and above that, are there any brain regions that are additionally recruited if the story describes a person's mental state?
To give you more a sense of these, here's another one. Ann mixed lasagna in a blue dish. After she leaves, Ian takes the lasagna out of the blue dish and puts the spaghetti in the dish. And, again, we can ask, what's in the dish? What does Ann think is in it?
OK. So people are reading that. This implements, basically, a false belief task and belief trials. And the control task-- the false photograph task-- is the control trial. And while they're reading it, we are measuring their blood oxygenation at every point in their brain; and, again, using the assumption that blood oxygenation is a proxy for neural activity.
And what we're going to look for in a classic contrast analysis, that you've already heard about this week, is brain regions where the average blood oxygenation when people are reading stories about mental state is over and above the oxygenation when they're reading their control story.
If you do this, you find that there are a group of five cortical areas that are recruited more when the stories involve beliefs and mental states, incredibly reliably. So that's the discovery that we made, and many other labs made, 15 years ago, which is now the kind of basic tool we use as the foundation on which we build all of our research.
And so now we take the average activity when people are reading about stories about beliefs and photos. And we get these five cortical areas very reliably-- right temporoparietal junction, its homologue on the left; left temporoparietal junction; the front of the brain, back of the brain; and then two brain regions along the midline, at least regions in the prefrontal cortex and regions in posterior cingulum.
To give you a sense of what activity looks like in this half, in one of these regions-- so this is the right temporoparietal junction-- this is the format in which I will show a lot of data over the course of this talk. So the x-axis here is time. Zero is when we put the stimulus on the screen.
So you start to read the story. The y-axis is the response in this brain region relative to rest. And so the first thing that you can see is that after we put the story on the screen, in this brain region, if the story has a belief in it, blood oxygenation ramps up pretty quickly and stays up for the course of the story.
I should say, these stories are 12 to 14 seconds long. And so this activity is being driven by reading the story, not by answering the question. This activity starts when we put the story on the screen, not when we later on ask them to solve a problem. I can also answer questions about that.
OK, this is if the stories involve no mental states, but still involve photographs and all the other cognitive activities. And, traditionally, what we find is a pattern like this, where responses to stories without mental states are approximately the same as rest, as if we put nothing on the screen, where stories that do involve mental states have this big robust response.
OK. We can see this in pretty much every individual subject we test. So this is just three individual subjects that we ran in the lab. You can see there's some variability in the size and position of this activation. We're interested in knowing whether that variability means something.
Again, I'm happy to talk about that. But the key strategy underlying all the research that we do about theory of mind in the lab, basically, is we use this contrast to find this region in each individual subject. So we're studying a functionally identified region that's the same function across subjects, not the same stereotactic position.
And then inside that region, or those voxels, we can then ask about other more detailed questions. So we can extract the response in those voxels to study anything else of interest. This procedure takes us about 10 minutes of scan time and works in about 90% of individuals. So if any of you ever want to do an fMRI study about theory of mind, you're welcome to use our localizer procedure. It will work, I promise.
It's being used all over the world and, I think, now 15 languages. And it's freely available on our website. It's been translated into many languages. You can present it visually or auditorily. We've done it in sign language. You can do it pretty much any way you want, and you will get this group of brain regions. And that gives us the capacity to target them for subsequent analysis and more detailed experiments. Yeah.
AUDIENCE: Does the localizer have a [? tradition ?] which is about people, but not about belief?
REBECCA SAXE: That is a great question. The localizer does not. But I'm about to show you data showing that it is about belief and not about people. But that is a great question. That was the first thing we wondered. Yeah.
AUDIENCE: And what about the connectivity among these regions. Did you do DTI and then look at that?
REBECCA SAXE: Yes, so connectivity is a great question also. The set of regions that I showed you in the localizer-- so that group of brain regions-- show strong functional connectivity with one another as measured by resting-state fluctuations in the BOLD signal.
So if you ask, when people are at rest, what brain regions are correlated, these brain regions are super-correlated with each other, which is not very informative, right. We know they're functionally related. They're correlated with each other. But that doesn't tell us the things we want to know about information flow, how information gets in, which order it proceeds in through the system.
And so, from that kind of data, we don't know. From DTI, it's very hard to say, because these brain regions are really far apart from one another. And so what we do know from DTI is that, for example, this region, the right temporoparietal junction, seems to have a distinct signature in DTI from neighboring parts of the inferior parietal cortex.
So you can use just DTI to tell you where this is going to be. So that tells you this region, as opposed to its neighbors, has a distinct DTI signature. But that hasn't yet told us, again, how does information flow between these regions.
And so one project that we're interested in my lab right now is trying to see, with all the limits of fMRI, with its limited temporal resolution and so on, can we try to get more information about information flow through these regions with techniques like [? Rangel ?] causal modeling or dynamic causal modeling. So we're doing that.
But if you really want to know about connectivity, I think you need to go beyond fMRI, because fMRI is not the ideal tool for studying connectivity, nor MRI nor DTI. So I'm interested in that. But that's pushing a little beyond where we are yet. Other questions. Yeah.
AUDIENCE: So what happens if a three-year-old or a subject that has been affected by one of these conditions that--
REBECCA SAXE: That's a great question. Ooh, and I don't have those data in this talk. I will try to point out, as I go along, things that we have found are different, either in young children or in people with autism. One thing to say, though, is that we don't have, and almost nobody has, functional data in three-year-olds.
We have tried to explain to a three-year-old what we mean by "lie still in the scanner." And we have never succeeded in explaining that. So although we have attempted on a number of occasions to scan three-year-olds, we have never collected any data from three-year-olds.
We have successfully scanned a few four-year-olds, but not very many. It's really hard to explain it to a four-year-old. The youngest age at which you can pretty reliably collect the data you intended to collect, like, bring a kid in and get maybe 50% of them through the scan session, is five. And so if you take the false belief task as the only measure of theory-of-mind development, you would say what's the point in studying five-year-olds and up-- all of theory-of-mind development is finished by then.
I mean, if you think about it, it's totally obvious that a five-year-old and an adult have really different theories of mind. It's just that we've been overly focused on false belief tasks as the only litmus that made us think that five-year-olds were fully developed. And so one of the big projects we've done in my lab is develop a battery of tests that measure more complicated aspects of theory of mind, show development between age 5 and 12, and then scan those kids and show corresponding development in these brain regions between 5 and 12.
AUDIENCE: And another question is, I don't know if you have specific ideas of what exactly happens [? or if it ?] [? appears as ?] activation in terms of what region it goes on as the kids or the subjects are trying to decide what the [? right answer to the ?] [? questions. ?] But, for example, there's this other set of areas, there's the default networks, in which you have this [? perspective ?] and [? retrospective ?] thinking. Do they overlap at all? Do you think that sort of--
REBECCA SAXE: So the question is, how do these brain regions overlap with the default mode network? And that is a slightly subtle question, because the default mode network-- so there's a gross pattern, which is to say, and you can get it in a number of different ways, and you get slightly different answers depending on how you define it.
So, as some of you may know, a very big discovery about 15 years ago in MRI, made in St. Louis, was that people who had been doing all kinds of tasks in MRI-- attention tasks, visual perception tasks, tactile tasks-- when you turned the contrast around-- so instead of asking, in my task of interest, what regions are more activated, you ask, in my task of interest, which regions are less activated, compared to in a controlled resting state.
So I ask you to do anything-- remember something, see something, make a judgment, attend to something. And then I ask what brain images show less activity while you're doing that, compared to while you're doing nothing at all, just lying resting in the scanner. And that produced this characteristic pattern of brain regions, including regions along the midline, [? ambilateral ?] regions, and inferior parietal cortex, they'll look a lot like the pattern that I showed you here.
Since then, most people have become interested in studying a group of brain regions, again, that look very similar, but defined by their correlations at rest. So, again, if you look at rest, you take a midline region, you ask what other regions are correlated with it at rest, you then get, again, what looks like a very similar region group of regions-- frontal and back, forward and back, in the midline, and left and right [? temporoparietal ?] regions.
And those two ways of defining the default mode network have been taken as equivalent. And that whole thing is called the default mode network. Now, inside that, and this is where it gets a little confusing, there's multiple different parallel networks that have different functional profiles, but that all have these front and back, midline components, and lateral components.
One of them is deactivated in all tasks relative to rest, and nearby is another one that's activated in social tasks relative to rest, but not in nonsocial tasks. So these two ways of defining them-- sometimes you'll get exactly these regions; sometimes you will get neighboring regions and a parallel profile that are distinct from those regions. And so some people have said the default-mode network is just these brain regions-- the theory-of-mind brain regions-- and that's true if you defined it in the resting state [? stats. ?]
And it's in virtue of these data that people have said there's a link between what people do at rest, when given no further instructions, and theory of mind or social cognition, because of these brain regions.
But as I've shown you, so just by definition, these are not default-mode brain regions in the original sense, because they're not deactivated by tasks relative to rest. They're activated by tasks relative to rest, but only social tasks. It gets very detailed, so I should probably stop there.
OK. All right. Is there any other questions about the basics. OK, you can keep coming back to all of this. This is the basic strategy. In all of the rest of the talk, I'm going to use this contrast of stories about beliefs versus photographs in individual subjects to find regions involved in thinking about other minds, and I'm going to then ask what it would do in more detailed tasks.
OK. And the first thing that we want to know is whether there's any evidence that this is really about mental states, as opposed to other information; for example, other information about people, because the only thing I've conducted now is people and mental states, to no people with no mental states. And so I'm going to show you some of the evidence we've gathered, that this pattern is a pretty specific signature of theory of mind from two experiments. This is two out of a very large number, so I can keep talking basically as long as you want on the evidence that this is specific. But if possible, I'd like to get beyond that question to ask, OK, so it is about theory of mind, like, what is it doing in some more detail.
OK. One of many experiments that we've done, asking whether or not this is about mental states, to do this experiment and to wake you guys up, I'm going to do this in a participatory manner. And so you guys are going to now participate in a version of this experiment. I'm going to split you in half, so down the middle.
If you're on my right side, from the middle, so your left is the middle, then when I read each story, I want you to use your hands to indicate a property of the story, which is, how much physical pain is the main character experiencing, how much pain of the body. The more pain the character is in, the higher your hands. Is that clear?
OK. You guys-- each time I read a story, I want you to judge how much is that story making you think about the thoughts of the character-- hopes, beliefs, desires, suspicions, doubts, thoughts. So the more the story is making you think about thoughts, the higher your hand. Is that clear?
OK, we're going to do a bunch of stories. Here's the first one. Larry's going to start his first day of a new job. It starts very early, so he was extremely tired. He made himself some fresh juice and took a big drink. The juice was cool. And Larry felt the tang of it on the soft tissue inside his mouth. Go. Everybody voting? OK.
Oscar was doing the dishes after dinner. He was talking with his friends while his hands were in the soapy water. Then his hand hit a sharp knife. The knife cut deep into the skin between his finger and the cut burned in the dirty water. Everybody voting. OK.
Maggie is in love with a man and wishes that he would reciprocate her feelings. One day this man sends a text to Maggie that he meant to send to someone else, saying he does not want to see her again. Maggie gets the text and starts crying. Max, you're not voting.
AUDIENCE: I am.
REBECCA SAXE: Doesn't make you think about dodging it-- fair enough. OK.
So these 144 stories are characterized on 33 different dimensions, different aspects of the people in them. So we can ask you characteristics of the people and characteristics of the story, like did they deserve what happened to them. Are they similar to you. How old are they? Are they male or female? And what the stories make you think about. Do you think about, how much pain are they in, how much emotional suffering are they experiencing?
We also asked general properties about the stories, like, how engaging are they. And syntactic properties-- how many words do they have, how many sentences, how many words before the main verb, et cetera. So we have these 144 stories, characterized by many different dimensions, characterized outside of the scanner.
So the kind of thing that you guys just did-- we asked about 50 participants for each story, each of these different questions. And so now we know for 144 different stories, there are ratings on 33 different dimensions. And the question we're going to ask is, can we predict variability in the neural response of a separate group of subjects across items by the item's rating on one of these different dimensions. Does that analysis strategy make sense?
OK. So, for example, for the story that the right-hand side of the room rated, the question, how much physical pain is the main character experiencing? Now, I'm going to show you, for each story, how much physical pain the character was said to be in, and then for the same story, the response in the right TPJ in an independent group of subjects.
OK. So here's those data. And you can see that there's a subset of the stories like the one about Oscar that had rated as the character experiencing a lot of physical pain. So each dot is a story. But stories that are described as having more physical pain do not get more activity in the right TPJ. Is that clear? You get this analysis strategy?
We can also use this as a [? regressor ?] in a full-brain analysis and actually find that brain regions that are involved in pain, like secondary sensory cortex and the middle cingulate cortex, respond most to these stories compared to the other stories. Yeah.
AUDIENCE: So in these stories is the rating of physical pain completely uncorrelated from ratings of other more mental states?
REBECCA SAXE: That's a good question. So, in these stories, physical pain and emotional suffering are somewhat positively correlated in the ratings, but anticorrelated in the brain. So two dimensions that are positively correlated in behavior are anti- to not correlated in the brain. Different brain regions are responding to these two dimensions even though the [? stimuli ?] [? happen to be ?] slightly confounded.
AUDIENCE: And beyond the pain aspect, though-- for instance, the knife story, we all rated as pretty high physical pain, but there's no real mental attributions there. I'm just wondering if that's kind of consistent across the stimuli in that either focuses pretty heavily on pain or focuses pretty heavily on--
REBECCA SAXE: Those two dimensions, one more mental and one more physical, are positively correlated across the stories. But they're not strongly positively correlated. They're weakly positively correlated. But it tended to be that stories that had more-- so I picked stories that are kind of more extremely divergent. They tended to be somewhat positively correlated with one another in the [? stimulus, ?] not anticorrelated.
So to some extent, finding that the neural regions that respond to them are completely disjoint, is interesting, especially because the stimuli don't have those [INAUDIBLE] as anticorrelated or [INAUDIBLE]. They're somewhat positively correlated, weakly positively correlated. That's a good question.
OK. One thing you might wonder about is it some stories are more engaging than others, so we had them rated on how vivid they are. The stories are all medium vivid. We're not that great at writing novels. But, again, there's reliable variability in how vivid the stories are rated and that more vivid stories don't elicit any more activation in this brain region. Again, there are brain regions that respond to this dimension, including the thalamus.
OK. So this is the one you guys answered. How much does it make you think about thoughts? And, of course, the reason I'm showing you this is because this is the only one that does predict variability across items in the right TPJ. And so one group of subjects' ratings of how much the story makes you think about thoughts, [INAUDIBLE] 20% of the variability in another group of subjects' brain activity while they're reading these stories.
This is both a highly positive correlation, and also suggests that there's 80% of the variance still to go, which I think is great-- lots to learn. OK, so that's one piece of evidence that of all the dimensions that these stories had, the one that predicts overall activity in the right [? TPJ ?] is thinking about that. So that's one way to look at it.
Another way we have looked at how specific this activity is is to ask not across stories-- so that whole analysis was looking across items. Now I'm going to look within stories-- so within items over time. So we have people reading stories where information is available at different periods of time.
For example, in this story, first I can tell you about a person's background. Your friend Erica lives in Los Angeles. One night she was in a bar. A fight broke out between two drunk men, and she was caught in the middle.
So this information has a bunch of background information on a person, where they come from, where they live, their characteristics, but not what they're thinking about the situation. Then we add what they're thinking. In this case, she has a strong dislike of violence. She believes conflict can be resolved without fists.
And then the story goes on. And later on, they ask you to tell us, how good or bad did she feel? The key idea in these scenarios is that we've put a bunch of information about the protagonist, not their mental state, first, to create a six second delay before we give you the information about the character's mental state. And then we can ask, how does that affect the timing of activation in this region?
And so this is the activity when the information about mental states is on the screen immediately. That's the activity when we delay it six seconds. And that's six seconds.
OK, so both across stories and in time within stories, we have a pretty precise control over when this region gets involved in a story, which is when the story describes or requires [INAUDIBLE] about mental states. We worked on that for a long time, showing the specificity of this region. But there's a bunch of things that are unsatisfying about that version of the question.
And the main one is that this is just asking, approximately when and approximately how much is a huge population of neurons engaged in a very general task? And in order to take some stories that describe mental states and some stories that don't, the stories are not minimal pairs. As you can see, they very on many dimensions.
We can try to get control over that. But this question of whether or not a story evokes theory of mind is actually not question we're interested in, right? We want to say given that a whole bunch of things are inferences about mental states, how does the brain compute the specific inference, the specific mental state, the specific judgment that follows from that mental state?
And so we want to go beyond saying the region is involved or not involved, to begin to ask, how is this region playing a role? What representation is being computed. And to do that, we're going to use, instead of overall univariate signal, we're going to start to try to use decoding.
So again, you guys have had a general introduction to MVPA. I'm just going to go back over it again briefly. See, in here the general idea of what we've done until now is we take stories that are presented to subjects. We average together all the stories that have mental states in them. And we look at the average recruitment of the whole population of neurons in one brain region, in this case, in their RTPJ, or in any other brain region.
But the assumption here is that each of these stories is driving into the region for a slightly different reason, right? Inside these stories are many different features of those mental states. So this story has features like-- so this is the one with Maggie getting the text message.
How did she get her information? She got it individually. She feels sad that she thinks he's having an affair. Is that a justified belief, given the evidence? It feels pretty justified.
How confident is she that she has the right conclusion? Probably pretty confident. Is she right about her conclusion? In this case, she's wrong about it. How does she feel about that? She feels bad. How strong of an emotion is she having? Pretty high, right?
All of these features are things that you could extract from that story, and that presumably are part of the computation you're doing when you read that story to represent the character's mental state. And those features can be different between different stories about mental states. That's sort of the whole point, right?
Theory of mind is not either thinking about mental states or not. It's figuring out a specific mental state, what its characteristics are, how reliable it is, and what its implications are. Does that distinction make sense to you guys, on why this one's more interesting? This audience is totally dead. It is 9:00 in the morning.
OK, so again, the idea here is that within a population of neurons, there might be sub-populations that are encoding different aspects of these mental states, different specific features. So when you switch between a story that has relatively high justification and relatively low justification-- just to pick one of these features-- you'll get different subsets of the neurons within this region responding to give you the information about where this story is on that dimension.
This is similar to probably some of the stuff Jim DiCarlo taught you about thinking about representing objects as positions in a perceptual space defined by dimensions of the objects. We can think of representing mental states as positions in a dimensional space where the dimensions are defined by features of mental states. But now those features are very abstract features.
So that's the same logic here. And again, just like Jim DiCarlo is going to try to decode which object it is by the population code over neurons in object regions, we're going to try to see, can we decode anything about the mental state you're thinking about by looking at the population code of the neurons that are representing that mental state?
Only we're not going to do it with neurons, because these are humans. And we're limited in our ability to cut holes in their head. And so instead we're going to do it with fMRI.
OK, so here's the idea. If different sub-populations of neurons are involved when the story has very different values along the key dimension, and if those sub-populations are somewhat organized spacially-- two big assumptions which need not be true-- then it will be possible to see distinct spatial patterns, even ones we're [? pooling ?] over hundreds of thousands of neurons using MRI, and look over the [INAUDIBLE] at spatial patterns in exactly where there is activation during a story to try to decode the features that are represented as that story.
And the idea is when stories share a feature, more similar sub-populations of neurons will be recruited than when the stories don't share that feature. And we'll be able to see those differences in the sub-populations that are recruited. Yeah?
AUDIENCE: Why do we constrain ourselves to this region at first, and not the direct [INAUDIBLE] on the whole brain?
REBECCA SAXE: That's a good question. So we are going to target regions initially that we think are involved in making these computations, partly just for power. But we also always are going to do whole brain analysis.
Whole brain analyses and [? these ?] analyses are pretty underpowered. And I'll show you some of the constraints that this kind of situation puts on us that makes us even more underpowered. But just to give you a sense of it, in order to get hundreds of thousands of trials in a visual object perception path, you can present one object every two seconds.
You know, I haven't actually done this calculation. But you can get a lot of objects in two hours if you do it that way, or in 10 hours, right? In our case, it takes up to about 30 seconds to present a single stimulus. And so we are 15 times slower at least in eliciting the response, meaning at least 15 times fewer stimuli.
Then you have to wonder also, can you elicit the full response if you do 1,000 of them? And so we are typically working with, instead of thousands of trials with dozens of trials. So that's one reason that we have limited power.
Of course, we could put people in the scanner for 48 hours and have them read as many stories as we wanted. Some day, probably we should do that. But for the moment, all of our analyses are relatively low powered. And so it helps to constrain our hypothesis. Good question. Anything else?
OK, so here's the general logic. We're going to take stories that share some feature. Another fundamental assumption which I'll go to as I go along is that we assume that you can't read the same story twice.
I don't know if you guys have ever read a novel twice. But the subjective impression I have is that second time you read a novel you are not reading the same novel that you read the first time. Because you now know a whole bunch of things about what's coming. You're inputting it differently. Certainly the third time you read it, you're not reading the same one.
And that's much more exaggerated if it's a single paragraph, right? Think about reading one paragraph three times in a row. You get different information the first time and the second time. By the third time, you're basically saturated. You can't read the same paragraph for the third time.
And so unlike visual object perception, where we can give you the same object and elicit the same perceptual operation, for example, 20 times, we aren't going to be able to present you the same stimulus. And we assume we can't even present it twice. That might be over-conservative. But we're going to present every stimulus only once. And each stimulus is unique, with a unique constellation of features.
So the strategy that we're going to use is gather together groups of stories that, even though they're diverse on many features, they have one feature in common. And we'll try to learn the population response to that feature by using a [INAUDIBLE] of sets of stories that differ among that feature. And then when we have a new story, which is again a unique story, what we will try to you to do is use the pattern of response to that new story to classify which featured it has on this one dimension that we've now learned to decode from the existing stories.
I'll work through that in more detail as we go on. But again, the idea is going to be for this new story, which feature value does it have for the future we're learning to decode in the experiment? I'm going to go back and forth between two different measures, just because it's not very interesting. [? Because ?] we sometimes use one and sometimes use the other.
One is we're going to just ask whether the difference in correlation between the template and the new story when they share a feature-- whether that correlation is higher than the correlation when they don't share that feature. Sometimes I'll use classification accuracy. These are essentially the same measure, just different ways of construing.
OK, so to figure out whether this would work at all as an abstracted domain of theory of mind, in which-- as an abstract dimension, those dimensions of mental states. And when you're [? pooling ?] over as many neurons as you have to do fMRIs, we took a test case of a mental state in that distinction that we knew was extremely relevant to people. It's a mental state distinction people use a lot in life and in our tasks.
And so we know this is an important distinction. If anything is going to be encoded, maybe this will be. And this is the distinction between accidental and intentional [? harm. ?]
OK, so to give sense this experiment, I'm going have you participate in it. I'm going to read you a short stimulus. And I want you now all to make the same judgment.
You guys are all to judge this character. How much moral blame does the character deserve for the action that they do? So at the end, using your hand, where higher is more blame, how much moral blame does the character deserve for their actions? Clear?
OK, your family is over for dinner. You're taking cooking classes And want to show off your culinary skills. For one of the dishes, adding peanuts will really bring out the flavor. You grind up some peanuts, add them to the dish, and serve everyone.
You cousin, one of your dinner guests, is severely allergic to peanuts. You had absolutely no idea about your cousin's allergy when you added the peanuts. How much blame do you deserve for putting peanuts in the dish?
OK, what about in this version? You definitely knew about your cousin's allergy when you added the peanuts to the dish. OK, good. OK, just as long as pretty clear-- so here the idea is some subjects will read one of these two versions for each scenario. We have 75 different scenarios in which you either on purpose or accidentally harm somebody.
What's satisfying about this is that the difference between the conditions is pretty minimal. It's just a couple of words in the last sentence of each story, right? So we set up the whole story. And in the very last sentence, we just tell you a tiny bit of information about whether you knew or didn't know what you were doing when you did this action.
So it makes a huge difference moral judgments. Like you guys, people say that if you had absolutely no idea about the allergies you deserve a lot less blame than if you knew what you were doing when you put the peanuts in the dish. Yeah?
AUDIENCE: Are the [? suggestive ?] images included in the [INAUDIBLE]?
REBECCA SAXE: No, that's for your entertainment and mine.
AUDIENCE: Is the signal different if you use "you" versus someone's name?
REBECCA SAXE: That's a great question. We've gone back and forth doing it both ways. It is not for harm. So for these kinds of stories about harm, it doesn't matter whether you use "you" or somebody's name-- so the signal, like the fMRI signal, or the moral judgment.
So neither fMRIs nor moral judgments are affected by third person versus second person stimuli for harm. There are some kinds of moral scenarios where people make stronger behavioral judgments if you put them in the second person, which is why sometimes we did put them in the second person. For the RTPJ response, there is at least an equal if not a greater response when the protagonist is you.
And then there was a whole paper arguing that we should take that as evidence that you're using the right TPJ to attribute mental states to yourself. And I think that's a really odd construal of what's going on when you read a story in the second person. I think even though it's in the second person, the mental state of knowing about your cousin's allergy is not actually something you're attributing to yourself. You're attributing it to the [? funny ?] protagonist of a story that's in the second person.
It's actually technically an experimental literary technique to make you feel like you're engaged in the story. But it's certainly not like what you when you actually think about your own mental states. Think back to a time when you harmed somebody. And what were you thinking?
I think the argument that this invokes that process is a stretch. Although you could takes this as evidence that theory of mind is [? useful ?] [INAUDIBLE] own mental states, I don't. I think this protagonist is some kind of third person protagonist in a weird way. Other questions?
OK, so the logic of this experiment is now we learned for a bunch of scenarios the pattern of activity at this time-- so while you're reading this sentence-- if the sentence describes not knowing what you're going to do, and for a bunch of other scenarios in each person, the pattern for this sentence when the sentence describes knowing what you're going to do. And now we're going to try to ask for a new scenario. Can we decode the value of this feature, whether the person harms intentionally or unintentionally?
And to give you a sense of how [? distant ?] that generalization is, I'm just going to show you one other example scenario. So this is another scenario from the same experiment. Again, you're going to make moral judgements.
You're in an English class prepping for the AP test at the end of the year. Your teacher passes out a sample essay you are to discuss openly and honestly. You suggest the essay must have been written by a third grader.
The student who wrote the essay is in the class listening to your critique. The essay was typed, so you didn't realize who had written it. Moral blame? Come on, guys. It was handwritten. And you realized right away who had written it-- a little more, right? That's kind of an ass hole thing to do.
OK, So the idea here is this distinction between knowing who had written the essay and not knowing who had written the essay is different in a lot of details. It's different in where you got your information, what kind of information you have.
It's different in the kind of harm you committed, right? There's a lot of features of these stories that are different. But one feature that's the same from the previous story, which is that if you get this sentence, it was an accident. If you get this sentence, it was intentional.
And so the idea is we can then ask the pattern of activity here and in a group of stories like this, is it more similar to the pattern elicited when you didn't know about the peanut allergy? And the pattern here, is it more similar to the pattern when you did? Yeah?
AUDIENCE: What is the question you ask the subjects?
REBECCA SAXE: How much moral blame do you deserve [INAUDIBLE]?
AUDIENCE: [INAUDIBLE], OK.
REBECCA SAXE: Just the same thing I'm asking you guys, on a scale from one to four, because that's how many [INAUDIBLE] we have.
AUDIENCE: Do people find that there's ambiguity in [INAUDIBLE]?
REBECCA SAXE: Yes. So these stories are designed to be different, to elicit different amounts of blame, so that we'll get people thinking about it, and to have some variability in the amount of blame, across items and across people. But the key thing is this distinction per item still makes a big difference, even though the items themselves also have a lot of variability. Yeah?
AUDIENCE: Before you give them any stories, or before you ask for their judgements, do you give them some examples of the stories, and how [INAUDIBLE]?
REBECCA SAXE: In these experiments, we haven't anymore. But you're going to be in the scanner for hours reading these stories. And so you probably habituate pretty quickly.
One of the problems that I have reading these stories that eventually, people are popping them off so fast because it starts to get ridiculous every time you kill somebody. It's like, oh, come on. You're killing somebody again.
But it was even worse the time we did a full set of experiments doing moral judgment of other kinds of moral harm, and in particular, incest. We did accidental and intentional incest. And we had to have 25 stories in which you accidentally slept with your sibling or parent. And that got ridiculous. There's only so many ways to accidentally sleep with your parents.
Good. So the question we're going to ask here is, when two stories share the feature of being accidental, do they elicit a more similar pattern of activation, because this is again, independent stories that share that feature, compared to independent stories that differ along all the same dimensions and differ along that feature. So we're going to compare the correlation between the stories that share the feature to stories that differ and don't share that feature. And in the first experiment in which we tested this, what we found is that there was a small but significant difference between the correlations when they shared the feature and when they didn't share the feature.
And that suggests there is some information in the population response. But it's a small effect. It's significant, like less than [? a few, ?] less than 0.05. But it's small. One reason it might be small is exactly the thing I'm telling you, which is that the feature, accidental versus intentional is not the only feature that's included in these brain regions. And these stories are very heterogeneous, so we're picking up a one sub-population difference that's superimposed on many others.
But another reason it could be small is it's not true. And you worry about that when you do experiments. So one of the first things I wanted to do when we saw this data was replicate it. It feels like you've got a small but significant effect. You should replicate it in independent data. We're about to run a new experiment with this thing, but we realized we'd actually already run this experiment many times before. And we had in the lab, data from a previous experiment that had intended to manipulate accidental versus intentional harm, although in structurally different ways, previous data that were just sitting in our computers.
And so instead of running a new experiment, we decided to go back and analyze this old data. So to give you a sense of how those stimuli go, these stimuli have a slightly different structure and manipulate intentional versus accidental mental states in a slightly different way.
AUDIENCE: [INAUDIBLE]?
REBECCA SAXE: Yeah.
AUDIENCE: In a previous experiment, if you showed multiple stories, so one of them is [INAUDIBLE]. And so we keep doing this, the subject might actually start predict the kind of experiments. So would that actually affect results?
REBECCA SAXE: You can't predict whether a particular action is going to be accidental or intentional, because it's 50-50, and it's counter-balanced. So for any given stimulus that you're getting, it's going to be either accidental or intentional. What the content definitely tells you is that that's one of the things we're manipulating. So every time you read a story, you should expect the one thing that's going to happen is you're going to find out whether it was accidental or intentional.
And in that first experiment, you know exactly when you're going to find that out. You always find it out at the end of story. So for every story you read for the whole experiment, you're going to read about an action, and that you'll find out whether it was accidental or intentional. And I'm sure that context tells you to focus on that dimension, which is fine. I wanted people focusing on that dimension. But as we said, the story's otherwise varied in any other dimensions that are morally relevant, like how bad the harm was.
And we did that to keep you going. We didn't want you to just say, this one's accidental. It's not that [INAUDIBLE]. It is bad, because moral judgments are more complicated than that. It depends how bad the harm is. It depends whether it was plausible to not know the thing you didn't know. And we wanted that variability so that you had to keep making the judgment instead of just detecting which condition you're in, making a binary choice. Yeah?
AUDIENCE: Because we're expected to do the [INAUDIBLE] for the classification.
REBECCA SAXE: So the classification is binary. This classification that I just showed you is binary.
AUDIENCE: It's simply a linear?
REBECCA SAXE: Yes, [INAUDIBLE]. Linear SVM or [INAUDIBLE], which is the same thing. This is a linear binary classification [INAUDIBLE].
AUDIENCE: So the previous two stories, both [INAUDIBLE] other people, like class and the family, does that have an effect on people's judgement, like if there are other people [INAUDIBLE] on the [INAUDIBLE]?
REBECCA SAXE: That is an interesting question. So there's the person doing the harm, the person being harmed. Those people are always in the scenario. They're asking whether there being other surrounding environmental people who might witness it, whether that affects your moral judgment. That seems like the kind of thing that should affect your moral judgment. It's not a feature we've ever tried to decode, and it varies across the stories like many other features.
This is exactly the idea, right? These stories are heterogeneous on many relevant features. And that's important to drive realistic moral judgement, where you keep taking all those features into account, but also, sets a bound on how good our decoding can be between these stories. Great. Yeah?
AUDIENCE: Do you guys look at how people's beliefs about intention changed between the first story and on the extra [? tidback, ?] and whether or not that difference is more relevant to what's coded, as opposed to, say, what the final belief is? [INAUDIBLE] read the first part of the story, and they have a particular bias associated with whether it's intentional or not, and whether it's error in their representation of the moral attributions as opposed [INAUDIBLE]?
REBECCA SAXE: We have the idea that this is an error signal, but we got that idea long after we had done the experiments. So we think that when you read these stories, you're biased to think the harm was intentional, because most of the time, when you do stuff like that, you knew what you were doing. We think accidental harms are probably less likely in these kinds of scenarios than intentional harm. And there are tiny hints in our data. By analogy, the predictive coding signals that have been seen in other places, that you can construe some of the response we're seeing as an error signal when you get the accidental information.
But it's a very indirect argument. We've never tested it directly. Yeah?
AUDIENCE: So this class of sub-populations makes me think of when I use fMRI in a patient in visual cortex. So if I read 100 stories where someone dies by the end, do people rate them lower? And does that correspond to adaptation [INAUDIBLE]?
REBECCA SAXE: Yeah. So we have tried to do fMRI adaptation stuff investment. There's a whole interesting conversation about how that went. As far as we can tell, and somewhat surprisingly, people can keep this up for the whole experiment. So this is 70 items. It takes about an hour and half. And people are still making robust moral judgments. They're still distinguishing between accidental, intentional, and we can still decode from their neural signature, which didn't have to be true. I think not neurally-- I don't think of this as neural adaptation.
I think just psychologically, whatever that distinction might mean. It does feel like the 75th time somebody accidentally pops off their cousin, it feels a little bit different. But we are able actually to engage with the stimulus and make moral judgments. And we think that maybe building all this heterogeneity into the stimuli helps that, because each story is a slightly different situation. Yeah?
AUDIENCE: Subjects outside [INAUDIBLE] moral judgements before you give the punchline? [INAUDIBLE]?
REBECCA SAXE: Gosh, I think we actually have done that, but I can't remember. What I will show you is the relationship between this and behavioral data [INAUDIBLE]. If I get there, I'll show you how this relates to behavioral data with all the information that we provide.
AUDIENCE: Because you could imagine there being quite a lot of variability?
REBECCA SAXE: Before, yeah. And in fact, as I say, I think that there is that variability, and we have it there on purpose. But gosh, we may have, in fact, measured it without the mental state of information. I just can't remember. We've certainly measured it with all the information. But good question. Yeah?
AUDIENCE: So wouldn't it be interesting to look at maybe if the character wanted to do something deliberately, but then decided not to? And then it happened [INAUDIBLE] anyway.
REBECCA SAXE: Yeah, [INAUDIBLE] scenarios. So there's a whole industry, and philosophy, and experimental philosophy of studying how we make moral judgments of cases where the causal process from the intention to the outcome is somewhat indirect or involves luck. So these are cases like there's this famous puzzle in moral philosophy. Well, in a certain very small subset of moral philosophy. If somebody says, I really want to kill my uncle. In fact, I'm going to go right now and kill my uncle, because he's making me crazy, and decides [INAUDIBLE] I'm going to inherit lots of money, I'm just going to hop in my car right now, and we'll kill my uncle.
And they're so busy thinking about killing their uncle that when they hop in their car and drive out of their driveway, they totally don't notice a pedestrian who's running past them. And they went straight over the pedestrian before they've noticed anything. And they hop out of their cars terrified, I've killed somebody, and come around to the back of the car. And it's their uncle. Did they kill their uncle intentionally or not? Should they be punished like you would have a murderer?
This is actually debated in murder cases. Was that murder? Was it not murder? I'm not worried about those weird puzzle cases at the moment. We're talking about cases where if you intended it, you did it, because you intended it via impending it.
You didn't intend it, you usually didn't do it. Although, again, in the incest case, it turns out that for incest, it matters how you-- if you attempt to commit incest, but fail, it matters how you fail to whether or not it was bad. So according to the moral judgments that we did, if you think you're sleeping with your sibling, and you sleep with somebody, and you only find out later that they're not your sibling, that is judged worse than if you can decide to sleep with your sibling, and you're interrupted by a fire alarm, and don't get to sleep with them.
So I think that's the right way around. But it's worse if you actually only find out later that it wasn't your sibling than if you're prevented from sleeping with them by subexternal force. I think that's the way it goes, and not the other way. But in any case, for harm, it doesn't matter. If you go to kill somebody with a gun, and you think it's a real gun, and you fire a bullet, and it turns out it's not loaded, or it's a fake gun. And so you don't kill them, or you fire the bullet. The bullet goes, but it misses them, it doesn't matter.
It doesn't matter how you fail. That's an attempted murder, and it's judged badly. It doesn't matter where in the process, whether it was a wrongful [INAUDIBLE] or an obstacle [INAUDIBLE] that prevented it. That was a weird diversion. Back to moral judgement of harm. So here is some old data that we had in the lab and which we had people making judgment of accidental, intentional harm.
But the scenarios were somewhat different. So here's the scenario. You're going to make the moral judgments again. Ready? Here we go. So Grace and her friend are taking a tour of a chemical plant. When Grace goes to the coffee machine to pour coffee, her friend asks for sugar. There's [INAUDIBLE] container by the coffee. The way [INAUDIBLE] is a toxic substance left behind by a scientist and deadly when ingested.
The container is labeled sugar, so Grace thinks the powder is sugar. She puts the powder in her coffee and gives it to her friend who drinks it and dies. How much more blame does she deserve? Still not that much. The container is labeled toxic, so Grace believes the powder is a toxic poison. She puts it in her friend's coffee. And her friend drinks the coffee, and dies. There are at least two or three scenarios.
You guys are maintaining your assertion that it's worse if you intended. So in this case, we manipulated the beliefs in the middle of the story and [? further ?] at the end, and we manipulated it slightly differently. And the first experiment I told you about, you said either you or you didn't know, so that's a positive versus negative. Here, they're both positive. And the syntax of the belief statement, that's the same. It's only the content that changes.
So again, a bunch of structural differences about how we implemented this distinction between accidental and the intentional harm. And when we went back to this data, we found that in data we [INAUDIBLE] a long time, ago this signature, more similar when the story shared that feature was also present in that old data. I can't exactly tell you why I'm finding this in the old data, rather than lending a new experiment or finding a new experiment [? felt like ?] better evidence. It just feels like magic.
This data, it's been in the lab for five years. And the signature was hiding in them, and all we had to do was go look for them. Anyway, it's very exciting to me. So here's the overall idea. They're more similar with they share the feature, less similar when they don't share the feature. And now we can ask, is that related to behavioral judgments? So as I told you, there's variability, both across items and across subjects in how much difference the intention makes to that story.
I think you guys got that sense that the intention in the school room case matters a little bit less than the intention in the toxic poison case. And then that is true of our subjects, that there's variability across items and variability across people. And we can then ask is the amount of difference-- so what we're going to now say, this is behavioral data, along the x-axis. And this is going to be subject wise. So on the x-axis, we're going to put people who say intentional harm are much worse than accidental harm at this end of the axis.
If they say, accidental harms are almost as bad as intentional harms, they're going to be at this end of the axis. Nobody goes the other way. There's nobody who says accidental harms are worse than intentional harms. So this difference is always positive. Intentional's always worse than accidental. But people disagree in how much worse. On the y-axis, we're going to vote for the same people how big the difference is between this pairs of stories that are matched on the feature, accident versus intentional, and the pairs of stories that are not matched.
And so we find it that if people's RTPJ contains more information about intentional versus accidental, so we could decode that distinction that better, those people say that stories involving intentional harms are worse than intentional harms. People whose RTPJ contains less information say that that difference is smaller. Does that make sense? There's two other bits of information that aren't on this slide. One is that this is different in autism. I was asked earlier what happens in autism.
Adults with autism don't distinguish intentional and accidental harms as much as typical people do. So behaviorally, they're in the center of the spectrum. They distinguish it, but less. And this patterned information is not [INAUDIBLE] autism. So that's one piece of information. The second thing is we've done the TMS study in which we TMS the right TPJ while you're making moral judgments. And we can decrease the difference between accidental and intentional harms behaviorally by TMS-ing the right TPJ compared to TMS-ing a central region. Yeah?
AUDIENCE: Typical question about getting behavioral data from autistic people. Can you use the scale in a similar way as adults for your tasks?
REBECCA SAXE: They are adults.
AUDIENCE: I mean for tpyically developed individuals.
REBECCA SAXE: Yes. So the question was when you have differences in the moral judgements of people with autism, is it that they're using the moral judgment scale differently? Or is it that they're valuing intention differently? And the way we can tell the difference is that in these scenarios, we always have these controlled cases where you didn't intend any harm, and you didn't cause any harm. And so we can anchor our scale by how bad you said neutral actions were.
You didn't intend harm, you didn't cause harm, and how bad you said intentional harms were. You intended harm, and you caused harm. And people with autism are similar to typically developing people in both of those two judgements. And it's only in the intermediate cases where you have to weigh the intention against the outcome in accidents [? and ?] attempts, that both autism and PMS show differences between groups.
AUDIENCE: You mentioned when you TMS people, you get this shrinkage between intentional, accidental. Is that shrinking more towards the intentional side or towards the accidental side [INAUDIBLE]?
REBECCA SAXE: It makes accidental harms worse and attempted harms less bad, failed attempt at [? harms? ?] It doesn't affect judgments of intentional. Does that make sense? So again, when there's two factors, what you intended and what you did, it makes you weigh what you [INAUDIBLE] less relative to what actually happened. If those two factors agree, it doesn't change your judgment. It changes the balance [? between two ?] factors.
AUDIENCE: Could you give some back story behind the intentional harm? Do you find that you get [? to stand ?] more with the incidental?
REBECCA SAXE: So there's all kinds of ways to mitigate simply worthiness of an intentional harm. So here, the only mitigating factor we've ever offered in these experiments is that you didn't know what you were doing. But there are lots of other ways to mitigate an intentional harm. So you could act out of compassion. You could tell somebody because you don't want them to suffer, you could act out of self defense, or out of other defense, or out of national duty.
There's many ways that a harm could be mitigated. And also, accidental harms can be more or less forgivable, depending on whether there was negligence involved, for example. So you may not have known you were going to harm somebody. But you should have known. So in these experiments that we did, the only mitigating factor we manipulated was beliefs. And we made the beliefs as justified as possible. I'm going to show you a later experiment in which we changed that.
We made some of the beliefs mitigate the harm justified, and other ones not justified. And I'll show you that that information is also coded. We haven't yet ever tried the other non-belief ways of mitigating harm, like self defense, natural glory, and compassion. In some ways, those are mental states. Those definitely affect moral judgements, so people who commit harm out of compassion, or out of use, or out of drug addiction.
All of those things mitigate moral judgments. They also mitigate punishment for our [INAUDIBLE]. And my assumption is that at least some of those mitigate the [? lame, ?] but not via the right TPJ's. So the mental image I have of how moral judgements proceed is that the right TPJ's computing one relevant source of information, which is what you thought and why you thought it. But that moral judgment depends on much more than that.
It also depends on how much harm you caused, and why you were causing that harm, and how you got yourself into that situation, and your causal analysis. So I don't think this is moral judgment. I think this is the computation of one input to moral judgment. And then we will find encoded in this region only the things that are about mental states. Not everything else that effects your moral judgment will be [INAUDIBLE] yet. Yeah?
AUDIENCE: About the coding, why is only this feature and not, for example, the regression [INAUDIBLE]? So you try to subtract others?
REBECCA SAXE: Because we don't even begin to know what any of the others are. So the first question is what is the future of mental states, and which stimuli have those features? And we are just at the very, very beginning of even trying to articulate, let alone, be able to quantify what the features are and where they are encoded. But so eventually, you want a theory of mental states, not a single dimension. You want a representational space [? sent to ?] all of the dimensions.
But we are not anywhere close to such a theory.
AUDIENCE: Within the linear, so a list of things you ask--
REBECCA SAXE: My guess, yep. And so what I'm going to do next, hopefully, is say, OK. That was one dimension. Let's begin to start to make a theory by at least coming up with a few other dimensions and testing them. That's step two. Not just one dimension, but maybe three or four dimensions. And now, where we are on the lab is, this stuff I'm showing you, I'm going to show you, I think, four dimensions, and we're going to show you where they're-- where we can decode those dimensions, instead of now being like, five, six, seven.
Now, let's stop, and at least for subsets of mental states, try to figure out how to find what the space is. But, for example, unlike vision, where the Dicarlo lab gets to benefit from a huge amount of history, people thinking about what other dimensions of objects, representations, what could they even possibly be, we are just making this up from scratch. So we could study the dimensions of our stimuli, but that would be very circular. We wrote our stimuli to get certain dimensions into them.
AUDIENCE: [INAUDIBLE].
REBECCA SAXE: Again, we could figure out what dimensions we put in our stimuli. And then to some extent, we're starting to do that. But the other thing we're trying to do is start to define bigger spaces with more multi-dimensional spaces, try to figure out what those spaces might be, try to come up with [INAUDIBLE] that would tell us what the dimensionality is. So we're starting to do that now. One of the things that's also hard in this is if anybody in here has good ideas to this.
So one is what are the hypothesized dimensions? We'd have alternative theories of alternative versions and have a [INAUDIBLE]. That's something we're working on. And I would love to talk about how you do that better. The other thing is how you do an unbiased stimulus construction? So if you want to study image perception, you search the web for images. If you want to study mental state perception, it's not that easy to search the web for mental states. And so thinking about how would you-- what does it even mean to say an unbiased sample of the mental states that you attribute?
What is that? I would like to know. Anyway, that was just one dimension. It's very limited, but it worked. That was exciting. That was the first thing that worked and made us think, OK. Again, it doesn't have to be that the neural populations of responded dimensions [INAUDIBLE] have any facial structure at all. I should say, by the way, everything I'm going to show you, there's no univariate cycle. So for every distinction I show you, the average response to the right TPJ is the same to both features. So the average response is the same to accidental, intentional harms.
It's the same to all the other features I'm going to show you. Or even if it isn't the same, the way we've done the pattern analysis drops the mean information. So everything I'm going to show you is times where what we're looking at is only this spatial distribution of activation, and never the overall [INAUDIBLE]. It didn't have to be that that was possible. These data give us confidence that it might be possible for at least one case. And so then we started saying, if it's true in one case, let's start thinking about what are the dimensions.
What's the feature states and mental states? And which of those might be represented, and where? So which features might be encoded? As you said, I gave you a list of the kinds of features we think that might exist. This is generating them off the top of our head. What things might you know about a person's mental state? One thing you might know are the things that we call epistemic that are related to the origins of the belief.
So that might include the source of your evidence, like how did you come to have that belief? Whether that evidence was good evidence or bad evidence, the justification of your beliefs, your own subjective confidence in your beliefs; how much you believe or don't believe, [? the thing you ?] included. Whether or not it's truth, so whether it matches the world or not, and probably, a bunch of other things. And the other thing is affected features. How you feel about it, so is it good or bad? How much arousal you have.
Is it very good or a little bit good? How motivationally relevant is it? Is it something that's very central to your goals or not very central to your goals? So this is a set of features or examples of features that we think are part of the representation you use, and thus, possibly could be represented somewhere within the system. And then the question's is that true, and how would we test that?
The other thing is as we've said, there's a whole bunch of brain regions involved in theory of mind. I have been talking about the right TPJ all along. But one hypothesis is maybe part of why there's so many regions is that different regions are involved in representing different aspects of mental states. And so maybe we would find different features in different regions that would help us to functionally pick apart what these different regions contribute to the computation, so starting with our [INAUDIBLE] features.
And the first thing we're going to look at is whether or not there's evidence [INAUDIBLE]. So after accidental versus intentional, it seems like the most important thing you might want to know about somebody else's beliefs is whether they have good or bad evidence for the thing they concluded. That's, for example, the thing that you cross examine somebody for in a court of law. You have good or bad evidence supporting your assertions. And so we're going to ask whether that distinction between good or bad reasons, or good and bad evidence, is another thing that's represented in these regions.
I'm going to give you a sense of these stimuli. And now, I want you guys to say, using your hand, does the character have good or bad evidence for their conclusion? So very good evidence, high up. [INAUDIBLE]. We're going to [INAUDIBLE]. So very good evidence, high up. Very bad evidence, low down.
Leanne [INAUDIBLE] always wore her bright, red hair in two braids. Dylan, a boy at school, constantly made fun of the braids. And so Leanne and her best friend, [? Deedee ?] tried to avoid him. One day, Leanne and [? Deedee ?] were in their classroom. The classroom was small and brightly lit. Nearby, Leanne saw Dylan make a face and point at her.
And she said, oh no. He's making fun of my braids again. Does she have good evidence or bad evidence? How about in this version? The classroom was large and crowded. Across the room, Dylan was pointing at something. Leanne said, he's making fun of my hair again.
Good. So you get the sense that we can manipulate for the same mental state whether the evidence was good or bad. In this case, now that we were doing this on purpose. So up until now, we had been analyzing data collectives for other reasons and experiments that we had to design for this reason. Now, we're doing it on purpose, so we've tried to specifically manipulate justification and try to match everything else.
And now, there's always three characters in the story. There's the main character. There's always a character, and there's the target of the belief. In this case, if it's good or bad evidence it's always visual evidence. So we're matching. You're always seeing something. But we change whether you see it well or you see it badly.
And it's always in this kind of structure. One thing is that we had behavioral ratings from independent leaders, like you guys, say whether there's good or bad evidence, and found that these stories differ reliably in how good or bad the evidence was. So the ones that we rated as being better evidence are rated as better evidence. And now, we're going to look at the response in the right TPJ.
At this point in the story, [INAUDIBLE] the sentence that describes the evidence. And again, ask, can we decode the value of just this feature, good or bad evidence? And in this experiment, in the right TPJ we could. Again, we want to replicate that in a different experiment in which we manipulate that distinction in a different way to make sure that it's not anything very specific about how we manipulated this distinction in these stimuli.
Again, we went back to old stimuli. Again, you're going to now make a good or bad justification judgment. So this is a different set of stimuli from a moral judgment experiment. Mitch is at home on his day off, giving his two year old son a bath. He fills the bath while his son stands by the tub. The phone rings in the next room.
Mitch tells his son to stay put by the tub while he gets the phone. Mitch's son always does what he's told, so Mitch believes his son will wait for him for just a moment. Did he have a good or bad reason for that belief? Somewhat good?
Mitch's son never does what he's told, but Mitch believes his son will wait for him for just a moment. In this case, that works out really badly. So this is an experiment we did to study how justification affects moral judgment. But we're not going to look-- all those stories that have the same sequence-- so there's a background. There's always a bad outcome. Although, we don't specify how bad, so you don't find out if he hurt, or killed, or whatever. But just a bad outcome happens.
Always, the character did not intend that bad action. It's always accidental. But they either had a good reason for thinking no harm would occur, or a bad reason for thinking no harm would occur in all these cases. And now, again, we're going to look at the pattern of activity, just in this minimal [INAUDIBLE]. And now, we're changing, as you see, very little about these stories in order to list the difference between good and bad evidence.
One of the things we found is [? as it's ?] intuitive, this really affects moral judgments. So the worse your evidence, the more blame you deserve for acting on that belief. And again, we can decode that feature in the right TPJ. So [INAUDIBLE] all accidental harms, the right TPJ contains information about how good or bad your evidence was, thinking no harm would occur. So that's two pieces of evidence that so now, the right TPJ encodes whether you intended it or not, and also, whether your evidence for your beliefs were good or bad, both in a moral and a non-moral context.
Another thing we've asked about is whether you not just [INAUDIBLE] the evidence is good or bad, but how you got your evidence. So going back to this scenario about Leanne. So I told you we had stories like this, where you had good visual evidence. We also have stories where you have auditory evidence. So instead of seeing something, you hear something. So here's a classroom small [INAUDIBLE]. And [INAUDIBLE] Leanne heard Dylan tell a joke and mention her.
So if she's hearing something that makes her conclude that he's making fun of her, these stories are matched on their justification. So now, we're manipulating the source, whether it's auditory or visual information. Or we're not manipulated, whether or not it was good or bad evidence. So we have another orthogonal dimension of simple states. And again, [INAUDIBLE] we can decode that in the right TPJ.
So now, three features. We can decode it was accidental or intentional. Or [INAUDIBLE], we can decode whether you had good or bad evidence. Orthogonally, we can decode whether you got that evidence from seeing or hearing. Again, to check that this is nothing about these stimulate in particular, we're just going to check in another experiment where we also manipulated seeing versus hearing evidence. So here, we run stories like this.
So Eric is going to meet his fiance's parents. And then either he sees his fiance standing with his parents, or he hears his fiance talking to his parents. And again, we can decode whether he got his evidence from seeing versus sharing in this other experiment. Yeah?
AUDIENCE: I think it's useful to have this information when [INAUDIBLE].
REBECCA SAXE: Seeing versus hearing were the other two. So I argued for accidental versus intentional as a fundamental behavioral relevance. I argued for good versus bad reasons as the next most likely thing that would have fundamental relevance. This one is a surprise. Why would you [INAUDIBLE] seeing versus hearing if it's not about good versus bad evidence? And I don't know exactly.
We've argued that developmentally, knowing where people got their evidence from and the consequences of that is very correlated with the development of theory in mind, as measured by false belief performance. So for example, here's a task. You give a three-year-old a box. And inside, there's one of ducks.
And the ducks can differ. Either they differ in color or they differ in weight. So either two visually identical ducks that weigh different amounts, or you have two identically weighed ducks that are different in color. And so you can either say you put in the two differently colored ducks, then you let one person, or one [INAUDIBLE] reach in and touch the ducks, and one person [? look in. ?] And you ask the child, who knows which color duck it is?
So three year olds can't do that task. They can't tell you, you need to see, in order to distinguish by color. And you need to feel, in order to distinguish by weight. And the development of the ability to do that to figure out how what information gets depends on how you got your information is very correlated to the development of [INAUDIBLE] task performance. And so both developmentally and now neurally, it looks like one of the things we encode is where you got your information, maybe because that contains information about what kind information you have.
AUDIENCE: This will be on the same idea here, because there is a different [INAUDIBLE] that that maybe it's easier to lie with your voice than with your face. So if you're talking to somebody, and they're telling you something that that's not true, you might see it.
REBECCA SAXE: Right. So in this case, what you're saying is there might actually be a difference in reliability here. And actually, we think there might be. So as I showed you in the [INAUDIBLE] about seeing versus hearing. This one was not designed for decoding. This is an experiment we designed for other things, many, many years earlier. When we designed an experiment for decoding things versus hearing, we matched it on reliability.
When we went back to decode this dimension in the old data, we don't know whether we're decoding seeing versus hearing, per say, or also some degree of difference in reliability, because we now know both of those pieces of information are present with [INAUDIBLE]. So that's a whole bunch of things we can do. So in the right TPJ, we can decode accidental versus intentional. We can decode the business of your reasons, and we can decode the source of your evidence, and at some point, telling you all the things we can [INAUDIBLE], so beyond what you can do to try to a contrast to something you can't do to understand what's going on here or build a theory.
And we're trying to do both. The first thing I will say is that one feature we cannot decode, no matter how hard we try, is true versus false beliefs. Neither the right TPJ, nor any region we've ever looked at contains information that we can decode about whether our beliefs is true or false. That's one thing we've never been able to decode. A second thing is that this information I've shown you in the right TPJ, it is present in the right TPJ. It's not present in any of the other regions.
So this is data from the same experiment in the medial prefrontal cortex, which is not significant in any of these contrasts. That might be a lack of power. Again, we can't say that for sure. But there's a significant region by function interaction. Essentially, this information is more present in the right TPJ than in any other of these regions, including the [? MPSC. ?]
And then the last thing is so I showed you a whole bunch of academics features which are included in the right TPJ. And maybe so that we have time for questions, what I will just tell you is that valence is encoded in the [? MPSC ?] and not in the right TPJ. I'll just skip to this data so you see that, and then put that here, and then we can spend the last little time for questions.
This is classification accuracy for valence across stimuli in the [? MPSC. ?] I can't do it. I can't click the right number of buttons. In the [INAUDIBLE] cortex, you can decode whether somebody feels good or bad in the [INAUDIBLE]. So here's the overall picture. Using decoding, we're beginning to get a sense of what some possible parts of the face of mental state attribution are.
In the right TPJ, there's representations that is decodable as linear decoding over [INAUDIBLE] of epistemic features regarding whether you intended what you did, whether you had good or bad evidence for your belief, and where you got your belief from. In the DMPFC, there's information not about epistemic features, [INAUDIBLE], feel good or bad about what happens. And that's not an RTPJ. Obviously, the point is now to go beyond this. So this with our first attempt to figure out whether there's facially structured information that could tell us about the populations of neurons.
What we now want to do is push beyond that to actually try to get the theory about what the representation is like, what features are represented, and also use this to go back to questions about development and autism that got us going in the first place, and I will stop there. Thank you.
[APPLAUSE]
Other questions? Yeah?
AUDIENCE: Do you have any ideas about why theory of mind adjustments are made there, as opposed to in some other region?
REBECCA SAXE: Why is the right TPJ in that physical part of the brain?
AUDIENCE: [INAUDIBLE].
REBECCA SAXE: I have an idea, and I tested it. And there was no evidence for it. So when I first started working on it, so I first started working on the right TPJ in 2001. People knew that there was a representation of social content from perception in the STS. So there's a region in the right posterior STS that responds to faces, especially dynamically moving faces, voices, biological motion, and so on.
Have you guys already seen this data? Have they been presented through yet? Anyway, in the right posterior [INAUDIBLE] is a brain region that represents other people's mental state. [INAUDIBLE] It represents social parameters estimated from visual stimuli. And that region is here. And so actually, in the very first analyses that people, including me, had done of the very first 5 or 10 fMRI experiments, the first review paper on the neural basis of theory of mind was written when there were five papers to review, which is amazing.
At that time, people assumed there was only one region. So there was one region, one big region here that responded to biological motion, and face perception, and voice perception, and theory of mind story. So this is one representation of social stuff that didn't distinguish between different aspects of social cognition. Early on, I showed, and now, many other people have shown that there's at least two.
Probably, there's way more than two. But there's at least two distinct representations, and that the right TPJ, which responds to mental states, is [INAUDIBLE] posterior to the STS region that responds to the visually perceived stuff. So the right is not activated basically at all by videos of basic actions, goal-directed actions, or faces, or any of that stuff. But then I thought, well, maybe it's not a coincidence that they're next to each other. And I suggested that maybe, there's a processing stream where information gets increasingly abstract as you go up this [? pure ?] temple [INAUDIBLE] so that more interiorally, you have stuff like goals, the kinds of representations that Josh probably talked in his talk so that goal directed actions, the things that babies know.
And as you get more posteriorally, you get these more abstract conceptions, including the ones that it takes a really long time to develop that are built on top of the more basic input. And so one of the things we tested when we first scanned kids is there any evidence that these two things start out, for example, as one thing and differentiate with development? Again, since we can only scan kids down to age four, it's possible that we totally missed that. But so far, we've never found any evidence.
So you should think of this as the end of a processing stream that's the most abstract from the representation of goals and actions. But it'd be cool if it were true. Yeah?
AUDIENCE: Is this TPJ the same TPJ that [INAUDIBLE]?
REBECCA SAXE: It's not. It's nearby, but it's not the same. You can show that it's not the same in a bunch of different ways. The simplest way is just run both experiments in the same subject and ask if you get the same region. And the answer is you get two regions near each other, but they're reliably distinct. Good question though. Yeah?
AUDIENCE: Is there any history of lesion or brain [INAUDIBLE] from this? Heart rate?
REBECCA SAXE: Yeah. So the question was is there lesion evidence for the causal role? So we did a TMS experiment, which I briefly mentioned to you, which is a reversible lesion. So when we deliberately targeted this region, we could show that it had some causal role. There has been lesion studies, lesion association studies. The most specific deficit and theory of mine that has ever been discovered through a lesion study, it was in the set of studies by Ian [INAUDIBLE] and [INAUDIBLE] Sampson that I think was done as right as you possibly could do it.
They designed a really careful task that separated theory of mind competence from a bunch of other related things that you need to solve the tasks. They ran a big battery of patients on those tasks lined to the site of their legions. [? So to speak, ?] the experimenters were blind. And then they found the participants, the patients, who had a disproportionate deficit in theory of mind relative to all their other cognitive abilities. And then they did lesion overlap studies that asked, these people are disproportionately [INAUDIBLE] in theory of mind. Where is their lesion?
That feels like a very clear way to look for an association between damage to a legion and a positive function instead of going the other way, because it means you really tested all the brain regions simultaneously in an even handed way with a caveat that brain damage doesn't happen easily. So you tested distribution of brain damage as it occurs. When they did that, they got the left TPJ, not the right TPJ. [INAUDIBLE] characterized that, and followed it up, and showed that it is quite a particular deficit in theory of mind that you get in the left TPJ. And then as you saw, we also get the left TPJ in all of our studies, with one exception, that the left TPJ, unlike the right TPJ, seems to also be involved in false photographs.
So thinking about other kinds of false representations. So all the other controlled tasks were unimpaired. But false photographs were specifically impaired, along with false beliefs. And in our data, the left TPJs were [INAUDIBLE] false beliefs and false photographs relative to a range of other controls. [INAUDIBLE] has concluded from this that there is no brain region that's specific to theory of mind to false beliefs and false photographs. I asked them what about the right TPJ?
What happens with patients who have damage in the right TPJ? Are they unimpaired on these tasks? And the answer was no, they're impaired on the theory of mind tasks. But they're also impaired on attention tasks and memory tasks. And so it's hard to say whether they have a specific deficit in theory of mind or a failing of tasks because they can't do the intentional and memory demanding parts.
Now, that could be because there's no specific neurobasis of theory of mind. Or it could be because naturally occurring brain damage is usually pretty big, and it's spreading to these neighboring brain regions that are necessary for attention and memory. And that was why we did TMS, because with TMS, you can target a particular brain region. Yeah?
AUDIENCE: I have one question. But before I ask you, I have to ask another question, which is are there any studies that use [INAUDIBLE] stimuli?
REBECCA SAXE: Yes.
AUDIENCE: And then my actual question is are there any studies that look at [INAUDIBLE]?
REBECCA SAXE: So there are a bunch of experiments that use [INAUDIBLE] six stimuli. But none of the [INAUDIBLE] stimuli that we have ever used to activate this region or that anyone has are particularly appropriate for [INAUDIBLE]. So most the powerful, nonverbal way to get this activation is you show the people movies of whatever you want. This is the [INAUDIBLE] for [? movies of ?] goal-directed actions and for [INAUDIBLE] facial expressions. And then you do a task manipulation.
So on [INAUDIBLE], you say, think about how the person is doing [INAUDIBLE]. So you think about the physical body, [INAUDIBLE], developed [INAUDIBLE]. Think about how they're moving their body to do that, because it's a facial expression. Think about how they're moving their face to create that facial expression. And the other challenge, we say, think about why they're doing that thing. What's the background and context that's causing them to do that thing?
That contrast, why versus how? I over [INAUDIBLE] regions. How [INAUDIBLE] a few different regions. And so that is a nonverbal stimulus for the task [INAUDIBLE]. It's a very different way to work with these [INAUDIBLE] the way we do it. But again, [INAUDIBLE] has to do with [INAUDIBLE]. It's hard to explain what you mean by why.
AUDIENCE: Is there anything you can do with the [INAUDIBLE]?
REBECCA SAXE: So we've talked about this, to some extent, [INAUDIBLE]. And there's lots of social cognition in monkeys. [INAUDIBLE] social cognition. One of the things they do, for example, is they know about dominance rank. They know it really well. They know who everybody in their group is and how they're related to one another by dominance. So far, when people have tried to study social stuff in monkeys. What they're studying is the STS.
So they're studying this region over here, we think, the homologue of the STS. And they've shown, for example, that the monkeys in the STS, they're [INAUDIBLE] responses. There's an association between activity in the STS and dominance, both your own dominance and perception of dominance. So one argument that people have made based on some anatomical and some ethological evidence, monkeys and humans share a bunch of social, perceptual, and social-cognitive capacity supported by the STS. And in addition, humans have unique social cognition that goes to these abstract levels of theory of mind.
And the unique social cognition is functions in the right TPJ, which has no monkey homologue. So the evidence that there's no monkey homologue for the TPJ comes from DTI connectivity fingerprinting. And so if you ask-- I don't know if you guys seen this, the [INAUDIBLE] stuff on DTI connectivity fingerprinting. So you can take a human brain and a monkey brain, and for each region, use only DTI to define the connectivity of that region to target all of the brain or to select the target [INAUDIBLE] you already believe in.
And then you can ask, can we identify the same region across species by the large scale pattern of the connectivity of that region to the rest of the brain? And so they've shown that for many, for most regions, anatomically defined regions, you can define a homology that way and say, here's a region in the monkey. Here's a region in the human that have the most similar connectivity fingerprints to the rest of the brain. And there's a very small number of cases where you can't, where there's no single best match between the human and the monkey, in terms of connectivity fingerprint to the rest of the brain.
And the two cases that I know of are both [INAUDIBLE] regions, one medium frontal region, and the TPJ, where they've claimed that there's no single best match, and in fact, no good match between the connectivity fingerprint in humans and in monkeys. For the TPJ in particular, [INAUDIBLE] has argued, which I think is totally cool if true, that the TPJ actually has arisen by duplication and differentiation from the STS, with respect to the macaque common ancestors. The argument is macaques and humans shared a common ancestor that had social perception in the FTS.
That region has been duplicated and differentiated, and the humans now have both the STS and a new region with a different function, which is TPJ. Way cool if true, very hard to test. But it means that for example, all the strategies that have been taken in the FSA can say, let's study the function of the neurons in this brain region by finding a homologue and studying that. Won't work, and not only because [INAUDIBLE].
AUDIENCE: Sorry. Are there any behavioral relationships between language and theory of mind [INAUDIBLE]?
REBECCA SAXE: Yes. Language is deeply, deeply related to theory of mind and development. So here's the simple version of the answer to that question. In development, the [INAUDIBLE] language delays theory of mind. One test of this is deaf kids born to signing versus non-signing parents. Deaf kids born to signing parents get exposed to language from the moment they're born. It's a visual language, but that's not [INAUDIBLE] language.
Deaf kids born to non-signing parents typically have a two to three year delay for they get regular exposure to language, because their parents don't speak a language they can perceive. That delay causes the delay in theory of mind. If you look at deaf kids whose exposure to language does delay longer, the theory of mind is delayed longer. And if you look at the signing competence of the deaf kid's parents, you can predict the theory of mind development of the deaf kids Again, only for kids of non-signing parents. With kids with signing parents, they develop just like typical kids.
This is not about deafness. It's about exposure to language. So that says in development, language plays a causal role in the development of this kind of theory of mind. On the other hand, in adulthood, language isn't necessary.
Associated Research Thrust:
have an interactive transcript feature enabled, which appears below the video when playing. Viewers can search for keywords in the video or click on any word in the transcript to jump to that point in the video. When searching, a dark bar with white vertical lines appears below the video frame. Each white line is an occurrence of the searched term and can be clicked on to jump to that spot in the video.
