Collaborator: Nancy Kanwisher, Rebecca Saxe; CBMM Thrust 4 – Social Intelligence

Progress of CBMM Challenge – Social Intelligence

Abstract:

The central aim of Thrust 4 is to understand nonverbal social perception, or the ability to make high-level social inferences from perceptual information, in order to allow the development of models able to answer CBMM challenge questions about people and their social interactions from images and videos. The study of social perception is in its relative infancy: we have a catalog of intriguing behavioral findings on what sorts of inferences can be made, but don’t have a sense of the full scope of these abilities, nor any understanding of their computational basis or neural implementation.

Here, we propose to investigate the functional organization of the neural machinery for social perception as a first step toward understanding the computational and neural architecture of these abilities.  Specifically, we target the superior temporal sulcus, which has been argued to play a role in a large number of social perceptual and cognitive processes: the perception of faces, biological motion, and voices, as well as the ability to understand language and mental state content.

In experiment 1, we use fMRI to measure STS responses during a number of different social perceptual tasks, and compare responses within individual participants.  We find that the STS has a strong spatial organization, with distinct processes eliciting distinct patterns of activity.  Furthermore, we find both evidence for selective STS subregions, which respond strongly to one task and not others, and multifunctional regions in which responses to multiple tasks overlap.

In experiment 2 we further investigate a particularly strong case of overlap from experiment 1 – overlap between responses to dynamic faces and vocal sounds. […]