Abstract:

Social interaction perception is a crucial part of the human visual experience that develops early in infancy and is shared with other primates. However, it remains largely unknown how humans compute information about others’ social interactions from visual input. In the first part of my talk, I will present work identifying a neural correlate of social interaction perception in the human brain (Isik et al., 2017). Specifically, we observe a strong univariate response in the posterior superior temporal sulcus (pSTS) to stimuli depicting social interactions between two agents, compared with (i) pairs of agents not interacting with each other, (ii) physical interactions between inanimate objects, and (iii) individual animate agents pursuing goals and interacting with inanimate objects.​ ​This region may underlie our ability to understand the structure of our social world and navigate within it. 

This work underscores the importance of social interaction perception, but leave unanswered the question of how quickly and automatically it occurs. Is social interaction detection a rapid, feedforward perceptual process, or a slower post-perceptual inference? To answer this question, we used magnetoencephalography (MEG) decoding to ask when the human brain detects third-party social interactions​. ​In particular, subjects in the MEG viewed snapshots of​ ​real-world scenes containing a pair of people who were either engaged in a social interaction or acting independently. We could read out the presence versus absence of a social interaction from subjects’ MEG data extremely quickly, as early as 150 ms after stimulus onset. This decoding latency is very similar to previously reported decoding latencies of primarily feedforward visual processes, such as invariant object recognition. Importantly, this decoding does not seem to be based on low-level image properties: these images are not decodable based on pixel intensity or the output of a V1-like model, and the social interaction decoding we observed occurs considerably later than the decoding of low-level image identity we observed in the same subjects. These results suggest that the detection of social interactions is a rapid feedforward perceptual process, rather than a slow post-perceptual inference.