The project that cuts best across all the thrusts consists of a set of questions on a set of databases of images and videos. The questions are designed to measure intelligent understanding of the physical and of the social world by our models. We will use the performance on these questions to measure and communicate our progress during the initial five years of the Center.
Understanding the development of intelligence in human infants is a key project for the Center. This project engages the fundamental tradeoff between nature and nurture, or priors and data, and ultimately the origin of priors, i.e., how constraints are selected by evolution, encoded in genes, and instantiated in genetically wired brain circuits.
This project also represents a novel developmental approach to building human-like artificial intelligence systems and comprehensive computational models of human cognitive architecture. Instead of branching out from a single area of adult human intelligence, we start with an integrated cognitive model of multiple core capacities in the young child’s mind, along with a set of developmental or learning mechanisms for scaling up to an adult model.
Abstract thinking and complex problem solving constitute paradigmatic examples of computation emerging from interconnected neuronal circuits. The biological hardware represents the output of millions of years of evolution leading to neuronal circuits that provide fast, efficient, and fault-tolerant solutions to complex problems. Progress toward a quantitative understanding of emergent intelligent computations in cortical circuits faces several empirical challenges (e.g., simultaneous recording and analysis of large ensembles of neurons and their interactions), and theoretical challenges (e.g., mathematical synthesis and modeling of the neuronal ensemble activity). Our team of theoreticians and neurophysiologists is focused on systematic, novel, and integrative approaches to deciphering the neuronal circuits underlying intelligence. Understanding neuronal circuits that implement solutions to complex challenges is an essential part of scientific reductionism, leading to insights useful for developing intelligent machines.
The goal of this research is to combine vision with aspects of language and social cognition to obtain complex knowledge about the environment. To obtain a full understanding of visual scenes, computational models should be able to extract from the scene any meaningful information that a human observer can extract about actions, agents, goals, scenes and object configurations, social interactions, and more. We refer to this as the ‘Turing test for vision,’ i.e., the ability of a machine to use vision to answer a large and flexible set of queries about objects and agents in an image in a human-like manner. Queries might be about objects, their parts, and spatial relations between objects, actions, goals, and interactions. Understanding queries and formulating answers requires interactions between vision and natural language. Interpreting goals and interactions requires connections between vision and social cognition. Answering queries also requires task-dependent processing, i.e., different visual processes to achieve different goals.
Social cognition is at the core of human intelligence. It is through social interactions that we learn. We believe that social interactions drove much of the evolution of the human brain. Indeed, the neural machinery of social cognition comprises a substantial proportion of the brain. The greatest feats of the human intellect are often the product not of individual brains, but people working together in social groups. Thus, intelligence simply cannot be understood without understanding social cognition. Yet we have no theory or even taxonomy of social intelligence, and little understanding of the underlying brain mechanisms, their development, or the computations they perform. Here we bring developmental, computational, and cognitive neuroscience approaches to bear on a newly tractable component of social intelligence: nonverbal social perception (NVSP), which is the ability to discern rich multidimensional social information from a few seconds of a silent video.
Understanding intelligence and the brain requires theories at different levels, including the biophysics of single neurons, algorithms and circuits, overall computations and behavior, and a theory of learning. Advances have been made in many of these areas from multiple perspectives in the past few decades. In fact several major contributors to these advances are members of our team.
This theoretical foundation provides a common framework for fields as diverse as computer science, cognitive science, and neuroscience. Recent successes in intelligent systems applications – from Google to Watson – would not have been possible without these developments. For the first time, we have the beginnings of a unifying and useful mathematics of brains, minds, and machines with rigorous foundations, demonstrated applicability in almost every area of cognitive and neural science, and real practical value for building intelligent systems.
The goal is to maintain diversity and serendipity in the Center. Formed in the middle of CBMM's third year, this research thrust contains new projects, often by new faculty that are not part of the other thrusts, but fit into the overall mission of the Center and may become, in the near future, mainstream research directions for the Center.