Vision
The mission of CBMM — to make progress on the Science and Engineering of Intelligence — has increasingly focused on the science component. Industry is still focusing exclusively on the Engineering of Intelligence, neglecting the basic Science of Intelligence. By Science we mean the study of nature itself—thus physics, biology, chemistry, and neuroscience. By Engineering, on the other hand, we mean the study and development of human artefacts—thus computer science, including machine learning, not to mention robotics, and even mathematics.
Visual Stream
In Module 1, the stream of visual information from the ventral areas needs to be planned and interpreted in the context of prior knowledge, current tasks and memory. We will improve experimentally and theoretically upon a class of successful hierarchical models of the ventral stream — from HMAX to the recent deep learning models.
Memory and Executive Function | Brain OS
We assume that an executive function module — the focus of our Module 2 — runs a set of routines to integrate vision and cognition. This includes routines for planning the next saccade critical for understanding a scene, as well as planning navigation and grasping. By combining behavioral experiments, neurophysiological recordings and computational models, we aim to build the critical building blocks for running these routines. The perceptual intelligence required to interpret a scene and perform complex common sense tasks on the visual input.
The Cognitive Core (geometry, physics, psychology)
Module 3 focuses on the library of routines that are run, representing common-sense, intuitive knowledge about the world. The three major domains we will study include the qualitative 3D geometry of the scene, the underlying physics and the domain of psychological knowledge, about agents, their beliefs, goals, and interactions. In other words, this Module will describe core cognition in geometry, physics, and psychology. It also will study how new routines can be synthesized from existing ones or learned from experience.
Symbolic Compositional Models
Module 4 contains the higher-level symbolic processes that are important for human visual understanding, including inference on images, language, and videos. We already have begun exploring these more abstract aspects of symbolic cognition.