Memory and Executive Function | Brain OS

There has been significant progress towards understanding the hierarchical sequence of operations during the first ~150 milliseconds of visual processing (“immediate vision”), building up to the formulation of a theoretical framework instantiated in successful computational algorithms for visual recognition. The bottom-up computations involved in immediate vision (Module 1) provide an initial and often accurate estimation of the contents within a radius subtending approximately 5 degrees around the fixation point. Module 2 aims to understand the visual routines and computations that take place during the subsequent ~300 ms of cortical processing and which are critical for the perceptual intelligence required to interpret a visual scene.

We think about the brain’s operating system as consisting of a series of visual routines (and subroutines), and the ability to flexibly and dynamically call upon them and combine them to solve specific visual tasks. As a working hypothesis, we postulate that the following visual routines would be important components required for scene understanding:

  1. Extracting initial sensory map → Call VisualSampling
  2. Propose image gist → Call RapidPeripheralAssessment
  3. Propose foveal objects → Call FovealRecognition
  4. Inference → Call VisualInference
  5. Specific detectors → Call ObjectClassifier, call ObjectLocator
  6. Temporary information storage → Vall VisualBuffer
  7. Task-dependent sampling → Call EyeMovement
  8. Determine spatial relationships → Call SpatialRelationships
  9. Determine object interactions → Call ObjectInteractions
  10. Decision making and answer → Call DecisionMaking, TaskReport

The projects in Module 2 are actively pursuing the neural and computational mechanisms instantiating these routines.

Recent Publications

W. A. Freiwald, Social interaction networks in the primate brain, Current Opinion in Neurobiology, vol. 65, pp. 49 - 58, 2020.
CBMM Funded
M. J. McPherson and McDermott, J. H., Time-dependent discrimination advantages for harmonic sounds suggest efficient coding for memory, Proceedings of the National Academy of Sciences, vol. 117, no. 50, pp. 32169 - 32180, 2020.
CBMM Related
W. Kool and Botvinick, M., Mental labour, Nature Human Behaviour, vol. 2, no. 12, pp. 899 - 908, 2018.
CBMM Funded