Memory and Executive Function | Brain OS

There has been significant progress towards understanding the hierarchical sequence of operations during the first ~150 milliseconds of visual processing (“immediate vision”), building up to the formulation of a theoretical framework instantiated in successful computational algorithms for visual recognition. The bottom-up computations involved in immediate vision (Module 1) provide an initial and often accurate estimation of the contents within a radius subtending approximately 5 degrees around the fixation point. Module 2 aims to understand the visual routines and computations that take place during the subsequent ~300 ms of cortical processing and which are critical for the perceptual intelligence required to interpret a visual scene.

We think about the brain’s operating system as consisting of a series of visual routines (and subroutines), and the ability to flexibly and dynamically call upon them and combine them to solve specific visual tasks. As a working hypothesis, we postulate that the following visual routines would be important components required for scene understanding:

  1. Extracting initial sensory map → Call VisualSampling
  2. Propose image gist → Call RapidPeripheralAssessment
  3. Propose foveal objects → Call FovealRecognition
  4. Inference → Call VisualInference
  5. Specific detectors → Call ObjectClassifier, call ObjectLocator
  6. Temporary information storage → Vall VisualBuffer
  7. Task-dependent sampling → Call EyeMovement
  8. Determine spatial relationships → Call SpatialRelationships
  9. Determine object interactions → Call ObjectInteractions
  10. Decision making and answer → Call DecisionMaking, TaskReport

The projects in Module 2 are actively pursuing the neural and computational mechanisms instantiating these routines.

Recent Publications

M. J. McPherson and McDermott, J. H., Time-dependent discrimination advantages for harmonic sounds suggest efficient coding for memory, Proceedings of the National Academy of Sciences, vol. 117, no. 50, pp. 32169 - 32180, 2020.
CBMM Related
W. A. Freiwald, Social interaction networks in the primate brain, Current Opinion in Neurobiology, vol. 65, pp. 49 - 58, 2020.
CBMM Funded
W. Kool and Botvinick, M., Mental labour, Nature Human Behaviour, vol. 2, no. 12, pp. 899 - 908, 2018.
CBMM Funded