Memory and Executive Function | Brain OS

There has been significant progress towards understanding the hierarchical sequence of operations during the first ~150 millisconds of visual processing (“immediate vision”), building up to the formulation of a theoretical framework instantiated in successful computational algorithms for visual recognition. The bottom-up computations involved in immediate vision (Module 1) provide an initial and often accurate estimation of the contents within a radius subtending approximately 5 degrees around the fixation point. Module 2 aims to understand the visual routines and computations that take place during the subsequent ~300 ms of cortical processing and which are critical for the perceptual intelligence required to interpret a visual scene.

We think about the brain’s operating system as consisting of a series of visual routines (and subroutines), and the ability to flexibly and dynamically call upon them and combine them to solve specific visual tasks. As a working hypothesis, we postulate that the following visual routines would be important components required for scene understanding:

  1. Extracting initial sensory map à Call VisualSampling
  2. Propose image gist à Call RapidPeripheralAssessment
  3. Propose foveal objects à Call FovealRecognition
  4. Inference à Call VisualInference
  5. Specific detectors à Call ObjectClassifier, call ObjectLocator
  6. Temporary information storage à Vall VisualBuffer
  7. Task-dependent sampling à Call EyeMovement
  8. Determine spatial relationships à Call SpatialRelationships
  9. Determine object interactions à Call ObjectInteractions
  10. Decision making and answer à Call DecisionMaking, TaskReport

The projects in Module 2 are actively pursuing the neural and computational mechanisms instantiating these routines.

Recent Publications