Memory and Executive Function | Brain OS

There has been significant progress towards understanding the hierarchical sequence of operations during the first ~150 milliseconds of visual processing (“immediate vision”), building up to the formulation of a theoretical framework instantiated in successful computational algorithms for visual recognition. The bottom-up computations involved in immediate vision (Module 1) provide an initial and often accurate estimation of the contents within a radius subtending approximately 5 degrees around the fixation point. Module 2 aims to understand the visual routines and computations that take place during the subsequent ~300 ms of cortical processing and which are critical for the perceptual intelligence required to interpret a visual scene.

We think about the brain’s operating system as consisting of a series of visual routines (and subroutines), and the ability to flexibly and dynamically call upon them and combine them to solve specific visual tasks. As a working hypothesis, we postulate that the following visual routines would be important components required for scene understanding:

  1. Extracting initial sensory map → Call VisualSampling
  2. Propose image gist → Call RapidPeripheralAssessment
  3. Propose foveal objects → Call FovealRecognition
  4. Inference → Call VisualInference
  5. Specific detectors → Call ObjectClassifier, call ObjectLocator
  6. Temporary information storage → Vall VisualBuffer
  7. Task-dependent sampling → Call EyeMovement
  8. Determine spatial relationships → Call SpatialRelationships
  9. Determine object interactions → Call ObjectInteractions
  10. Decision making and answer → Call DecisionMaking, TaskReport

The projects in Module 2 are actively pursuing the neural and computational mechanisms instantiating these routines.

Recent Publications