In Module 4, we are seeking to understand how complex data and knowledge are represented, manipulated, and learned. We aim to understand how humans perform abstract tasks, some of which are quintessentially human, such as the creation and use of concepts, logical reasoning, mathematics, and language, and long-range planning over long periods. Separate accounts for some of these abilities exist, but they are generally limited by not including perception and learning in real-world settings as well as not sharing knowledge or representation between abilities. Module 4 integrates capabilities from all modules in order to develop models that carry out these larger tasks, such as models that acquire language and allow agents to follow commands. Within Module 4, our projects span low-level perception for the symbols needed for higher-level reasoning, heading toward how to structure image understanding capabilities, expanding those capabilities to include social understanding and culminate in understanding long plans and stories.
Symbolic Compositional Models
Recent Publications
“Partially Occluded Hands: A challenging new dataset for single-image hand pose estimation”, in The 14th Asian Conference on Computer Vision (ACCV 2018), 2018.
partially-occluded-hands-6.pdf (8.29 MB) ,

CBMM Funded
“Modeling Expectation Violation in Intuitive Physics with Coarse Probabilistic Object Representations”, 33rd Conference on Neural Information Processing Systems (NeurIPS 2019). Vancouver, Canada, 2019.
ADEPT_NeurIPS.pdf (11.07 MB) ,

CBMM Funded
“Deep sequential models for sampling-based planning”, in The IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2018), Madrid, Spain, 2018.
kuo2018planning.pdf (637.67 KB) ,

CBMM Funded
“Grounding language acquisition by training semantic parsersusing captioned videos”, in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP 2018), , Brussels, Belgium, 2018.
Ross-et-al_ACL2018_Grounding language acquisition by training semantic parsing using caption videos.pdf (3.5 MB) ,

CBMM Funded
CBMM Memo No.
101
“Hippocampal Remapping as Hidden State Inference”. 2019.
CBMM-Memo-101.pdf (12.78 MB) ,

CBMM Funded