Modeling Human Cognition & Learning

Modeling Human Cognition & Learning

Josh Tenenbaum and colleagues propose that our intuitions about properties like the stability of a stack of blocks may be based on “probabilistic programs” in our head that simulate the approximate physics of how objects behave in space and time.

Overview

How do we make intelligent inferences about objects, events, and relations in the world from our sensory input? From infancy to adulthood, how do we learn new concepts and reason about novel situations, from little experience? How can we build a robotic system that embodies this intelligence? This unit explores learning and common-sense reasoning in humans and machines.

Videos

Josh Tenenbaum: Computational models of cognition, Part 1

Models of human intelligence have framed the underlying processes as primarily engaged in pattern recognition, prediction, or symbol manipulation. In Part 1, Josh Tenenbaum argues that systems that can reason broadly about the physical and social world must embody models of the world that enable explanation and understanding of what we sense, prediction of future states, problem solving and action planning, and learning of new models with experience. Such systems may be created through an approach that starts with the intelligence of a baby and learns like a child.

Josh Tenenbaum: Computational models of cognition, Part 2
In Part 2, Josh Tenenbaum describes an intuitive physics engine that uses a probabilistic framework and inverse graphics to capture our understanding of the physical behavior of objects. This physics engine embodies a generative model that incorporates uncertainty in a way that enables approximate, intuitive inference, and predictions of object behavior. This framework is also used to construct an intuitive psychology engine that enables inferences about the beliefs and desires of other agents, their goals and actions, and social interactions with other agents.
Josh Tenenbaum: Computational models of cognition, Part 3
In Part 3, Josh Tenenbaum describes recent efforts to formulate neurally plausible models for face recognition, intuitive physics, and intuitive psychology, that integrate the probabilistic programming framework with deep neural networks. Results of empirical studies provide hints about the brain areas that may be engaged in these computations. The lecture concludes with a reflection on how the understanding of intuitive physics may be learned from infancy through childhood.
Pietro Perona: How did you learn the natural numbers?
Pietro Perona explores the ability to understand and manipulate numbers and quantities, which emerges during childhood. He proposes a model of its development in which the spontaneous and undirected manipulation of small objects trains perception to predict the resulting scene changes, without explicit teacher supervision. In this model, an image representation emerges that foreshadows numbers and quantity, including distinct categories for the first few natural numbers, a notion of order, and a signal that correlates with quantity.
Leslie Kaelbling: Recurrent computations to the rescue
Building a physical robot that interacts with the world provides a useful test bed for understanding the kind of reasoning, perception, and control needed to create an intelligent system. Leslie Kaelbling describes an integrated approach that weaves together perception, estimation, geometric reasoning, symbolic task planning, learning, and control to generate robust behavior in a real robot that performs manipulation tasks in complex, uncertain domains, such as the context of a mobile manipulator doing household tasks.

Further Study

Online Resources

Additional information about the speakers’ research and publications can be found at these websites:

The unit on Development of Intelligence in the Brains, Minds, and Machines Summer Course published on MIT OpenCourseWare, includes talks by leading researchers in cognitive development that elucidate the cognitive capacities of infants and how they develop through early childhood. 

The interactive eBook, Probabilistic Models of Cognition, by Noah Goodman and Josh Tenenbaum, explores the probabilistic approach to cognitive science through models implemented in the WebPPL programming language. 

The tutorial by Kevin Smith introduces the WebPPL probabilistic programming language through examples of generative models and inference techniques implemented in this language. 

ThreeDWorld (TDW) is a general-purpose virtual world simulation platform to support physical interactions between mobile agents and objects in rich 3D environments, to support modeling and experiments in physical reasoning. 

Readings

Battaglia, P. W., Hamrick, J. B., Tenenbaum, J. B. (2013) Simulation as an engine of physical scene understanding, Proceedings of the National Academy of Sciences, 110(45), 18327-18332

Kaelbling, L. (2019) Engineering AI, online essay

Kaelbling, L., Lozano-Perez, T. (2013) Integrated task and motion planning in belief space, International Journal of Robotics Research, 32(9-10), 1194-1227

Kondapaneni, N., Perona, P. (2020) A number sense as an emergent property of the manipulating brain, arXiv arXiv:2012.04132v2 

Lake, B. M., Salakhutdinov, R., Tenenbaum, J. B. (2015) Human-level concept learning through probabilistic program induction, Science, 350(6266), 1332-1338

Lake, B. M., Ullman, T. D., Tenenbaum, J. B., Gershman, S. J. (2017) Building machines that learn and think like people, Behavioral and Brain Sciences, 40:e253

Liu, S., Ullman, T. D., Tenenbaum, J. B., Spelke, E. S. (2017) Ten-month-old infants infer the value of goals from the costs of actions, Science, 358, 1038-1041

Rule, J. S., Tenenbaum, J. B., Piantadosi, S. T. (2020) The child as hacker, Trends in Cognitive Sciences, 24(11), 900-915

Silver, T., Allen, K. R., Lew, A. K., Kaelbling, L., Tenenbaum, J. (2020) Few-shot Bayesian imitation learning with logical program policies, arXiv arXiv:1904.06317v2

Teglas, E., Vul, E., Girotto, V., Gonzalez, M., Tenenbaum, J. B., Bonatti, L. L., Pure reasoning in 12-month-old infants as probabilistic inference, Science, 332(6033), 1054-1059

Wu, J., Xue, T., Lim, J. J., Tian, Y., Tenenbaum, J. B., Torralba, A., Freeman, W. T. (2018) 3D interpreter networks for viewer-centered wireframe modeling, International Journal of Computer Vision, 126(9), 1009-1026

Yildirim, I., Belledonne, M., Freiwald, W., Tenenbaum, J. (2020) Efficient inverse graphics in biological face processing, Science Advances, 6, eaax5979