Cost-Benefit Arbitration Between Multiple Reinforcement-Learning Systems.

TitleCost-Benefit Arbitration Between Multiple Reinforcement-Learning Systems.
Publication TypeJournal Article
Year of Publication2017
AuthorsKool, W, Gershman, SJ, Cushman, FA
JournalPsychol Sci
Date Published2017 Sep

Human behavior is sometimes determined by habit and other times by goal-directed planning. Modern reinforcement-learning theories formalize this distinction as a competition between a computationally cheap but inaccurate model-free system that gives rise to habits and a computationally expensive but accurate model-based system that implements planning. It is unclear, however, how people choose to allocate control between these systems. Here, we propose that arbitration occurs by comparing each system's task-specific costs and benefits. To investigate this proposal, we conducted two experiments showing that people increase model-based control when it achieves greater accuracy than model-free control, and especially when the rewards of accurate performance are amplified. In contrast, they are insensitive to reward amplification when model-based and model-free control yield equivalent accuracy. This suggests that humans adaptively balance habitual and planned action through on-line cost-benefit analysis.

Alternate JournalPsychol Sci
PubMed ID28731839

Research Area: 

CBMM Relationship: 

  • CBMM Funded