Foraging models explain human exploration in uncertain tasks

Abstract Different fields have fundamentally different models of how decisions are made. Psychology and neuroscience tend to assume that decisions are made by calculating and comparing the values of all options. In ethology, conversely, decisions tend to be binary regardless of the number of options: Decision-makers calculate the value of continuing to exploit some option and explore only when this value drops below a threshold. Because these fields use incompatible methods, it remains unclear which view better describes human decision-making. We find that humans use compare-to-threshold computations even in classic compare-alternative tasks. Because the reinforcement-learning models typically used in the cognitive and brain sciences depend on compare-alternative computations, we also develop a compare-to-threshold foraging model. Compared to previous models, the foraging model better fits participant behavior, predicts the tendency to repeat choices, and predicts held-out participants that were almost impossible under traditional compare-alternative models. These results suggest that humans use compare-to-threshold computations in more environments than were previously known.