Stochastic Decision Policy Is Optimal Under Costly Control

Kenway Louie; Aadith Vittala; Marcelo Mattar; Paul William Glimcher

doi:10.64898/2026.10.06.757200

Stochastic Decision Policy Is Optimal Under Costly Control | ResearchHub

Abstract Human and animal decisions are often noisy. This variability is usually attributed to noise defects that an ideal decision maker would eliminate. Here we ask whether stochastic choice can instead be optimal once the cost of implementing prescribed policies is counted. We treat the decision maker as an ideal observer that maximizes expected gain but pays costs to sharpen its policy away from a random default. Using sequential sampling, a setting in which the optimal policy is precisely defined, we show that this trade-off makes a stochastic policy optimal. It replaces the hard decision boundaries of classical theory with soft stopping zones. This soft policy aligns with the strategy used in cutting-edge artificial learning agents for robustness. Noise that corrupts the decision-maker’s belief, an important confound of noisy behavior, produces superficially similar yet distinctive signatures in behavioral policies. Leveraging this, we recovered the policies of two macaques with low-dimensional recurrent neural networks and found that their behavior carries both a stochastic policy and belief noise. These results establish that stochastic decision policies can themselves be optimal, offering a common principle for understanding behavioral variability in biological and artificial decision makers. Significance Humans and animals often make different choices from one to the next even facing identical evidence. This variability is usually treated as noise that an ideal decision maker would eliminate. We show that when precise control over action selections carries a cost, choosing stochastically is the optimal strategy. This turns the hard decision boundaries of classical theory into soft stopping zones. We further show that a stochastic policy and noisy evidence, although they look alike, leave distinguishable behavioral signatures, and we find both in the macaque decisions. These findings establish that behavioral variability as partly a rational allocation of limited control, with implications for interpreting choice data in neuroscience, psychology, and machine learning.

External reference records

Unresolved referenced work

Kept as external metadata until matched

10.2307/1905525

10.1146/annurev.neuro.28.061604.135709

10.1038/nn1209

10.1126/science.153.3731.34

10.1037/0033-295x.113.4.700

10.1016/j.brainresrev.2008.07.001

10.1126/science.1233912

Learning to select computations

2017

10.3115/v1/d14-1179

10.1146/annurev.neuro.051508.135409

10.1523/jneurosci.1844-09.2009

10.1038/nature04766

10.1016/j.pneurobio.2009.01.006

10.1016/j.neuron.2004.09.009

10.1038/nn2077

10.1523/jneurosci.4010-11.2012

10.1016/j.neuron.2016.11.005.

Unresolved referenced work

Kept as external metadata until matched

10.1038/nrn2258

10.1126/sciadv.adl3931

Unresolved referenced work

Kept as external metadata until matched

10.1006/game.1993.1021

10.1146/annurev.psych.55.090902.141429

10.1146/annurev.neuro.29.051605.113038

Unresolved referenced work

Kept as external metadata until matched

Unresolved referenced work

Kept as external metadata until matched

Unresolved referenced work

Kept as external metadata until matched

Dopaminergic control of the exploration–exploitation trade-off via the basal ganglia

2012

10.7554/elife.79450

Discovering cognitive strategies with tiny recurrent neural networks

10.1038/s41586-025-09142-4 · 2025

10.1038/nature03127

Adam: A method for stochastic optimization

2014

10.1016/j.neuron.2015.01.007

10.1037/a0020198

10.1016/bs.plm.2021.02.004

10.1016/bs.plm.2021.02.004 · 2021

Human decision making balances reward maximization and policy compression

10.1371/journal.pcbi.1012057 · 2024

10.1147/rd.53.0183

10.1016/j.cogbrainres.2004.07.007

Unresolved referenced work

Kept as external metadata until matched

Additional external references

10.1016/j.cobeha.2021.02.015

10.1016/j.cobeha.2021.02.015 · doi-reference

10.1613/jair.678

10.1613/jair.678 · doi-reference

10.1287/opre.21.5.1071

10.1287/opre.21.5.1071 · doi-reference

10.1038/nature05852

10.1038/nature05852 · doi-reference

10.1257/aer.104.5.495

10.1257/aer.104.5.495 · doi-reference

10.1523/jneurosci.3733-05.2006

10.1523/jneurosci.3733-05.2006 · doi-reference

10.1037/a0038199

10.1037/a0038199 · doi-reference

10.1016/s0896-6273(02)01092-9

10.1016/s0896-6273(02)01092-9 · doi-reference

10.1214/aoms/1177730197

10.1214/aoms/1177730197 · doi-reference

Optimal policy for value-based decision-making

10.1038/ncomms12400 · doi-reference

10.1038/s41593-019-0453-9

10.1038/s41593-019-0453-9 · doi-reference

10.1007/bf02289729

10.1007/bf02289729 · doi-reference

10.1146/annurev-neuro-072116-031526

Stochastic Decision Policy Is Optimal Under Costly Control

Referenced works

Cited by

External reference records

Additional external references