Prospective Optimization with Limited Resources

The future is uncertain because some forthcoming events are unpredictable and also because our ability to foresee the myriad consequences of our own actions is limited. Here we studied how humans select actions under such extrinsic and intrinsic uncertainty, in view of an exponentially expanding number of prospects on a branching multivalued visual stimulus. A triangular grid of disks of different sizes scrolled down a touchscreen at a variable speed. The larger disks represented larger rewards. The task was to maximize the cumulative reward by touching one disk at a time in a rapid sequence, forming an upward path across the grid, while every step along the path constrained the part of the grid accessible in the future. This task captured some of the complexity of natural behavior in the risky and dynamic world, where ongoing decisions alter the landscape of future rewards. By comparing human behavior with behavior of ideal actors, we identified the strategies used by humans in terms of how far into the future they looked (their “depth of computation”) and how often they attempted to incorporate new information about the future rewards (their “recalculation period”). We found that, for a given task difficulty, humans traded off their depth of computation for the recalculation period. The form of this tradeoff was consistent with a complete, brute-force exploration of all possible paths up to a resource-limited finite depth. A step-by-step analysis of the human behavior revealed that participants took into account very fine distinctions between the future rewards and that they abstained from some simple heuristics in assessment of the alternative paths, such as seeking only the largest disks or avoiding the smaller disks. The participants preferred to reduce their depth of computation or increase the recalculation period rather than sacrifice the precision of computation.Author Summary: We investigated the human ability to organize behavior prospectively, for multiple future steps in risky, dynamic environments. In a setting that resembled a video game, participants selected the most rewarding paths traversing a triangular lattice of disks of different sizes, while the lattice scrolled down a touchscreen at a constant speed. Disk sizes represented the rewards; missing a disk incurred a penalty. Every choice excluded a number of the disks accessible in the future, encouraging subjects to examine prospective paths as far into the future as they could. In contrast to previous evidence that humans tend to reduce the computational difficulty of decision making by means of simplifying heuristics, our participants appeared to perform an exhaustive computation of all possible future scenarios within a horizon limited by a fixed number of computations. Under increasing time pressure, participants either reduced the computational horizon or recalculated the expected rewards less frequently, revealing a resource-limited ability for rapid detailed computation of prospective actions. To perform such intensive computations, participants could take advantage of the massively parallel neural architecture of the visual system allowing one to concurrently process information from multiple retinal locations.