Optimal allocation strategies in a discrete-time bandit problem

We study a discrete-time, two-armed “breakthrough” bandit in which an agent allocates a perfectly divisible resource each period between a safe arm and a risky arm. Departing from the binary “either–or” paradigm, we consider continuous allocation strategies and a general success technology F with nonincreasing hazard rate. Using a variational, pathwise approach combined with dynamic programming, we characterize the unique optimal belief–allocation path via a time-invariant backward/forward transformation. The optimal path features interior, tapering allocations that never stop prior to a breakthrough, and it delivers a strictly higher eventual success probability and expected payoff than the optimal binary (bang-bang) benchmark. In the exponential case, the mappings become explicit, making computation immediate and revealing a Goldilocks principle: total planned allocations to exploration is maximized at intermediate task difficulty. The framework highlights comparative dynamics—how entire optimal paths shift with primitives—while remaining robust to the functional form of F.