Dynamic choice in a complex world

We consider an environment in which a Decision Maker (DM) finds it sufficiently complex to even describe the state space, let alone guess the parameters of the underlying data generating process. He is therefore unable to use the standard Bayesian methods. Instead, at each moment in time, the DM constructs a preference relation on the set of available actions based on their past performance. We postulate a set of axioms on this family of preference relations indexed by histories (of rewards). Two key conditions, a typical Exchangeability axiom and another labeled Consistency, that constraints the DM not to update her preferences after certain events (loosely based the “the principle of insufficient reason”), characterize thumb rules for ranking actions that are akin to the familiar fictitious play in game theory. Moreover, if in fact the stochastic process that generates the rewards is exchangeable, a DM that obeys our axioms (almost surely) cannot be distinguished in the long run from one that is fully cognizant of the environment and satisfies the expected utility hypothesis. The main result requires proving a representation result on multisets which may be of independent interest.