Long-run choice anomalies in reinforcement learning with bounded memory

Violations of expected utility (EU) maximization have been demonstrated in many settings; however, anomalies are often reduced after repeated choices. We examine if sufficient experiential learning allows convergence to EU-maximization. In the model, a decision maker with long but finite memory repeatedly makes choices in the same decision problem with uncertainty. We focus on the existence and severity of a certain unambiguous type of long-run choice anomaly: ranking reversals (a non-EU maximizing action being most frequently chosen in the long run). We show reversals exist for almost all preferences, even in realistic examples. Reversals tend to happen when payoff differences are heavily skewed. Longer memory does not eliminate the possibility of ranking reversals, but it does make reversals less severe. Our key takeaway is that finite memory can produce major violations of the expected utility ranking even in a model where both memory and the decision-making process are unbiased.