Deep Exploration

When the environment is relatively stable, the forces of natural selection would favor preferences aligned with survival. However, as the environment changes, individual preferences could become suboptimal, i.e., they could encourage harmful behavior and discourage beneficial behavior. The environment could change for various reasons. For example, historically, the earth has experienced periods of sudden and dramatic changes of its climate. Not all changes, however, are as dramatic. Technological development has fundamentally transformed our environment over short periods of time. AI will further reshape all aspects of our lives. The chapter introduces the concept of deep exploration, defined as the accumulation of knowledge about the optimality of primary preferences. Given that people cannot learn about the optimality of their own preferences through personal experiences alone, societies would encourage collective efforts aimed at a better understanding of the implications of their preferences. The knowledge generated is then transferred from one generation to the next in oral and written forms. A preference system that internalizes this knowledge is referred to as secondary preferences. When people start questioning the signals of their reward system, this creates the need for external feedback on their actions. This is one important reason people are prone to share their experiences, especially the ones they regard as extreme. Bragging, although prevailing in life, has been difficult to explain. One potential benefit of bragging is that it solicits feedback on actions that are more likely to be repeated. The chapter also discusses the importance of habit formation for behavior modification and the ability of individuals to change their primary preferences.