Catching a Baseball: A Reinforcement Learning Perspective Using a Neural Network

Moments after a baseball batter has hit a fly ball, an outfielder has to decide whether to run forward or backward to catch the ball. Judging a fly ball is a difficult task, especially when the fielder is in the plane of the ball's trajectory. There exists several alternative hypotheses in the literature which identify different perceptual features available to the fielder that may provide useful cues as to the location of the ball's landing point. A recent study in experimental psychology suggests that to intercept the ball, the fielder has to run such that the double derivative of $\tan\phi$ with respect to time is close to zero $d^2(\tan\phi)/dt^2\approx o$. Where $\phi$ is the elevation angle of the ball from the fielder's perspective (MCLeod \& Dlenes 1993). We investigate whether $d^2(\tan\phi)/dt^2$ information is a useful cue to learn this task in the Adaptive Heuristic Critic (${\cal AHC}$) reinforcement learning framework. Our results provide supporting evidence that $d^2(\tan\phi)/dt^2$ information furnishes strong initial cue in determinimg the landing point of the ball and plays a key role in the learning process. However, our simulations show that during later stages of the ball's flight, yet another perceptual feature, the perpendicular velocity of the ball $(v_p)$ with respect to the fielder, provides stronger cues as to the location of the landing point. The trained network generalized to novel circumstances and also exhibited some of the characteristic behavior that has been recorded by experimental psychologists among experienced fielders. We believe learning approaches to learn common physical tasks, and similarly motivated work could stimulate useful interdisciplinary research on the subject.