Very Slow Search and Reach: Failure to Maximize Expected Gain in an Eye-Hand Coordination Task

We examined an eye-hand coordination task where optimal visual search and hand movement strategies were inter-related. Observers were asked to find and touch a target among five distractors on a touch screen. Their reward for touching the target was reduced by an amount proportional to how long they took to locate and reach to it. Coordinating the eye and the hand appropriately would markedly reduce the search-reach time. Using statistical decision theory we derived the sequence of interrelated eye and hand movements that would maximize expected gain and we predicted how hand movements should change as the eye gathered further information about target location. We recorded human observers' eye movements and hand movements and compared them with the optimal strategy that would have maximized expected gain. We found that most observers failed to adopt the optimal search-reach strategy. We analyze and describe the strategies they did adopt. Author Summary: A variety of human daily activities, such as cooking, drawing, and driving, involve coordination of eye and hand. Typically your hand moves towards whatever you have just looked at. But is this coupling compulsory? To test whether human observers can adopt appropriate eye-hand coordination strategies to maximize rewards, we created an unusual task where good performance required that hand and eye move independently of each other in order to rapidly find and touch a target. Observers were rewarded for minimizing the overall time to find and touch a target among distractors and we made the visual search and hand movements very slow so that a simple “hand-follows-eye” strategy would reduce observers' winnings considerably. Most observers failed to choose the optimal visual search strategy but did intelligently coordinate hand movements to the visual search strategy they did pick. The “very slow search and reach” task we developed provides a novel approach to investigate coordination between perceptual and motor systems experimentally and computationally.