EyesGAN: Synthesize Human Face from Human Eyes

Face recognition has achieved notable success across various domains, including mobile payment, authentication, criminal investigation, and urban management. Despite these advances, face occlusion remains a critical challenge in person identification, particularly in anti-terrorism efforts, criminal cases, and public security contexts. To address this issue, we introduce an enhanced deep generative adversarial network (EyesGAN) designed to synthesize human faces from eye images, offering a promising approach for masked face recognition. BicycleGAN is chosen as the baseline and effective improvements have been achieved. First, the self-attentional mechanism is introduced so that the improved model can more effectively learn about the internal mapping between human eyes and face. Second, the perceptual loss is applied to guide the model cyclic training and help with updating the network parameters so that the synthesized face can be of higher-similarity to the ground truth face. Third, EyesGAN has been designed by getting the utmost out of the performance of the perceptual loss and the self-attentional mechanism in GANs. To train and evaluate EyesGAN, we have reconstructed a dataset for eyes-to-face synthesis, leveraging public face datasets. The synthesized faces generated by EyesGAN have been rigorously compared with existing methods, both quantitatively and qualitatively. Extensive experiments demonstrate that our method outperforms state-of-the-art techniques across multiple metrics including Average Euclidean Distance, Average Cosine Similarity, Synthesis Accuracy Percentage, Fréchet Inception Distance. Notably, we achieved a Baidu face recognition rate of 96.1% on 615 test samples from the CelebA database. This study explores the feasibility of facial synthesis from eye images, with the attention map indicating that our network can accurately predict other facial regions based on the eyes alone. Furthermore, we extend our investigation to assess the performance of our proposed method in the recovery of noisy X-ray images. Our approach successfully synthesizes high-quality images that demonstrate a high degree of consistency with the corresponding ground truth images, underscoring its potential for enhancing image quality in medical imaging applications.