Extensive Investigation on Smart Healthcare and Alzheimer’s Disease Prediction Using Multimodal Data

Information regarding the same occurrence can be obtained from a variety of detector types, under a variety of settings, in a number of experiments or individuals, and in a variety of disciplines. We refer to each of these acquisition frameworks as “modality.” It is uncommon for a single modality to offer comprehensive understanding of the topic of interest due to the complex nature of natural events. New degrees of freedom are introduced by the growing availability of reporting many modalities on the same system, which raises issues beyond those associated with exploiting each modality independently. We contend that a large number of these queries, or “challenges,” are shared across several fields. With applications spanning from healthcare research to triage, diagnosis, and treatment, multimodal data-driven approaches have become a key component of smart healthcare systems. New requirements for decision-making and data management brought about by smart healthcare systems have spurred the growing development of artificial intelligence-based healthcare services as well as new developments in the healthcare sector. In this work, we offer an extensive overview of existing research approaches, encompassing not just cutting-edge approaches but also the latest advancements in the area. This paper is divided into two halves. First half reviews different modalities in the healthcare system and its challenges. It is derived from numerous examples which showcase how data is fused and applied in healthcare monitoring, disease detection, and disease prediction. And in the second half, prediction of Alzheimer’s disease is done using different modalities like MRI and PET scan using cascaded convolutional neural networks (CNNs). This will provide a concept of general multimodal healthcare review to specific Alzheimer’s disease detection model. First, to convert the local brain image into more compact high-level features, numerous deep 3D CNNs are trained on various local image patches. Following that, to create the latent multimodal correlation features of the pertinent image patches for the classification function and to ensemble the high-level features acquired from the multimodality, a ReLU layer and an upper high-level 3D CNN are cascaded. A fully linked layer and a ReLU layer are then used to combine this learned information for AD classification. The proposed method can automatically learn generic multilevel and multimodal characteristics from several imaging modalities for classification and is relatively robust to size and rotation changes. Accuracy, precision, recall, and F1-score are used to evaluate the results.