A reproducible 3D convolutional neural network with dual attention
  module (3D-DAM) for Alzheimer's disease classification
Alzheimer's disease is one of the most common types of neurodegenerative
disease, characterized by the accumulation of amyloid-beta plaque and tau
tangles. Recently, deep learning approaches have shown promise in Alzheimer's
disease diagnosis. In this study, we propose a reproducible model that utilizes
a 3D convolutional neural network with a dual attention module for Alzheimer's
disease classification. We trained the model in the ADNI database and verified
the generalizability of our method in two independent datasets (AIBL and
OASIS1). Our method achieved state-of-the-art classification performance, with
an accuracy of 91.94% for MCI progression classification and 96.30% for
Alzheimer's disease classification on the ADNI dataset. Furthermore, the model
demonstrated good generalizability, achieving an accuracy of 86.37% on the AIBL
dataset and 83.42% on the OASIS1 dataset. These results indicate that our
proposed approach has competitive performance and generalizability when
compared to recent studies in the field.
Gia Minh Hoang, Youngjoo Lee, Jae Gwan Kim
A Reproducible 3D Convolutional Neural Network with Dual Attention Module (3D-Dam) for Alzheimer's Disease Classification

###### Abstract

 Alzheimer's disease is one of the most common types of neurodegenerative disease, characterized by the accumulation of amyloid-beta plaque and tau tangles. Recently, deep learning approaches have shown promise in Alzheimer's disease diagnosis. In this study, we propose a reproducible model that utilizes a 3D convolutional neural network with a dual attention module for Alzheimer's disease classification. We trained the model in the ADNI database and verified the generalizability of our method in two independent datasets (AIBL and OASIS1). Our method achieved state-of-the-art classification performance, with an accuracy of 91.94% for MCI progression classification and 96.30% for Alzheimer's disease classification on the ADNI dataset. Furthermore, the model demonstrated good generalizability, achieving an accuracy of 86.37% on the AIBL dataset and 83.42% on the OASIS1 dataset. These results indicate that our proposed approach has competitive performance and generalizability when compared to recent studies in the field.

 Alzheimer's disease, Feature extraction, Magnetic resonance imaging, Brain modeling, Deep learning

## I Introduction

Alzheimer's disease is one of the most common causes of neurodegenerative disease among the elderly population. It progressively causes memory impairment and greatly affects the activities of daily living. According to the Alzheimer's Association [1], more than 6.7 million Americans aged 65 or older are living with Alzheimer's disease, and by 2060, this number could rise to nearly 13.8 million. In patients with Alzheimer's disease, the brain undergoes progressive cerebral atrophy, initially manifesting in the medial temporal lobe, followed by the hippocampus, amygdala, and para-hippocamps [2, 3]. While there is currently no effective cure for Alzheimer's disease, there are treatments available that may help manage symptoms or delay disease progression. Therefore, an accurate diagnosis of Alzheimer's disease is crucial as it enables patients to initiate preventive interventions to slow or halt the disease's progression.

With the development of advanced neuroimaging techniques, brain atrophy has emerged as a potential biomarker for Alzheimer's disease diagnosis, visualized through MRI scans. However, manual diagnosis by doctors using MRI scans is time-consuming and relies on the expertise of the clinician. Therefore, there is a growing need for computer-aided approaches in Alzheimer's disease diagnosis. In recent years, various categories of computer-aided diagnostic methods have been increasingly applied to AD diagnosis. One category, known as regional feature-based methods, focuses on determining regions of interest (ROIs) associated with Alzheimer's disease. Segmentation is then performed on these regions, followed by prediction. For instance, Ahmed et al. achieved an accuracy of 90.73% and a precision of 90.17% in Alzheimer's disease classification by extracting the hippocampus region from MRI scans [4]. Similarly, Li et al. demonstrated the effective prediction of early Alzheimer's disease using the hippocampus extracted from MRI scans [5]. However, the features extracted from these regions may not capture the intricate alterations that occur in the brain. To overcome this limitation, whole brain-based methods have been employed to capture the structural changes more effectively in MRI scans. Zhang et al. achieved an accuracy of 91.3% in Alzheimer's Disease classification using self-attention CNN models [6]. Feng et al. demonstrated that whole brain-based deep learning methods outperformed other clinical biomarkers of AD [7]. Recent studies have also reported the efficacy of whole brain-based deep learning approaches not only AD-CN classification but also MCI conversion classification [8, 9, 10, 11]. In this study, we applied whole-brain MRI scans for the diagnosis of Alzheimer's disease and MCI progression.

The Convolutional Block Attention Module (CBAM) was first introduced by Woo et al. in 2018 at the European Conference on Computer Vision [12]. The module included two main components: spatial attention and channel attention, which can learn 'what' and 'where' to focus in the channel and spatial axes, respectively. In our study, we applied dual attention to our convolutional neural networks to capture slice information across the channel and structural information across spatial axes, taking into consideration the 3D structure of whole brain MRI scans.

In this study, we proposed the 3D Convolutional Neural Networks with Dual Attention Module (3D-DAM) for Alzheimer's disease classification using whole brain-based MRI scan. We make the following contributions. Firstly, we enhance the 3D-CNN architecture by incorporating a dual attention module, leading to improved accuracy compared to recent studies. Secondly, we demonstrate the generalizability of our model by achieving promising results on an independent dataset. Additionally, we visualize the regions that our model primarily focused on to ensure interpretability and reliability. Our findings align with previous studies, indicating that the hippocampus and
temporal lobe are strong predictors of Alzheimer's disease diagnosis.

## II MATERIAL AND METHOD

### _Dataset description and image pre-processing._

In this study, the 1.5 T T1-weighted MRI images were collected from the public database of Alzheimer's Disease Neuroimaging Initiative (ADNI). The ADNI, initiated in 2003by Principal Investigator Michael W. Weiner, collected brain images and clinical assessments from thousands of participants across North America. For our study, we selected 403 Alzheimer's disease patients (AD) and 653 cognitive normal patients (CN). The Australian Imaging, Biomarker & Lifestyle Flagship Study of Ageing (AIBL) and the Open Access Series of Imaging Studies 1 (OASIS-1) were used as independent dataset to evaluate the generalizability of our method. Table I presents the demographic information of the subjects included in the study. The ADNI data have been curated and converted to the Brain Imaging Data Structure (BIDS) format following the processing pipeline of the Clinica software [13] to address data leakage concerns highlighted by previous studies [14, 15], such as incorrect data split and delayed split. The images were then pre-processed to extract more informative image features for classification. The pre-processing steps followed the t1-linear pipeline of Clinica software [13]. Firstly, bias field correction was applied using the N4ITK method [15]. Next, an affine registration was performed using the SyN algorithm from ANTs [16] to align each image to the MNI space with the ICBM 2009c nonlinear symmetric template.

### _3D Convolutional Neural Networks with Dual Attention Module (3D-DAM)._

Inspired by the Convolutional Block Attention Module (CBAM) [12], we applied a dual attention module to our 3D-CNN model to capture slice information across the channel and structural information across spatial axes. Firstly, we generate a channel attention map by examining the inter-channel relationships of features. Each feature map channel is treated as a feature detector, and the channel attention mechanism focuses on sliced information within an input MRI scan. Given an intermediate feature map F\(\text{\text{\text{\text{\text{\text{\text{e}}}}}}}\)R\(\text{\text{\text{\text{\text{e}}}}}\)R\(\text{\text{\text{\text{\text{e}}}}}\)S as input, we employ average-pooling and max-pooling operations to derive two different spatial representations of the feature map, denoted as F\({}^{\text{\text{\text{\text{\text{e}}}}}}_{\text{\text{\text{\text{\text{\text{ \text{e}}}}}}}}\) and F\({}^{\text{\text{\text{\text{\text{\text{e}}}}}}}_{\text{\text{\text{\text{ \text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{ \texttext{\text
module is applied, followed by feature pooling using the Avg Pooling layer. Finally, a Fully Connected layer is utilized for classification.

## III Results

### _Evaluation in ADNI datasets_

The proposed architecture was implemented in Python using the PyTorch package. The implementation was carried out on a computer equipped with an Intel(R) Xeon(R) Gold 6258R CPU @ 2.70GHz with 256GB RAM, and 4x NVIDIA GeForce RTX 3090 GPUs. The network was trained using the SGD optimizer with a first momentum of 0.9 and a second momentum of 0.999. The initial learning rate was set to \(10^{6}\) and L2-Regularization value was set to \(10^{6}\). The maximum number of training epochs was set to 200, and a batch size of 16 was used at each iteration.

The performances on AD classification achieved by our 3D-DAM method and the other methods on the test set from ADNI were shown in Table II. As shown in Table II, our method achieved an accuracy of 96.30%, a sensitivity of 93.76%, and a specificity of 97.95% in AD classification. Our proposed method demonstrated top-ranked performance in AD classification. The Area Under the Curve of the Receiver Operating Characteristic curve (ROC-AUC) for AD classification in the ADNI dataset was 0.98 (Fig. 2(a)). The 5-fold cross validation test accuracy results of AD classification in ADNI have been shown in Fig. 4.

### _Generalizability Evaluation using AIBL and OASIS datasets_

To investigate the generalizability of our proposed model, we conducted evaluations on two independent datasets: AIBL and OASIS1. We perform two evaluation experiments as follow. Firstly, we trained our proposed model based on the ADNI dataset and evaluated it using two independent datasets. Secondly, we reversed the training and test datasets in which we use AIBL and OASIS1 as training and ADNI as the test set. The classification performance is shown in Table III and Fig 2(b)-c. The 3D-DAM method obtained potential classification results with an accuracy of 86.3%, sensitivity of 80.2%, and specificity of 87.1% for AD classification in the AIBL dataset. For the OASIS dataset, the model trained on the ADNI dataset achieved classification results of 83.4%, 85.8%, and 82.6% in accuracy, sensitivity, and specificity respectively. The 5-fold cross validation test accuracy results of AD classification in AIBL and OASIS have been shown in Fig. 4. Furthermore, when we trained our model with the AIBL and OASIS datasets and evaluated it on the ADNI set, the classification results were 85.4%, 80.1%, and 89.5% in accuracy, sensitivity, and specificity respectively.

### _Pathological brain region by 3D-DAM model_

One of the most important tasks in computer-aided diagnosis is defining the pathological region that the model focuses on. In this study, we employed Grad-CAM to investigate the most crucial brain region of our 3D-DAM model, ensuring interpretability and reliability. Fig. 5, which depicts sagittal, coronal, and axial plane slices with attention score overlays, demonstrates the highest attention region. As shown in Fig. 5, the hippocampus, medial temporal lobe, and amygdala are three brain regions that play important roles in classifying AD and CN in our proposed model. These regions are consistent with numerous previous studies on AD diagnosis [4,20,21].
## IV Discussion

Effective and precise Alzheimer's disease classification based on computer-aid techniques is critical for early intervention to slow down the progress of the disease. With the development of medical imaging techniques, especially MRI, we could use them as an effective biomarker to detect brain atrophy in Alzheimer's disease patients. In this study, we investigated a comparative study focusing on the classification performance of the 3D convolutional neural networks with dual attention modules. Our proposed method outperformed the current SOTA MRI-based studies [17, 18, 19] on AD classification task. These results imply that the CNN model equipped with channel and spatial attention could achieve better classification performance compared with the current traditional CNN model. In our opinion, one of the possible reasons is that the dual attention mechanism could effectively capture the disease-related structural changing features in the brain.

To verify that hypothesis, we visualize the brain region mostly focused on by our model using Grad-CAM. In Fig. 5, we observed that in AD vs CN classification, the 3D-DAM model focused mostly on the hippocampus, medial temporal, and amygdala regions in agreement with previous studies on AD diagnosis [4, 20, 21]. The medial temporal lobe including the hippocampus is crucial for episodic and spatial memory. The amygdala is involved with the formation of long-term declarative memory. In Alzheimer's disease, brain atrophy is first manifested in the medial temporal lobe several years before diagnosis and then closely followed by the hippocampus, and amygdala. The pathological brain region (hippocampus, temporal lobe, and amygdala) evaluated by attention scores in our model could suggest informative regions for future feature extraction to improve our proposed method by focusing on these ROIs. In addition, these brain regions could give more useful clues or information for doctors in clinical diagnosis.

Application of the Deep Learning model in medical clinical is still challenging due to its difficulty in reproducibility and generalizability. To test our model generalizability, we use two independent datasets to evaluate our method. We have promising performance when evaluating AIBL and OASIS1 using the ADNI-trained model and vice versa. The evaluation result indicates the good generalization capability of our method for AD diagnosis. The Acc, Sen, and Spec of our proposed method based on ADNI are slightly higher than other datasets. The main reason is the ADNI dataset and the AIBL as well as the OASIS1 dataset are collected from distinct protocols, which could have different signal-to-noise ratios and device parameters.

## V Conclusion

Inspired by the dual attention mechanism and 3D convolutional neural networks, we propose a 3D convolutional neural network with a dual attention module (3D-DAM) approach to advance computer-aided Alzheimer's disease diagnosis. Our proposed method makes three major contributions. We evaluated our proposed method on the largest Alzheimer's disease database, ADNI, and verified its generalizability on other independent datasets, namely AIBL and OASIS1. We achieved an Alzheimer's disease classification accuracy of 96.30%, a sensitivity of 93.76%, and a specificity of 97.95%. Furthermore, our proposed method demonstrated good generalizability performance in AIBL and OASIS1 datasets. The high attention scores in the hippocampus and medial temporal lobe, as revealed by an explainable AI, highlight the interpretability and reliability of our model.

## Acknowledgment

This work was supported by Healthcare AI Convergence Research & Development Program through the National IT Industry Promotion Agency of Korea (NIPA) funded by the

Fig 5: Visualization of the pathological brain region identified by the proposed method of AD Classification. The left panel shows the informative locations suggested by attention scores. The right panel shows the related brain region, respectively, with mark.
Ministry of Science and ICT (No. S1601-20-1016); National Research Foundation of Korea (NRF-2022R1A2C3009749, 2022K1A3A1A20014975) and "GIST Research Institute (GRI) IIBR" grant funded by the GIST in 2023.

## Compliance with Ethical standards

This work is a study using human subject data made available in open access by ADNI, AIBL and OASIS. Ethical approval was not required as confirmed by license attached with the open access data.
