Limited but consistent gains in adversarial robustness by co-training object recognition models with human EEG
Manshan Guo Bhavin Choksi Sari Sadiya Alessandro T. Gifford Martina G. Vilas Radoslaw M. Cichy Gemma Roig




 



Adversarial Robustness Gains via EEG Co-Training






Guo et al.




Department of Computer Science, Goethe University, Frankfurt am Main, Germany 
 
{m.guo,choksi,Saba-Sadiya,roignoguera}@em.uni-frankfurt.de The Hessian Center for Artificial Intelligence (hessian.AI), Darmstadt, Germany Frankfurt Institute for Advanced Studies (FIAS), Frankfurt, Germany Department of Education and Psychology, Freie Universität Berlin, Berlin, Germany 

{nikiguo93,rmcichy}@zedat.fu-berlin.de,alessandro.gifford@gmail.com Ernst Strüngmann Institute for Neuroscience, Frankfurt am Main, Germany 

martina.vilas@esi-frankfurt.de 


Limited but consistent gains in adversarial robustness by co-training object recognition models with human EEG
    Manshan Guo1,2,40000-0002-5506-6854 Bhavin Choksi 1,20000-0002-6475-4149 Sari Sadiya1,2,30009-0005-7482-3274 

Alessandro T. Gifford40000-0002-8923-9477 Martina G. Vilas1,2,50000-0002-1097-8534 

Radoslaw M. Cichy4jointly directed work0000-0003-4190-6071 Gemma Roig1,2,⋆0000-0002-6439-8076

    
======================================================================================================================================================================================================================================================================================================



§ ABSTRACT 
  In contrast to human vision, artificial neural networks (ANNs) remain relatively susceptible to adversarial attacks. To address this vulnerability, efforts have been made to transfer inductive bias from human brains to ANNs, often by training the ANN representations to match their biological counterparts. Previous works relied on brain data acquired in rodents or primates using 
  invasive techniques, from specific regions of the brain, under non-natural conditions (anesthetized animals), and with stimulus datasets lacking diversity and naturalness. In this work, we explored whether aligning model representations to human EEG responses to a rich set of real-world images increases robustness to ANNs. Specifically, we trained ResNet50-backbone models on a dual task of classification and EEG prediction; and evaluated their EEG prediction accuracy and robustness to adversarial attacks. We observed significant correlation between the networks' EEG prediction accuracy, often highest around 100 ms post stimulus onset, and their gains in adversarial robustness. Although effect size was limited, effects were  consistent across different random initializations and robust for architectural variants. We further teased apart the data from individual EEG channels and observed strongest contribution from electrodes in the parieto-occipital regions. The demonstrated utility of human EEG for such tasks opens up avenues for future efforts that scale to larger datasets under diverse stimuli conditions with the promise of stronger effects. 
  





§ INTRODUCTION




Despite the remarkable performance of artificial neural networks (ANNs) in object recognition <cit.>, ANNs are sensitive to small so-called adversarial perturbations in the inputs <cit.>. Since the initial discovery of this vulnerability, the field has moved rapidly to devise various adversarial defenses against it<cit.>.


In contrast, human perception is robust to adversarial attacks that are detrimental for ANNs<cit.>. This inspires the idea that adding more bio-inspired elements into ANNs might help alleviate their sensitivity to adversarial attacks and make them more robust. These biological inductive biases often are architecture-based, optimization-based or both <cit.>. Studies have also directly constrained the ANN representations with their biological counterparts, often using neural data from rodents or non-human primates under non-ecological conditions as regularizers<cit.>. These attempts have led to promising, but modest gains in the robustness of the resulting ANNs <cit.>. Yet, the cost and the practical challenges associated with acquiring such data also limits the diversity of stimuli used, often restricting the images from small datasets like CIFAR in grayscale, even for approaches relying on fMRI brain data<cit.>.



Thus, in this work, we used a large-scale EEG dataset collected on diverse real-world images with the aim to improve ANNs robustness to adversarial attacks. Specifically, we experimented with the EEG dataset collected by <cit.> on participants viewing images from the THINGS dataset<cit.>. We report improvements against adversarial attacks which, though modest, were observed consistently across different random initializations of various architectural variants. These robustness gains were positively correlated with the ability of the models to predict EEG at early time points. Interestingly, as already observed in previous work, we also report similar robustness gains when using shuffled versions of the EEG data. While our results do not give state-of-the-art robustness, they provide important pointers guiding future research. We further analyze the EEG dataset across individual channels to investigate any channel-specific effects. We observe that mid-level channels (PO7, PO3, POz, PO4, PO8), though not as well-predicted by the ANNs as the early channels (Oz, O1, O2), are better (positively) correlated with the gains in robustness.
To further facilitate investigations into these methods, we publicly provide the code to the broader scientific community[Code available at: <https://github.com/cvai-roig-lab/eeg_cotraining_robustness>].




§ RELATED WORK


Various efforts have aimed to imbue deep neural networks (ANNs) with human-like cognitive abilities by training them using brain data. Khosla <cit.> showcased that ANNs optimized to predict fMRI activity in the fusiform face area (FFA) and extrastriate body area (EBA) could detect `faces' and `bodies', despite lacking direct exposure to such images during training. Fu <cit.> illustrated that aligning CNN representations with human fMRI data can improve model's performance in video emotion recognition tasks.


Among these, some studies have focused specifically on improving the adversarial robustness of object recognition models using brain data. Li  <cit.> recorded mice's neural response in primary visual cortex (V1) with photon-scans and then, along with classification, use it to penalize the representations of ResNet18<cit.>,  resulting in reduced vulnerability to white-box adversarial perturbations and gaussian noise.


Federer <cit.> used publicly available recordings from V1 in monkeys using micro-electrode arrays<cit.> and similarly penalized representations of VGG with RSA, observing a brain-like response to white-box adversarial noise and label corruption. In a similar fashion, Safarani <cit.> first jointly trained VGG19 for classification and predicting neural data collected in monkeys' V1, enhancing the models' robustness against 14 image distortions. Besides, they showed that their co-trained models were sensitive to salient regions of objects, reminiscent of V1's role in detecting object borders and bottom-up map.


Pirlot <cit.> argued that the concurrent methods used for penalizing the representations could be limited, and instead proposed a Deep Canonical Correlation Analysis (DCCA)-based regularization, observing a reduction of vulnerability against adversarial noise. Dapello <cit.> employed Centered Kernel Alignments (CKA) to penalize representations, leveraging monkeys' data in Inferior temporal cortex (IT) to improve robustness against white-box attacks and align with human behavior error patterns. Based on the findings from Stringer et al<cit.> that, regardless of the input, the eigenspectrum of covariance matrix of the neural code (in mice's visual cortex) followed a power law, Nassar <cit.> inquired whether a similar constraint on the ANN representations might help in enhancing their robustness. They found that, when implemented on smaller convolutional neural networks, their proposed spectral regularization improved robustness against L_∞-FGSM and Projected Gradient Descent attacks<cit.>.

Unlike ours, most studies have restricted themselves to using brain data collected from non-human animals using costly invasive techniques. We note that a contemporary study took a similar rationale to ours and introduced a multi-layer alignment framework to align ANN representations with human EEG<cit.>. Using the same EEG dataset, they trained an additional multi-layer module to predict image categories and human EEG from CORnet-S features (pretrained on ImageNet). While they tested their networks on FGSM attacks, which are known to be quite weak and prone to gradient-masking<cit.>, their main focus was to demonstrate the utility of the learned representations for neuroscience—to better explain fMRI and behavioral data. 








§ METHODS




  
Dataset We used a publicly available dataset containing images and corresponding EEG recordings from 10 subjects viewing images from the THINGS database <cit.>. The training set included 16,540 natural images across 1,654 object categories, with each category containing 10 images presented in 4 separate runs. We split the training set into 9:1 split for training and validation, allocating 9 images per category for training (14,886 images total) and 1 image per category for validation (1,654 images total). The raw EEG signals were epoched from 200ms before to 800ms after stimulus onset and down-sampled to 100Hz. Seventeen channels were selected, including `Pz', `P3', `P7', `O1', `Oz', `O2', `P4', `P8', `P1', `P5', `PO7', `PO3', `POz', `PO4', `PO8', `P6', and `P2', which record signals from the occipital and parietal cortex where the visual signals are the strongest. Further details on EEG-image pair preprocessing are in <Ref>.




  
Architectures and training

The DTL networks consisted of an image classification branch (a ResNet50 backbone) and an EEG prediction branch. Along with certain layers that were shared from the classification branch, the EEG prediction branch comprised of an independent component of either dense linear layers (CNN), recurrent layers (RNN), Transformer, or attention layers that were appended to the ResNet backbone. Overall 24 models (see <Ref> for additional details) were trained for two objectives—classification and EEG prediction. For the classification branch, 1654 object-categories were used. For EEG prediction, MSE loss was applied to predict the EEG data consisting of 100 timepoints (output of size  1  image × 17  channels × 100  tps). To balance the losses between these tasks, we applied the following total loss function from <cit.> :  


    L(W, δ_1, δ_2) = 1/2 δ_1^2 L_1(W) + 1/2 δ_2^2 L_2(W) + log δ_1 +log δ_2.


Here, L_1 and L_2 represent the EEG prediction and image classification losses, respectively, with 1/2 δ_1^2 and 1/2 δ_1^2 as loss coefficients. These parameters, along with the model weights W were updated using the Adam optimizer with a learning rate of 5e-6 and a weight decay of 0.0. The image classification branch was pre-trained on ImageNet, and the EEG prediction branch was initialized with three different training seeds (0, 17, and 337). Each model was trained for 200 epochs with a batch size of 64. As control experiments, we also trained the networks on three simulated EEG datasets obtained by shuffling the original, and randomly drawing from a geometrical or normal distribution. The models trained with such data are labeled as DTL-shuffled,  DTL-random, and DTL-random-normal and were contrasted with those trained on original EEG data (DTL-real). Additional details regarding all the architectures and model training can be found in <Ref>. 




  
EEG prediction evaluation

Following <cit.>, we evaluated the EEG prediction results by measuring Pearson correlation between the predicted and the actual EEG. A PCC matrix (of shape 17 × 100) was constructed where each element represented the linear correlation between the predicted and actual EEG. We then averaged across the channel dimension to get a global value. Results were averaged across the 10 subjects and models initialized with three random seeds. Further details are available in <Ref>.




  
Adversarial robustness evaluation and Robustness gain Adversarial perturbations are image transformations capable of fooling ANNs while remaining imperceptible for humans. To assess the adversarial robustness of our models, we employed Foolbox <cit.> to create adversarial versions of the 1654 original validation images under different attack strengths ϵ. In particular, 1654 adversarial examples with each ϵ value were fed into each DTL model to obtain the the top-1 classification accuracy, denoted as acc_DTL(ϵ). Similarly, we obtained acc_baseline(ϵ) for the baseline model–the ResNet50 model trained for classification. The adversarial robustness gain was defined as Gain_DTL(ϵ) = acc_DTL(ϵ)- acc_baseline(ϵ). We applied L_2-
and L_∞-norm bounded untargeted projected gradient descent (PGD) <cit.>, and L_2 Carlini & Wagner (C&W) attack <cit.>, as described in <Ref>. 




  
Correlation between adversarial robustness gain and EEG prediction
We co-trained 24 architectures on EEG from 10 subjects using 3 training seeds, resulting in a total of  24 × 10 × 3 = 720 samples for correlation analysis. The mean adversarial robustness gain Avg_Gain_DTL was computed by averaging Gain_DTL^n(ϵ) across subjects and training seeds (0 ≤ n ≤ 720) as well as attack strength ϵ. To identify significant time points (tps) contributing to robustness gain, we averaged PCC (ci,tps) across all channels (ci, channel index) and time points (tps) within an optimal sliding window size, resulting in Avg_PCC_tps, which denotes mean prediction accuracy of all channels within the window. The sliding window size was optimized to encompass as many significant time points as possible, further details of which are provided in <Ref>. 


After identifying these significant time points, we investigated key electrodes by similarly measuring correlation between robustness gain and mean prediction accuracy per channel across critical time points.











§ RESULTS






  
Adversarial robustness gains were positively correlated with the models' EEG prediction

We first investigated if there is a relationship between the model's EEG prediction ability and its gains in adversarial robustness. For this, we measured the correlation between the gains in adversarial robustness (Gain_DTL(ϵ)) of the 720 models considered here and their EEG prediction (AVG_PCC_tps); see Figure <ref>A. We observed significant positive correlation values (between 0.53-0.61, p-values < 1e-6) implying that the robustness of the models scaled with the ability of the networks to better capture the statistics from the neural data. We note that similar results were obtained by <cit.> albeit for neural data from individual neurons in the macaque brain. Concerning the role of architecture, we observed that evaluations that combined the features from both the 3rd and 4th block obtained good results for both adversarial robustness and EEG prediction.


We further illustrate the robustness gains of our most robust model (in Figure <ref>B and D). This network (shown with a red arrow in Figure <ref>B) concatenated the features from both the 3rd and 4th block. We observed highest correlations of  0.32 around 100 ms post-stimulus onset (Figure <ref>C), that is at the time of highest discriminability in the EEG data<cit.>. In Figure <ref>D, we show the robustness of this model against all the attacks used in our analysis. We also include the robustness gains obtained after training the model with the control (shuffled and random) versions of EEG . While these also showed some gains in robustness (as also reported in previous works), the model trained with the real EEG showed the highest robustness gains. While the gains are modest—a clear limitation of our results and similar to those reported in earlier studies—they are nonetheless surprising given the high strengths of the attacks (epsilon budget of 1. in L2 norm). Moreover, they demonstrate the potential for the utility of human EEG for rendering robustness. 









  
Electrodes from mid-level EEG channels contribute most strongly to robustness

To find out which EEG channels mostly contributed to robustness, we first determined the EEG channels that were best predicted after our dual-task training. We measured the correlation between the original and predicted EEG, and observed that the models best predicted the data from early occipital channels (Oz, O1, O2) with the group of parieto-occipital channels (PO7, PO3, PO4, PO8, POz) being second best (<Ref>A). These EEG channels overlay the visual cortex (see <Ref>B), consistent with the origin of the observed signals<cit.>. 


But do these channels actually contribute to the robustness of the networks? To ascertain that, we measured the correlations between the robustness gains for each attack and the PCC values for each individual channel (<Ref>C). While the channels in the early visual areas seemed to be best predicted, those in the parieto-occipital region (from electrodes PO7, PO3, PO4, PO8, POz) showed the highest correlation values, indicating that it were the statistics from these channels that particularly aided in enhancing the robustness of the networks. This suggests that brain signal from the later visual processing stages in the human brain contribute more to the robustness than earlier processing stages.





§ DISCUSSION AND CONCLUSION



In this study, we explored the effectiveness of human EEG to render robustness to ANNs. Specifically, we co-trained the ANNs to predict human EEG signals in addition to image classification, and tested their robustness to adversarial perturbations. We observed consistent robustness gains across different variants of the networks. Our investigations revealed a positive correlation between a model's robustness and its ability to predict the EEG. We further teased apart the contribution from individual EEG channels, and observed that though the channels overlaying the early visual cortex were best predicted (with Oz even reaching the upper estimates of the noise ceilings), the ones in the parieto-occipital region correlated better with the gains in adversarial robustness. 



Our work validates the use of human EEG data for enhancing the robustness of ANNs which, compared to intracranial recordings that were often used previously, is cheaper and easier to collect. Given that there is an ongoing trend in NeuroAI <cit.> to collect massive datasets, our methods could not only benefit from the new datasets, but also inform future data collection process. Future works could investigate if larger EEG datasets, collected with different stimulus conditions, say from the auditory domain, can similarly help in improving the robustness of artificial neural networks. 



As reported in previous works, the robustness gain was consistent, yet modest. This could be because of the exact methods that we used to regularize our networks, as also suggested by <cit.>, or could be the inherent limitations of this approach in itself. Like earlier works, we found that the control (shuffled and random) versions of the EEG, and thus the mere statistics of the signal, also helped in rendering robustness to the ANNs. Indeed, this consistent observation, now observed across intra- and extracranial neural activity, deserves future scientific inquiry in its own right. Exactly what (statistical) elements of the neural activity are the networks utilizing to improve their robustness? Can we use these over conventional initialization methods to improve robustness of ANNs? These questions raise the need and guide future research efforts.





§ ACKNOWLEDGEMENTS

This project was funded by the German Research Foundation (DFG) - DFG Research Unit FOR 5368 (GR) awarded to Gemma Roig, Deutsche Forschungsgemeinschaft (DFG; CI241/1-1, CI241/3-1, and CI241/7-1) awarded to Radoslaw Cichy, and a European Research Council (ERC) starting grant (ERC-2018-STG 803370) awarded to Radoslaw Cichy. We are grateful for access to the computing facilities of the Center for Scientific Computing at Goethe University and Freie universität Berlin. M. Guo is supported by a PhD stipend from the China Scholarship Council (CSC).







splncs04




§ APPENDIX





 §.§ EEG-Images pairs





  
EEG-Image pre-processing
The raw EEG signals were first epoched into trials ranging from 200ms before the stimulus onset (denoted as -0.2s) to 800ms after the stimulus onset (denoted as 0.8s) and later down-sampled to 100Hz. 17 channels overlying occipital and parietal cortex where the visual signals are strongest were finally selected. Consequently, our EEG data matrix for model training is of shape (14,886  images × 4  trials × 17  EEG  channels × 100  EEG  tps (time  points)) and our EEG data matrix for model validation is of shape (1654  images × 4  trails × 17  EEG  channels × 100  EEG  tps). As EEG signals are noisy, we averaged EEG data across the trial dimension and normalized it across the temporal dimension with Z-score. All the image stimuli were normalized and resized to 3 × 224 × 224  pixels. 








 §.§ Dual task learning (DTL)





  
Architecture
In CNN cluster, we concatenated or averaged output from different ResNet50-blocks in the shared net and then used fully connected layers to map from image features to EEG signals directly. In RNN cluster, shallow RNNs where feedback were realized with Resnet-like skip connections, or Long short-term memory (LSTM) units were integrated with the shared net, as recurrence is capable of capturing temporal dynamics and patterns in time-series data. In Transformer cluster, 6 layers of transformer encoders with 32 multi-heads, dropout value of 0.5 and embedding dimension of 256 were combined with the shared net, as transformer architectures have been shown to be beneficial for fMRI and MEG prediction <cit.>. In Attention layer cluster, 2 self-attention layers were considered. One is a self-attention layer in <cit.>. We used 4 multi-heads and an embedding dimension of 256 after optimization experiments. Another utilized the position attention modules (PAM) and channel attention modules (CAM) used in <cit.>. We followed the default settings. All the 24 architectures from the 4 clusters are included in <Ref>. Some architectures from the 4 clusters are depicted in <Ref>, <Ref> and <Ref>.















  
Model training
<cit.> used the total loss function in Equation<Ref> to balance the loss updating during dual task learning. 

    L(W, δ_1, δ_2) = 1/2 δ_1^2 L_1(W) + 1/2 δ_2^2 L_2(W) + log δ_1 +log δ_2

Where L_1 and L_2 were EEG prediction and image classification loss, respectively. 1/2 δ_1^2 and 1/2 δ_1^2 were loss coefficients. 1/2 δ_1^2, 1/2 δ_1^2, L_1 and L_2 were updated using Adam optimizer, with a learning rate of 5e-6 and a weight decay of 0.0. The learning rate and weight decay values were optimized with cross validation. W denoted weights of model. The image classification branch was pre-trained on Imagenet and the independent net of EEG prediction branch were initialized with 3 different pre-chosen training seeds (0, 17 and 337).  




 §.§ Evaluation of EEG prediction and adversarial robustness





  
EEG prediction evaluation

Given the hierarchical visual processing in the brain, EEG signals at different time points may capture activities in different regions. We used Pearson correlation analysis to evaluate EEG predictions at various time points using 1654 images from the validation set. The Pearson correlation coefficient (PCC) at each time point was computed as follows: (1) By looping through the channel and temporal dimension, we measured the PCC between predicted EEG and biological EEG at different channel index (ci) and time points. Consequently, we obtained a PCC array of size 17 × 100, denoted as PCC(ci,tps). (2) We averaged the PCC(ci,tps) across the channel dimension and obtained PCC(pts), which denoted linear relationship between predicted and biological EEG at different time points. Here, pts ∈ [-0.02s,-0.01s,...,0.07s,0.08s]. The formula for PCC was given as follows: PCC = cov(signal_1,signal_2)/σ_1,σ_2 , where cov denoted the covariance while σ_1 and σ_2 denoted the standard deviation of signal_1 and signal_2.



  
Adversarial examples generated with PGD

Projected Gradient Descent, or PGD, iteratively constructs adversarial examples as follows : 


    x^t+1 = Proj_x+S(x^t + α sgn(∇_x L(θ,x^t,y)))


where θ is the parameters of models, x^t is the input and y is the associated label. L(θ,x^t,y) is the loss of model training and ∇_x L(θ,x^t,y) is the gradient of loss L with respect to input x. α is the gradient step size. t is the number of steps for iteration. x^t and x^t+1 are adversarial examples before and after next iteration, respectively. The Proj is an operator to construct x^t+1 within space x+S, such as l_∞ ball or l_2 norm ball around x. In our setting, t was 50 and 40 for l_2-bounded PGD and l_∞-bounded PGD, respectively. In l_∞-constrained PGD, the attack strength ϵ∈ [1e^-5,2e^-5,3e^-5,4e^-5,5e^-5,6e^-5,7e^-5,8e^-5,1e^-4,3e^-4,5e^-4,7e^-4,8e^-4
,1e^-3,8e^-3,1e^-2] and the step size relative to ϵ was 0.01/0.3. In the l_2-constrained PGD, the attack strength ϵ∈ [1e^-3,5e^-3, 7e^-3, 1e^-2, 2e^-2, 3e^-2, 5e^-2, 7e^-2, 1e^-1
, 2e^-1, 3e^-1, 5e^-1, 7e^-1,1.0] and the step size relative to ϵ was set to 0.025.



  
Adversarial examples generated with  Carlini & Wagner (C&W) attack
C&W attack formulates the generation of adversarial examples as an optimization problem, finding the smallest perturbations to the input data that causes misclassification of the target model. C&W attack defines the objective function J(x^') as follows:

    J(x^') = α· dist(x,x^') + β· loss(f(x^'),y)

Where x is the original image; x^' is the perturbed image; In the case of L2 C&W attack, dist(x,x^') measures the perturbation using the l_2 norm. loss(f(x^'),y) represents the misclassification loss of target model f on the perturbed input with respect to the target class y. α and β are weights to balance the dist(x,x^') and loss(f(x^'),y). The C&W attack iteratively adjusts the perturbations to improve the chances of misclassification while keeping the perturbations imperceptible. Thus the term dist(x,x^') is minimized and loss(f(x^'),y) is maximized. Gradient descent is used for optimization. The perturbation is updated until it converged towards an examples x^' = x^' - η·∇_x^'J(x^'), where η is the step size. The attack strength ϵ∈ [1e^-5,7e^-5,1e^-4,7e^-4,1e^-3,1e^-2,1e^-1,3e^-1,5e^-1,7e^-1,
9e^-1,1.0,1.2,1.4,1.6,1.8,2.0,2.2,2.4,2.6,2.8,3.0] and the step size relative to ϵ was 0.01. 



 §.§ Mean adversarial robustness gain and mean EEG prediction accuracy


Mean adversarial robustness gain and mean EEG prediction accuracy were used to measure the relationship between adversarial robustness gain and EEG prediction accuracy. We trained 24 architectures from all 4 clusters on EEG of 10 subjects with 3 training seeds. As a result, we had totally 24 × 10 × 3 = 720 sample models for analysis. 



  
Mean adversarial robustness gain Avg_Gain_DTL
For each sample model n (0 ≤ n ≤ 720), we first computed Avg_Gain_DTL^n which was the averaged value of Gain_DTL^n(ϵ) across selected attack strength. In C&W, Gain_DTL(ϵ) used for Avg_Gain_DTL^n calculation included those obtained under attack strength ϵ∈ [5e^-1,7e^-1,9e^-1,1.0,1.2,1.4,1.6,1.8,2.0,2.2,2.4,2.6,2.8,3.0]. In l_∞-constrained PGD, Gain_DTL(ϵ) used for Avg_Gain_DTL^n calculation included those obtained under attack strength ϵ∈ [8e^-5,1e^-4,3e^-4,5e^-4, 7e^-4,8e^-4,1e^-3,8e^-3,1e^-2
,1e^-1]. In l_2-constrained PGD, we considered the attack strength ϵ∈ [7^e-2,1e^-1
,2e^-1,3e^-1,5e^-1,7e^-1,1.0]. We selected high ϵ values because under high attack strength, robust models would perform much better than fragile ones in classifying adversarial examples, which enabled us to better quantify relationship between EEG prediction accuracy and robustness gain. We finally had 720 Avg_Gain_DTL^n values and the Avg_Gain_DTL was computed by averaged Avg_Gain_DTL^n across 3 training seeds and 10 subjects.






  
Mean EEG prediction accuracy Avg_PCC_tps and optimal sliding window selection
Based on our initial experimentation, we posited that higher prediction accuracy of EEG signals around 100ms might result in increased gains in adversarial robustness. To rigorously investigate this association, we initially deployed three sliding windows covering the vicinity of 100ms: 0.10s to 0.12s, 0.09s to 0.14s, and 0.05s to 0.3s, respectively. Within these windows, for each model iteration (n), we computed Pearson correlation coefficient (PCC) values (PCC^n(pts)) as described in <Ref>, subsequently averaging these values (Avg_PCC^n) across all time points within the sliding window. By averaging Avg_PCC^n values across 10 subjects and 3 training seeds, we derived the Avg_PCC_tps. Assessing the correlation between Avg_Gain_DTL and different Avg_PCC_tps obtained with the three window sizes, we identified an optimal window size of 0.06s, resulting in the highest correlation value between Avg_Gain_DTL and Avg_PCC_tps around 100ms. Subsequently, we proceeded to identify the most informative EEG signals crucial for robustness gain by moving this optimal sliding window across all time points, with a step size of 0.01s, and measuring the correlation values between Avg_Gain_DTL and Avg_PCC_tps across all time points. In total, we obtained 100 correlation values, which represent the correlation between Avg_Gain_DTL and Avg_PCC_tps when the optimal sliding window arrives at different time points. Similarly, to discern the most significant channels for enhancing the robustness gain, after identifying these significant time points, we averaged PCC(ci,tps) across critical time points, yielding the averaged prediction accuracy across critical time points for each channel. 



§ ADDITIONAL RESULTS





 §.§ Relationship between Avg_Gain_DTL and Avg_PCC_tps within different sliding windows



In <Ref> (A), the correlation values between Avg_Gain_DTL and Avg_PCC_tps within two specific sliding windows, one ranging from 0.05s to 0.3s and the other from 0.10s to 0.12s, were inferior to those in <Ref> (B), which suggested that using a sliding window size of 0.06s allowed us to capture more significant EEG signals around 100ms for adversarial robustness improvements. Using this optimal window size, <Ref> rigorously examines the correlation between EEG prediction accuracy and adversarial robustness gain. 








