Paper Title: Diffusion-Based Generation of Neural Activity from Disentangled Latent Codes.



Based Dynamic Noise Mapping with Machine Learning for Urban Noise Enforcement
    
		
			H. Emre Erdem
			, Henry Leung
		
		This work was supported in part by the Alberta Jobs, Economy and Innovation Ministry’s Major Innovation Fund to the Centre for Autonomous Systems in Strengthening Future Communities and in part by the Natural Sciences and Engineering Research Council of Canada.
	
    
======================================================================================================================================================================================================================================================================================================================




§ ABSTRACT 
 Recent advances in recording technology have allowed neuroscientists to monitor activity from thousands of neurons simultaneously. Latent variable models are increasingly valuable for distilling these recordings into compact and interpretable representations. Here we propose a new approach to neural data analysis that leverages advances in conditional generative modeling to enable the unsupervised inference of disentangled behavioral variables from recorded neural activity. Our approach builds on InfoDiffusion, which augments diffusion models with a set of latent variables that capture important factors of variation in the data. We apply our model, called Generating Neural Observations Conditioned on Codes with High Information (GNOCCHI), to time series neural data and test its application to synthetic and biological recordings of neural activity during reaching. In comparison to a VAE-based sequential autoencoder, GNOCCHI learns higher-quality latent spaces that are more clearly structured and more disentangled with respect to key behavioral variables. These properties enable accurate generation of novel samples (unseen behavioral conditions) through simple linear traversal of the latent spaces produced by GNOCCHI. Our work demonstrates the potential of unsupervised, information-based models for the discovery of interpretable latent spaces from neural data, enabling researchers to generate high-quality samples from unseen conditions.




§ INTRODUCTION



New neural interfaces enable the simultaneous recording of hundreds to thousands of neurons <cit.>. Interpreting the resulting high-dimensional datasets is a major challenge, as neural activity may have non-linear and time-varying relationships to many variables of interest. Thus neuroscientists have begun to rely on latent variable models to distill high-D recordings of neural population activity into compact and more easily interpretable representations. Of particular interest are generative latent variable models, which allow neuroscientists to map back from the latent variables to the high-D population activity itself to study how variables of interest are geometrically encoded within neural activity.

Brains can produce a rich behavioral repertoire, making it challenging to know what variables are encoded by a particular brain region or neural sub-population. Traditionally, neuroscientists assess how variables are encoded in a brain region by designing experiments that constrain behavior to a small set of predetermined variables of interest. An alternative approach uses latent variable models to discover which variables are encoded in a neural circuit and how those variables are encoded at the level of individual neurons. While auxiliary variables (observed behavior) may be used to structure latent spaces <cit.>, such approaches are not possible when key variables are unobserved or unknown. To promote more general discovery of latent structure, models would ideally have the following properties: unsupervised application to complex and diverse time-series data, structured latent spaces that are disentangled with respect to behavioral variables of interest, and queryable, conditional generation of high-quality samples for unseen behavioral conditions. 

We propose a new approach to neural data analysis that builds on advances in conditional generative models to enable the unsupervised inference of disentangled behavioral variables from recorded neural activity. Our approach uses InfoDiffusion <cit.>, which augments diffusion models with a set of latent variables (here termed “codes”) that capture important factors of variation in the data. By maximizing the mutual information between the observed and latent variables, InfoDiffusion has been shown to learn disentangled representations of human-interpretable latent factors.

We extend InfoDiffusion to sequential data and test its performance on synthetic and biological datasets from neural circuits performing a motor task. We compare our approach, Generating Neural Observations Conditioned on Codes with High Information (GNOCCHI), to an alternative unsupervised generative latent variable model for neural time-series data, latent factor analysis via dynamical systems (LFADS). While both models achieve high-quality generation of neural data, GNOCCHI achieves higher-quality latent spaces that are more clearly structured with respect to behavioral variables. Moreover, the latent spaces produced by GNOCCHI are more disentangled than those produced by LFADS, such that simple linear traversal of the latent space along identified paths produces novel samples with isolated changes in their relationship to behavior. Finally, we show that these properties enable high-quality conditional generation of neural activity for unseen behavioral conditions.




§ RELATED WORK




Generative models have frequently been used to uncover latent, low-dimensional descriptions of neural population activity, including models based on Gaussian processes <cit.>, variational autoencoders (VAEs) <cit.> including those using self-supervised learning <cit.>, neural ordinary differential equations <cit.>, and recent diffusion-based approaches<cit.>. In this work we chose a sequential autoencoder, latent factor analysis via dynamical systems (LFADS <cit.>), as our point of comparison, as it reduces time series neural data to a latent representation whose structure is determined by a standard VAE objective <cit.>. LFADS allows samples to be generated for novel conditions through a simple interpolation approach we detail in Section <ref>. We note an alternate point of comparison could be SwapVAE <cit.>, which decomposes its latent space into qualitatively different latents for “style” and “content”; in this case we chose LFADS due to its frequent applications to neural time series data <cit.>.

Prior work in interpretable deep generative models explicitly conditions generation on behavioral labels <cit.>. These approaches may allow identifiable and interpretable recovery of latent structure for observed variables, but are not applicable when key variables are unobserved or unknown.

Generative models have been used in other domains to produce samples from outside of the training set without access to the underlying variables of interest. This has been achieved with success on image datasets through modifying the generative cost of VAEs <cit.>. Another popular class of approaches has been to regularize the mutual information between the observed data and the latent space of the generator <cit.>. Through interpolation and extrapolation in the latent space, these models can generate samples that reflect an interpretable transition between variables of interest. However, to our knowledge these information-based deep generative models – which do not rely on behavioral labels – have not previously been applied to neuroscientific datasets.





§ METHODS 





 §.§ Artificial neural datasets 




To generate realistic neural activity on which to test our models, we trained an RNN (128-hidden unit GRU), to perform planar reaching tasks using a 2-link, 6 muscle arm model <cit.>. The RNN model received a 17-dimensional input: endpoint coordinates (2D), muscle lengths (6D) and velocities (6D), target coordinates (2D) and a go cue input (1D). The loss function of this model was to minimize the mean-squared error between the effector endpoint and the target, while minimizing the weighted squared muscle activation. This training paradigm produced a biologically plausible set of neural activations with associated task parameters that can be used as a benchmark for evaluating the latents/codes produced by the different methods. More details on the inputs and outputs of the RNN are shown in Figure <ref>A, with full training details given in Section <ref> of the Appendix. 

We consider a random target task, where the RNN must control the set of 6 muscles to manipulate the effector endpoint from a start location to a target location, both of which are randomly selected from a 6 × 6 grid. At a random time during the trial, a target was presented, followed by a delayed go cue that instructed the RNN to initiate movement towards the presented target. 

To create a training dataset for our generative models, we generated 1000 total trials of RNN activity from each task. We then aligned to a window around the go cue time, producing trials of length 0.4 seconds (40 bins). When assessing the generalization of the representations learned by the generative models, we would remove trials that had specific target locations. For the random target task, we held out trials corresponding to target locations on the middle square of grid points (see Figure <ref>A). These trials were used as a "held-out" dataset to quantify how well our models could generate data from conditions that were not in their training set. Additional details can be found in Tables <ref> and <ref>.




 §.§ Biological neural datasets 



To assess whether our approach works for biological neural recordings, we used previously published recordings from a multi-electrode array <cit.> implanted in monkey primary motor cortex (M1) during a self-paced random target reaching task <cit.>. Voltage signals from each of the 96 electrodes were converted into threshold crossings, binned at 20 ms, and aligned to a window [-200, 500] ms around movement onset. When assessing the generalization capability of the generative models, we held-out reaches to targets in the upper-right quadrant (∼14% of the trials, see Fig. <ref>). We analyze the generation quality of these data when testing the ability of our models to generate accurate examples of neural activity from outside its training set. For modeling this dataset, we used spiking activity with LFADS, and smoothed spiking activity with GNOCCHI (see Section <ref> for more details).



 §.§ Model Architectures 






  §.§.§ Generating Neural Observations Conditioned on Codes with High Information 

Our model is an adaptation of InfoDiffusion <cit.> that operates on time-series data. The model consists of two networks: the auxiliary variable encoder, and the noise prediction network. The architecture of both the auxiliary encoder and the noise predictor is a bidirectional GRU with a linear readout to the desired output dimensionality. The auxiliary variable encoder is passed a window of neural activity {n_t}_t=1^T ∈ℝ^T × N as input, and outputs the latent representation vector, or code, c∈ℝ^L. The code is penalized against a standard normal prior using Maximum Mean Discrepancy (MMD) <cit.>. This code c is used to condition a diffusion process, wherein the noise prediction network is trained to predict the noise that is added to the data during the forward process for any step sampled randomly from the sequence of steps in the total diffusion process, consistent with the score matching objective and training procedure described in <cit.>. The forward process is defined by Equation <ref>:

    x̃_i = √(α̅_i)x_0 + √(1-α̅_i)ϵ ;  ϵ∼𝒩(0,1)

Where x_0 is a window of neural activity, x̃_i is the noised neural data sample at step i of the forward process, α̅_i is the cumulative product of the coefficients α_i = 1-β_i up to diffusion step i, where the parameters β_i ∈ [0.001, 0.01] are fixed and define a linear noising schedule of the forward process.

In addition to the MMD and score matching costs, there is a cost on the fidelity of the reconstructed neural data sample x̂_0 computed using the output of the noise predictor ϵ̃ and the noised neural data sample  x̃_̃ĩ. In total, the relevant quantities for training the model are given by Equations <ref>-<ref> :

    c   = AuxEncoder({n_t}_t=1^T) 
    ϵ̃_i    = NoisePredictor(x̃_i, i, c) 
    x̂_0    = 1/√(α̅_i)(x̃_̃ĩ - √(1-α̅_i)ϵ̃_i)

Note that Equation <ref> is simply a rearrangement of Equation <ref> with the reconstructed neural data sample x̂_0 used instead of the original neural data sample x_0 and the predicted noise ϵ̃ used in place of the sampled noise ϵ. In total, the loss terms computed based on these quantities are given by Equations <ref>-<ref>:

    ℒ_score   = 𝔼[||ϵ_i - ϵ̃_i||_2^2] 
    ℒ_recon   =  𝔼[||x_0 - x̂_0||_2^2] 
    ℒ_MMD = 𝔼_c_prior∼𝒩(0,1)[k(c_prior,c_prior)]    + 𝔼_c∼AuxEnc[k(c,c) ] - 2𝔼_c_prior∼𝒩(0,1), c∼AuxEnc[k(c_prior,c) ]

Where k(·, ·) : ℝ^L ×ℝ^L →ℝ is the Gaussian radial basis function kernel.

In total, the loss function is the weighted sum of ℒ_MMD, ℒ_score and ℒ_recon. The weightings of these terms as well as the rest of the hyperparameters used for the experiments are given in Section <ref>. The weighting and implementation of the MMD term is computed analogously to the InfoVAE implementation of <cit.>. Additional details on the model architecture and hyperparameter selection are also in Section <ref>. A model schematic showing the training and inference pipelines for GNOCCHI is shown in Figure <ref>, as well as low-dimensional visualizations of generated neural activity when conditioning on the codes, versus sampling from the code prior (Fig. <ref>C).



 §.§ Latent Factor Analysis via Dynamical Systems (LFADS) 


LFADS<cit.> is a sequential adaptation of the variational autoencoder (VAE) <cit.> that incorporates inductive biases from the dynamical systems perspective <cit.> into an end-to-end latent dynamics model of neural activity. LFADS utilizes recurrent neural networks (RNNs) to read over neural activity and produce a summarizing vector representation, which is used as the initial condition of a dynamical system that is learned and unrolled by another RNN. The resulting architecture is capable of learning variational parameters that can be used to encode a dynamics-based representation of the neural recordings, expressed using a lower-dimensional set of latent variables. We trained LFADS models with the initial condition representation matched to the same dimensionality as the representations used with GNOCCHI. Additionally, we turned off the LFADS controller, since the appropriate strategy for taking time-varying inputs into account when performing latent navigation (see Section <ref>) is unclear. More details on the training, architecture choices and hyperparameters used with LFADS can be found in Section <ref>. 




 §.§ Generating neural activity reflecting unseen conditions via Latent Navigation 



To generate condition-specific samples from the latent space learned by the generative models, we developed a simple method to find locations in the latent space that corresponded to trials with specific behavioral features (e.g., for the random target tasks, starting location and desired target location.) This method, termed latent navigation, consists of fitting a linear regression from the latent representations learned by the generative model, c∈ℝ^L to each behavioral feature. We then obtain new samples related to each behavioral feature by using latents obtained by stepping along the directions learned by the linear mapping, starting from a reference point in the latent space, referred to as an “anchor latent". This process is summarized by Equations <ref> and <ref>:

    min_W, b||s - Wc + b||^2_2 
    c̃^(i)_j    = c_anchor± i ·δ c ·w_j

Where s∈ℝ^K is the set of behavioral features, W∈ℝ^K× L is the matrix fit by linear regression with bias b∈ℝ^K, w_j is the j^th row of W, i indicates how many steps are being taken away from the anchor latent, and δ c ∈ℝ is the step size in latent space. In this work, we considered j corresponding to the horizontal and vertical target coordinates. For the artificial random target task, we used δ c = 1.0. 





 §.§ Behavioral Decoding 



To assess the nature of behavior that can be produced by conditionally generating neural activity, we decode behavior from the generated neural activity for both seen and unseen conditions. For the artificial neural datasets, we predicted the hand endpoint position. 
The position decoder f : ℝ^N→ℝ^2 is given by Equation <ref>:

    x_t    = f(n_t)

Where x_t is the endpoint position at time t, n_t is the neural activity at time t. For f, we use ridge regression <cit.>.

The position decoders for the artificial neural response experiments are fit on the ground truth neural responses and behaviors, and we use these decoders to assess the responses produced by the generative models. The ridge regularization parameter of the position decoders are set to 1.0 to mitigate noise differences between the model-produced neural activity and the ground truth RNN hidden states used to train the decoders.



§ RESULTS





 §.§ GNOCCHI learns well-structured codes that reflect task variables

We  aimed to validate three features of the GNOCCHI model: 1) unsupervised discovery of structured codes, 2) that those codes should be disentangled, and 3) that the model would generate more accurate samples for unseen behavioral conditions than alternative models. 



We began by applying GNOCCHI to a complex, biologically-plausible synthetic dataset with known ground-truth activity. We generated this dataset by training an RNN to perform a Random Target reaching task by controlling a 2 DoF manipulandum in a simulation environment (Fig. <ref>A). After training, individual RNN units had complex, time-varying activity patterns that were well-modulated by task parameters (Fig <ref>B, top row, traces colored by angle of reach) and qualitatively resembled typical response patterns of motor cortical neurons. We fit GNOCCHI and LFADS models to this synthetic neural activity and found both models were able to accurately reconstruct single unit activities (Fig. <ref>B, lower rows).

We then examined the latent representations inferred by each model and found that while both sets of latents had clear structure related to the target position, GNOCCHI’s codes appeared more tightly organized than LFADS (Fig. <ref>C). To quantify how tightly the codes were clustered for a particular reach condition, we computed a signal-to-noise ratio (SNR) metric, the ratio of the total variance to the mean intra-condition variance. The SNR of GNOCCHI codes was substantially higher than for LFADS (Fig. <ref>D), suggesting that GNOCCHI learned more structured code representations that better reflected the underlying behavior.



 §.§ GNOCCHI learns disentangled codes that allow for targeted data generation


A disentangled code space allows a generative model to produce new samples in which the relevant task variables encoded by the data change independently from one another. To test whether GNOCCHI learns a disentangled code space, we trained GNOCCHI and LFADS models on a subset of the previous Random Target task data, only including conditions in the middle or on the edges of the workspace (Fig. <ref>A).


Using our latent navigation procedure, we isolated the “Target-tuned” axes of code space, generated artificial neural data with codes associated with changing a single task-variable, and used a position decoder to transform these artificial neural signals into a predicted hand position over the course of a trial. When we generated activity by moving along the target-tuned dimensions of the latent space learned by GNOCCHI, the decoded position varied for the behavioral variable of interest largely in isolation, e.g. with minimal change in the start position when varying along the target location directions of code space (Fig <ref>B, left, right, respectively). 

In contrast, moving along both the X and Y target-tuned dimensions of the LFADS latent space resulted in generated activity for which the decoded start position varied substantially for both target dimensions (Fig. <ref>C, left: Target X, right: Target Y), indicating an entangled representation for start and target locations in the LFADS model. Additionally, interpolating novel samples within the range of heldout targets resulted in decoded trajectories that ended far outside the intended boundary for LFADS, while decoding samples that were generated through GNOCCHI produced trajectories that were largely confined to the intended range.

We quantified these observations in two ways. First, we indirectly measured disentanglement by calculating the pair-wise normalized dot products between the axes of the linear mapping used for latent navigation. When these values are close to zero, the learned representations are nearly orthogonal in the code space. The average projection magnitude was significantly larger for LFADS compared to GNOCCHI (Fig. <ref>D, paired t-test across 10-fold cross validation,.p-val < 2.8e-34). For reference, we include a horizontal line indicating the average magnitude of the normalized dot product for vectors of the same dimension as the target-tuned axes, whose entries were drawn from a standard normal distribution. Second, we calculated the effect of moving along one target-tuned dimension on the other variables. Using our decoders, we quantified how much unintended movement in non-queried variables was observed as we changed the target-tuned code. We found that the unintended movement was substantially larger for LFADS than for GNOCCHI (Fig. <ref>D, paired t-test across 10 different reach interpolations,.p-val < 8.6e-14). Together, these results suggest that GNOCCHI learns a more disentangled representation of task variables than LFADS.

Last, a desired feature of generative models is to obtain high quality generated samples both within the domain of the training and to generalize outside of the training set. To test the ability of our models to generalize, we used neural activity from the held-out conditions to infer codes with the LFADS and GNOCCHI code encoders (Fig. <ref>E). We found that the codes inferred by GNOCCHI were generally closer to the correct location in code space, suggesting that GNOCCHI can generate higher quality samples from unseen conditions than the alternative model (Fig. <ref>F).



 §.§ Biological Neural Data

We applied GNOCCHI and LFADS to neural data from the motor cortex of a monkey performing a similar random-target reaching task as our synthetic dataset (Fig. <ref>A). Both models were able to capture significant features of single trial neural responses (Fig. <ref>B, single trial neural activity color-coded by reach angle) and inferred codes that had significant structure that reflected the underlying task (Fig. <ref>C, codes color-coded by Target X position). 



We first wanted to compare the ability of GNOCCHI to model the neural activity compared to the LFADS; higher predictive performance would reflect a more structured code space. We compared the R^2 values of LFADS to GNOCCHI, and found that on average GNOCCHI made better predictions of the smoothed single trial neural activity across neurons (Fig <ref>D, paired t-test across neurons,.p-val < 1e-4). We also tested whether the representations learned by GNOCCHI were disentangled, using the same methods as applied to the synthetic data (Fig <ref>B, E). 

We next tested the ability of GNOCCHI to generalize to unseen conditions. We trained new GNOCCHI and LFADS models on a subset of target locations, leaving a quadrant of the target locations out of the training set (Fig <ref>E, Heldin, filled circles, Heldout unfilled circles). We then used the neural activity from the heldout conditions to infer codes for each model. When we passed the inferred codes through a decoder trained to predict held-in target position from heldin codes, we found that GNOCCHI codes predicted reaches that extended much further into the heldout targets than did the LFADS codes (Fig <ref>E, blue (top), red (bottom)  ×s, respectively).  We quantified this boost in generalization performance by the distance between the target predicted by the code to the true target of each trial. We found that GNOCCHI codes were more predictive of the true target than LFADS for almost all of the heldout trials (Fig <ref>F, points below unity line were closer for GNOCCHI than LFADS), demonstrating the utility of GNOCCHI for generating out-of-training neural activity relative to LFADS.




§ LIMITATIONS



Aside from some of the comparative advantages and disadvantages of each model, there are some limitations to GNOCCHI as proposed here. One limitation of this work is that the models we considered only generate neural activity, and not also behavior. Without generating behavior and neural activity concurrently, we cannot augment datasets for training neural decoders without making additional assumptions on how the generated neural responses relate to behavior. However, GNOCCHI may be appropriate for decoding applications (e.g., brain-computer interfaces; BCIs) as-is. Typically, in clinical applications with paralyzed individuals, behavior is not directly observed; rather, decoders are trained by making assumptions about the relationship between neural activity and behavior, and those same assumptions could be applied to samples generated by GNOCCHI. However, integrating GNOCCHI with methods that can jointly generate neural activity and behavior from a shared latent representation <cit.> presents a promising next step for this work, though the matter of overcoming the lack of ground truth behavior in clinical settings would remain a challenge. Additionally, the data in this work are aligned to a stimulus (go cue) or movement onset. Achieving interpretable representations that support latent navigation when modeling continuous time-series data (without pre-specified alignment) would enable application to a broader range of behaviors, and validating the efficacy of GNOCCHI in such settings is an important improvement for future work.


This work demonstrates an exciting opportunity for increasing both the generative capability and representational interpretability of neural population models. Such improvements could enable higher-performing BCIs, which are used in rehabilitation settings to enable increased autonomy and quality of life for those with neurological conditions. Given the potential impact of this work on BCI, we encourage appropriate consideration to preserving personal privacy and equity in any such downstream applications.




§ DISCUSSION



GNOCCHI leverages advances from InfoDiffusion to learn structured, disentangled, and generalizable code spaces without requiring behavioral labels. GNOCCHI builds upon previous advances in generative modeling, which have provided powerful new lenses into neural population activity. Such advances will be critical as neuroscientists gain access to high-dimensional neural population recordings from a variety of brain areas during increasingly complex and unconstrained behaviors. The ability of methods like GNOCCHI to interpolate or even extrapolate neural data samples along interpretable latent axes may provide the opportunity to probe high dimensional behaviors with greater sampling efficiency, leading to a deeper understanding of the structure and geometry of neural activity during complex behaviors.





This work was supported by NIH-NINDS/OD DP2NS127291, NIH BRAIN/NIDA RF1 DA055667, and the Simons Foundation as part of the Simons-Emory International Consortium on Motor Control (CP). This work was also supported by NIH BRAIN Initiative grant 5F32MH132175 (CV) and NIH NIBIB T32EB025816 (DM). 

CP is a consultant for Meta (Reality Labs). This entity did not support this work, did not have a role in the study, and did not have any competing interests related to this work. 

The authors would like to acknowledge Brianna Karpowicz for discussions that helped shape the analysis and pre-processing of the biological neural population data.









§ APPENDIX




§ GENERAL DATASET INFORMATION










§ ARTIFICIAL NEURAL ACTIVITY FROM TASK-TRAINED RNNS 




 §.§ Task training details

The architecture of the RNN was a Gated Recurrent Unit (GRU). The RNNs were trained for 1000 epochs with learning rate 1× 10^-3 and no weight decay. The loss function was defined in terms of a weighted combination of the L_1 norm of the effector endpoint distance to the target, and an L_2 norm of the total activation of the muscles during behavior. Together with the variable timing of the go cue and a limited number of steps within a trial (see Section <ref>), these loss terms incentivize timely completion of the task with minimal energy expenditure. The loss in total is given by Equation <ref>:

    ℒ_task = 1/2∑_i||x_i - x_target||_1 + 1/2∑_i||a_i||_2^2

Where x∈ℝ^2 is the position of the effector endpoint, x_target∈ℝ^2 is the target location, and a∈ℝ^6 are the muscle activations. Additionally, a small (10^-3) amount of normally distributed noise was added to the outputs of the RNN to simulate motor noise.


 §.§ Data sizes, alignment, and trial information 

In each task, there were 1000 trials, split 800/200 between training and validation. The RNN has 128 hidden units, and each trial has a duration of 200 time steps. In order to analyze the neural activity of the reaching behavior, we aligned the data to a window of [-10,30] time steps around the go cue. Since the go cue is given at a random time during the trial, any trials for which the go cue placement conflicted with the alignment window were discarded. The resulting number of trials for each task are given in Tables <ref> and <ref>. The resulting datasets used for modeling had trials consisting of 40 timesteps of activity across the 128 dimensions of the RNN during the window around the go cue.




§ GENERATING NEURAL OBSERVATIONS CONDITIONED ON CODES WITH HIGH INFORMATION (GNOCCHI) 





 §.§ Hyperparameter selection 

To select hyperparameters for GNOCCHI, we searched for hyperparameters using a simplified task-trained RNN neural dataset, consisting of a single ring task in which the RNN produced reaches to a set of radially distributed targets starting from a common center point for each reach. We first ran a random search of 200 models, selecting hyper-parameters according to the ranges and sampling procedures given in Table <ref>, with final parameters given in Table <ref>. After this search, we selected the best model according to an even consideration of the MMD and score losses. To select the diffusion-based hyperparameters, we ran a grid search over the ranges given in Table <ref>, with all other parameters fixed. For each grid search, we also searched over 3 random seeds, and the parameter selected was the one for which the model performance was the best, with consideration given to the consistency across seeds. For the final hyperparameters, most of the choices were shared across all the datasets, only varying the number of diffusion steps or the latent dimensionality as needed (e.g. the code dimensionality for the random target task and biological random target task were 5 and 15 respectively, as shown by the bracketed values in Table <ref>). For the biological neural data, we analyzed the trends in MMD and score loss to assess whether the parameters chosen from the synthetic sweeps were effective. We found that increased code dimensionality and a larger number of diffusion steps led to models with better score loss, settling on 15 and 500 for the code dimensionality and number of diffusion steps respectively for modeling the biological neural data.







§ LFADS 



 §.§ Model architecture and training details

For the artificial neural response experiments, LFADS was trained to minimize the mean-squared error between the model output and the RNN hidden states, with KL regularization against a multivariate normal prior with scaling given in Table <ref>. For the biological neural data experiment, LFADS was trained to minimize the Poisson negative log-likelihood of the spiking activity, given the model output, regularized by KL with scaling also given in Table <ref>. For each experiment, LFADS was applied using the same set of hyperparameters, with the initial condition dimensionality for the artificial random target task and biological random target task given in this order inside brackets in Table <ref>.




 §.§ Smoothing the input to GNOCCHI 

For the biological neural data experiments, we used the smoothed spiking activity from an LFADS model as the input to the GNOCCHI-based model. This was due to the GNOCCHI-based model only currently having been validated for MSE-based reconstructions. The hyperparameters of the LFADS model are given in Table <ref>. These hyperparameters were selected according to Binary tournament population-based training <cit.>, as in the AutoLFADS framework <cit.>. As a control on whether the held-out generalization results shown in Figure <ref>E are dependent on the method of smoothing for the input to GNOCCHI, we compare to a GNOCCHI model trained on spikes smoothed with a Gaussian kernel (s.d. 4 bins or 80ms). The results shown in Figure <ref> demonstrate that held-out prediction of target location is comparable between smoothing approaches.










 §.§ Hyperparameter selection

The default parameters specified in <cit.> were used initially, and we found that these parameters work well across each dataset. As a result, we have continued with these parameters, changing only the initial condition dimensionality and turning off use of the controller network to ensure comparable usage to GNOCCHI when doing latent navigation. To select the KL penalty, we did a sweep across 4 values spanning the range [1× 10^-8, 1× 10^-4] across 5 random seeds, choosing the top performing model according to validation set reconstruction performance. The parameters used in each experiment are summarized in Table <ref>.



.




§ COMPUTE RESOURCES

We used an internal computing cluster consisting of a combination of NVIDIA GeForce 2080 RTX Ti and A40 GPUs. The single model runtime for GNOCCHI and LFADS is approximately 30 minutes wall time, and the model parameter file sizes are roughly 450 MB and 0.66 MB, respectively. 




§ OPEN SOURCE PACKAGES USED


    
  *  <cit.> (BSD license)
    
  *  <cit.> (Apache 2.0 license)
    
  *  <cit.> (Apache 2.0 license)
    
  *  <cit.> (BSD license) 
    
  *  <cit.> (Apache 2.0 license) 
    
  *  <cit.> (Apache 2.0 license)
    
  *  <cit.> (MIT License)




 Paper Title: MuJoCo MPC for Humanoid Control: Evaluation on HumanoidBench.




Anti-correlation between Flux and Photon Index of Hard X-ray Emission from The Crab
    [
    August 5, 2024
===================================================================================

empty
empty

§ ABSTRACT 
We tackle the recently introduced benchmark for whole-body humanoid control  <cit.> using MuJoCo MPC <cit.>.
We find that sparse reward functions of  yield undesirable and unrealistic behaviors when optimized; therefore, we propose a set of regularization terms that stabilize the robot behavior across tasks.
Current evaluations on a subset of tasks demonstrate that our proposed reward function allows achieving the highest  scores while maintaining realistic posture and smooth control signals.
Our code is publicly available and will become a part of MuJoCo MPC[<https://github.com/google-deepmind/mujoco_mpc/pull/328>],
enabling rapid prototyping of robot behaviors.






§ INTRODUCTION AND RELATED WORK

Designing dynamic behaviors for humanoid robots is a challenging problem <cit.>.
One promising approach is to craft task-specific reward functions and train policies via Reinforcement Learning (RL).
To this end, <cit.> proposed , a suite of 27 tasks for Unitree H1 robot based on MuJoCo <cit.>, with 12 tasks assessing locomotion abilities and 15 tasks evaluating object manipulation skills.
In <cit.>, performances of four popular RL algorithms are reported: DreamerV3 <cit.>, TD-MPC2 <cit.>, SAC <cit.>,  PPO <cit.>.
However, none of these methods is able to solve the tasks satisfactorily, producing
jittery motions ill-suited for execution on real hardware.
Therefore, we evaluate an alternative approach to humanoid behavior synthesis that exploits the power of simulation: Model Predictive Control (MPC).

MPC does not require training; at each time-step, it selects the optimal action based on a multi-step lookahead search, and repeats this procedure at the next time-step in a receding-horizon manner <cit.>.
MPC is a mature paradigm that admits various planning algorithms for this search, including iLQG <cit.> and even randomly sampled action sequences <cit.>.
We apply MPC to a subset of the  locomotion and manipulation tasks using MuJoCo MPC (MJPC) <cit.>, which implements the aforementioned planners.




§ MJPC WITH SHAPED REWARDS




MPC repeatedly solves a finite-horizon optimal control problem min_u_0:T∑_t=0^Tc(x_t, u_t) where c(x_t, u_t) is the instantaneous cost.
MJPC further decomposes the cost into a weighted sum 
c(x,u) = ∑_i=0^M w_i · n_i (c_i(x,u))
where c_i(x,u) are (signed) residuals, n_i are twice-differentiable norm functions, and w_i are tunable non-negative weights <cit.>.

The  tasks are formulated in terms of rewards r_hb rather than costs.
Therefore, we apply a transformation c_hb(x,u) = |r_max - r_hb| before passing it to MJPC, where r_max is the maximum achievable instantaneous reward (set to r_max = 1) and the norm is approximated by  to ensure differentiability. 

We observed that r_hb alone yields physically undesirable behaviors (Fig. <ref>) and results in low rewards (Fig. <ref>).
Therefore, we introduced a number of shaping terms that i) improve robot stability and ii) provide dense reward signal.

Stability-enhancing cost terms penalize excessively strong movements and promote postural balance:
1) height of the robot's head from the ground;
2) height difference between the pelvis and the feet;
3) linear velocity of the center of mass (CoM);
4) `balance' (projection of CoM between the feet);
5) deviation from a `canonical' posture;
6) direction the robot is facing;
7) magnitude of the control signal.


Dense reward residuals:
1) distance between an object and its target location;
2) distance between the left hand and the object;
3) distance between the right hand and the object.
Although some of these residuals are already encoded in the  reward, we find that providing them individually to the optimizer improves  performance (Fig. <ref>).



§ EXTENDED TASK EVALUATION PROTOCOL


Along with the pointed out uninformativeness of the reward function in , we observe one further factor that impacts the generated behaviors and evaluations—namely, the episode length.
The evaluation period
is very short—2 seconds for the  and  tasks; and for the  task, the simulation terminates as soon as the box reaches its target location.
Such short episode lengths result in policies that do not maintain balance at the end of the episode, not expecting further tasks (Fig. <ref>, bottom).
We provide a quantitative evaluation of the impact of the episode length in Fig. <ref> on the  task.
A qualitative evaluation is provided in Fig. <ref>, where new targets are re-spawn, requiring the robot to perform the task repeatedly.
Based on these evaluations, we argue for longer episode lengths and for repeated tasks with changing goals. 




§ ANALYSIS OF BEHAVIORS AND PLANNERS

In this section, we analyze the generated behaviors in a quantitative manner, beyond only reporting the return value.
r0.6
    

    Control smoothness cost: average squared joint velocity.
    
    Task     MPC-ours     MPC-hb 

         1.45 ± 0.13     1.67 ± 0.87 

         0.01 ± 0.00     1.49 ± 0.23 

         1.60 ± 0.14     1.66 ± 0.05 

    
    
    


First, we evaluate the smoothness of trajectories (energy cost), as judged by the average squared joint velocity (Table <ref>).
Our reward generates consistently smoother trajectories on the  and  tasks (while also achieving much higher reward—see Fig. <ref>).
On the  task, the cost is comparable due to the short episode duration.
Analysis with respect to other criteria will be provided in an extended version of the article.


While cheap at train-time (no pre-training needed), MPC is expensive at test-time because of online re-planning.
Therefore, the choice of the i) planner, ii) planning horizon, and iii) number of planning iterations is crucial.
Table <ref> shows the run times of our experiments on a MacBook Air M1, 16GB, averaged over 12 runs.
Each planner was given a fixed number of iterations to ensure a consistent comparison (normally, the planner would run as many iterations as possible within a real-time simulation timestep).




§ CONCLUSION


In this paper, we made a step towards enabling easy experimentation with MPC for humanoid robot control in simulation.
We ported  to MuJoCo MPC and provided evaluations on a subset of tasks.
We showed that the current reward functions are insufficient and proposed shaping terms.
Furthermore, we identified an issue with short episode lengths and argued for evaluating on repeated tasks with changing goals.
Our results show superior performance of our method over MPC and RL baselines.

plain
 



 Paper Title: DDU-Net: A Domain Decomposition-based CNN for High-Resolution Image Segmentation on Multiple GPUs.



SPECtrophotometer for TRansmission spectroscopy of exoplanets (SPECTR)
    Byeong-Cheol Lee
    August 5, 2024
======================================================================




§ ABSTRACT 
The segmentation of ultra-high resolution images poses challenges such as loss of spatial information or computational inefficiency. In this work, a novel approach that combines encoder-decoder architectures with domain decomposition strategies to address these challenges is proposed. Specifically, a domain decomposition-based U-Net (DDU-Net) architecture is introduced, which partitions input images into non-overlapping patches that can be processed independently on separate devices. A communication network is added to facilitate inter-patch information exchange to enhance the understanding of spatial context. Experimental validation is performed on a synthetic dataset that is designed to measure the effectiveness of the communication network. Then, the performance is tested on the DeepGlobe land cover classification dataset as a real-world benchmark data set. The results demonstrate that the approach, which includes inter-patch communication for images divided into 16×16 non-overlapping subimages, achieves a 2-3 % higher intersection over union (IoU) score compared to the same network without inter-patch communication. The performance of the network which includes communication is equivalent to that of a baseline U-Net trained on the full image, showing that our model provides an effective solution for segmenting ultra-high-resolution images while preserving spatial context. The code is available at https://github.com/corne00/HiRes-Seg-CNN.




Convolutional neural networks, deep learning, ultra-high-resolution images, image processing, parallel processing, semantic segmentation, U-Net, spatial context.









§ INTRODUCTION


The vast majority of deep learning models in computer vision focuses on low-resolution 2D and 3D images, typically 256 × 256 pixels or smaller. However, the increased utilization of high-resolution image datasets introduces new challenges due to the memory constraints of a single GPU, especially for memory-intensive tasks such as semantic segmentation of images <cit.>. Semantic segmentation is the computer vision task of classifying the pixels in the input into distinct, non-overlapping semantic categories. Ultra-high-resolution image segmentation holds significance in diverse fields such as object segmentation in satellite images <cit.>, metallic surface defect detection <cit.>, and computer-aided medical diagnosis <cit.>. Whereas deep convolutional neural networks (CNNs) have achieved remarkable success in image segmentation <cit.>, most of these models are unsuitable for model training and inference for high-resolution images due to their high memory requirements.

To illustrate this, consider computed tomography (CT) scans with sub-millimeter resolution, resulting in voxel image data with a typical resolution of 512×512×512 voxels. Even with half-precision floating-point numbers and a modest batch size of 8, processing such images with a 1-layer convolutional neural network with 64 filters demands over 137GB of GPU/TPU memory, as highlighted in <cit.>. Dealing with such high-resolution inputs using conventional strategies, like down-sampling or patch cropping, generally leads to the loss of detailed information or spatial context, resulting in a lower segmentation accuracy <cit.>. 

In this paper, we propose a novel network architecture based on the well-known U-Net <cit.> developed for biomedical image segmentation tasks. The U-Net is a convolutional neural network (CNN) with a bottleneck structure and skip connections between the encoder and decoder paths. This foundational architecture and variations of it <cit.> have demonstrated remarkable, state-of-the-art accuracy in semantic segmentation and other image-to-image tasks <cit.>. However, its substantial memory requirements are prohibitive for application to high-resolution images on computational devices with limited memory <cit.>. The high memory demands for training stem from the required storage of intermediate feature maps obtained during forward propagation to be used during the subsequent backward propagation pass. This is especially acute when dealing with high-resolution inputs, as it leads to high-dimensional feature maps in the first and last few layers of the network. 

Our proposed model integrates the established U-Net architecture with a divide-and-conquer strategy inspired by domain decomposition methods (DDMs) <cit.> to deal with computational device memory limitations. Previous parallelization strategies employ a decomposition of the image into subimages (subdomains) <cit.>, with communicated margins before each convolution or redundant computations providing global contextual information between subimages.
Our approach similarly decomposes the image. However, communication is explicitly limited to only the bottleneck within the U-Net architecture. This minimizes the communication overhead while preserving essential contextual information.

We summarize our main contributions as follows:

    
  * We propose a novel approach combining U-Net architectures with domain decomposition strategies to segment ultra-high-resolution images efficiently while preserving spatial context. 
    
  * We show that the communication network, which is an important component of our approach, can be employed to exchange information between different subimages. It enhances understanding spatial context without significant computational overhead and with minimal extra communication and memory cost. 
    
  * By evaluating our architecture on a synthetic and a realistic image dataset, we demonstrate competitive segmentation performance compared to baseline U-Net models. Our approach remains scalable, even when training is confined to the largest image portion that the available devices can handle.


The remainder of this paper is organized as follows: In <ref>, we discuss the U-Net architecture in more detail as well as existing memory parallelization strategies. Furthermore, we highlight approaches using ideas from the field of domain decomposition to speed up and parallelize CNNs.  <Ref> introduces our novel parallel architecture and training methodology as well as the considered test datasets. In <ref>, we discuss the influence of this new architecture on memory requirements and the receptive field. Then, we evaluate our proposed network model and the related training strategy using these datasets in <ref>. We provide experimental results for the different datasets in terms of the segmentation accuracy of our approach compared to global semantic segmentation U-Net models. Finally, in <ref>, we conclude and present possible future research directions. 



§ RELATED WORK


In this section, we discuss the U-Net, which serves as the basis for our proposed approach, and the receptive field, a key concept for understanding CNNs. Additionally, we provide an overview of past efforts to tackle the memory challenges associated with the U-Net and other CNN-based semantic segmentation models; we discuss both strengths and drawbacks of these methods. Finally, we introduce the idea of domain decomposition as a natural way to handle memory constraints, highlighting some previous works that adopt this approach for both classification and  segmentation.



 §.§ The U-Net
 
The new segmentation approach introduced in the this paper is based on the U-Net architecture <cit.>. Since its publication, many variants and extensions of the U-Net architecture have been developed, for instance, 3D U-Net <cit.>, UNet++ <cit.>, Attention U-Net <cit.>, and ResUNet-a <cit.>. We assume that the reader is familiar with the fundamental concepts underlying CNNs <cit.> and the building blocks of such models – convolutional and pooling layers. For a detailed explanation of these concepts, we refer to the rich literature on this topic, for instance, <cit.> or <cit.>. 



The U-Net architecture <cit.>, depicted in <ref>, consists of two pathways: the contraction path or encoder path, and the expansive path or decoder path. The contraction path involves repeated blocks, each consisting of two successive 3 × 3 convolutions, followed by a ReLU activation and a max-pooling layer. Conversely, the expansion path employs blocks that up-sample the feature map using 2 × 2 transposed convolutions. Subsequently, the feature maps from the corresponding layer in the contracting path are cropped (if the up-sampling path discards the boundary pixels) and concatenated to the up-sampled feature maps. This concatenation is followed by two successive 3 × 3 convolutions and a ReLU activation. 

In the final stage, the number of feature maps is reduced to the desired number of classes by a 1 × 1 convolution, producing the segmented image. The skip connections between the contraction path, which captures (global) context and features, and the expansive path, which enables precise, fine-grained localization, distinguishes the U-Net from other related network architectures, such as the Fully Convolutional Network (FCN) presented in <cit.>. Note that the originally proposed U-Net returns a cropped segmentation mask (due to the loss of border pixels in every convolution). However, more recent implementations of the U-Net often use zero padding at the borders to obtain output dimensions equal to the input dimensions, such that the input dimensions are preserved and cropping is not required; cf. <cit.>.



 §.§ The Receptive Field

An important concept for understanding CNNs is their receptive field, or field of view. The receptive field of a CNN is the area of the input image that influences the output of a pixel in the CNN output. Unlike fully-connected neural networks, where each input neuron is connected with each output neuron, CNNs have outputs that depend on specific regions of the input layer. In each layer of the CNN, the receptive field expands as we move deeper into the network, allowing higher-level features to be influenced by larger portions of the input image.

The theoretical size of the receptive field can be analyzed both experimentally and theoretically, as has been discussed extensively in <cit.>. The size of the receptive field significantly impacts the predictive performance of the CNN. If the receptive field is too small, the network may not capture all relevant information, leading to suboptimal predictions. For tasks like segmentation, the receptive field should be large enough to include all pixels in the input image that are relevant for predicting the correct pixel class. Note that the optimal size of the receptive field depends on both the network architecture as well as the data on which the network is trained.

In practice, the receptive field size of a CNN is influenced by several factors, including the number of convolutional layers, the size of the convolutional kernels, and the presence of pooling layers; cf. <cit.>. Increasing the depth of the network and the size of the kernels can expand the receptive field, allowing the network to capture more context from the input image. However, this also increases computational complexity and the risk of overfitting.



 §.§ Overcoming U-Net Memory Limits


One issue shared by most U-Net variants is the large memory requirements due to the storage of intermediate feature maps during training, making the model unsuitable for high-resolution applications with limited-memory computing devices <cit.>. A natural approach to overcome these constraints 
is to partition the memory load of the feature maps over different computational devices by using data and/or model parallelism. Several parallelized forms of the U-Net employ parallelism to reduce the memory per device. Both in <cit.> and in <cit.>, the authors implement a spatial partitioning technique that decomposes the input and output of the convolutional layers into smaller, non-overlapping subimages. Before each convolution operation, devices exchange patch margins of the feature maps of half the convolution kernel size with each other. While this approach introduces a communication overhead, it effectively distributes memory across devices without altering the fundamental architecture of the U-Net. Our approach, on the other hand, involves architectural modifications to the U-Net to further optimize performance and limit the communication to the bottleneck of the architecture.

The authors of <cit.> partition the image into overlapping subimages, with the overlap size determined by the receptive field size. Due to redundant computations (𝒪(q4ϵ/N) for an N × N pixel image partitioned into q × q subimages and a U-Net with receptive field size ϵ×ϵ), this approach allows for a fully parallelized execution of both forward and backward passes. <cit.> adopts a similar strategy, but with an application to image classification rather than image segmentation using the ResNet architecture <cit.>.

These approaches have been successful in partitioning the U-Net in such a way that the model can be trained and evaluated in parallel, even for ultra-high-resolution image datasets. However, they either involve communicating margins before each convolutional operation (this is the case for <cit.>), which leads to communication overhead through the many point-to-point messages, or they entail many redundant computations (this is the case for <cit.>). 



 §.§ Memory Optimization for CNNs Using Domain Decomposition Approaches
 
For improving memory efficiency, DDMs are inspiring due to their inherent parallelization and scalability properties, resulting from localization of the computations. DDMs are effective iterative solvers for (discretized) partial differential equations (PDEs), exhibiting scalability through a divide-and-conquer strategy that partitions the computational domain into overlapping or non-overlapping subdomains; see, for instance, <cit.>). The PDE problem is partitioned into subproblems defined on these subdomains. This enables parallel execution of computations within the subdomains. Whereas global convergence is ensured by well-balanced neighbor communication at potentially overlapping subdomain boundaries, and a limited amount of global communication. Highly scalable state-of-the-art DDMs are, for instance, variants of overlapping Schwarz methods <cit.> or balancing domain decomposition by constraints (BDDC) <cit.> and finite element tearing and interconnecting - dual primal (FETI-DP) <cit.>.

Recently, there has been an increasing interest in integrating DDMs with machine learning (ML) algorithms, combining the strengths of both fields <cit.>. Up to now, the majority of works focuses on combining DDMs and ML for solving PDEs. The present paper, however, pursues the combination of CNN with DDM techniques for semantic image segmentation. Notably, to the best of the authors' knowledge, no prior studies have tackled image segmentation tasks using ML explicitly based on domain decomposition strategies, although some methods have been proposed that have similarities with domain decomposition strategies. Existing research mainly focuses on image classification, which is closely related to semantic segmentation; however, due to the different model output, network architectures are somewhat different. Here, we provide a concise overview of methods combining DDM strategies with ML for image classification and semantic image segmentation tasks.



In <cit.>, the authors propose a image partitioning approach for image segmentation, reducing network complexity and enhancing parallelizability. They partition input images into non-overlapping subimages and subsequently train smaller local CNNs on these subimages instead of a large global CNN. This enhances model specialization to specific image regions and reduces overall complexity without a significant impact on test errors compared to a global CNN. Importantly, the authors process the subimages completely independently, with no coupling between them.
In <cit.>, DDM strategies are applied to train a CNN-DNN (convolutional neural network - deep neural network) architecture for image classification. In particular, the authors partition the input image into subimages, each used to train an individual CNN for predicting the class locally. A DNN then aggregates these local predictions into one global prediction. The authors interpret the DNN as a coarse problem solver that combines the finer-grained information from the local networks. In <cit.>, the authors decompose a global CNN for image classification into a finite number of smaller, independent local subnetworks along its width. The weights obtained from training these local subnetworks serve as an initialization for the subsequent training of the global network, employing a transfer learning strategy. In <cit.>, the authors propose a physically-motivated neural network topology for estimating extensive parameters such as energy or entropy. They decompose the domain into subdomains with focus and context regions, overlapping based on the locality of the extensive parameter estimated. The same subnetwork weights are used on the different subdomains (we employ this strategy as well in our approach), leading to a reduction of the number of parameters, enhanced parallelizability, and faster inference. Finally, in <cit.>, a DDM-inspired segmentation algorithm is proposed that divides the input image into several overlapping subimages and trains one segmentation network on these subimages, allowing for parallel inference. The authors conclude that this approach leads to better segmentation accuracy of small objects.




§ METHODOLOGY


In this section, we introduce our domain decomposition-based U-Net (DDU-Net) architecture, along with our training approach. We also describe the datasets used for testing the model and outline the model training procedure.



 §.§ Network architecture
 


  §.§.§ Decomposition of the images and masks

<Ref> shows a schematic representation of our proposed network architecture. Starting with a dataset of (high-resolution) images and corresponding segmentation masks, the model processes a 2D pixel image with dimensions H × W (left) and outputs a probability distribution for K ∈ℕ classes for each pixel of the image (right). 

Following DDM strategies, we decompose the input data into N × M non-overlapping subimages, with N, M ∈ℕ. In particular, each image is divided into N × M subimages with heights H_i and widths W_j, i=1,…, N and j=1,…,M, such that ∑_i=1^N H_i = H and ∑_j=1^M W_i = W. The corresponding segmentation masks are partitioned similarly. The subimages and sub-masks are distributed across the computational devices (e.g., GPUs/TPUs) to balance the workload evenly.

After partitioning, the subimages are processed independently, in parallel, by the encoders on the separate computational devices. For more details on the architectures of encoders and decoders, we refer to <ref>. A chosen number of feature maps from the last layer of the encoder is sent to the communication network (see <ref>), allowing for the exchange of relevant context between subimages. The communication network concatenates the encoded feature maps from all subimages corresponding to the arrangement of the subimages in the original full resolution image and processes them. Each output feature map of the communication network is again partitioned into N × M sub-feature maps, that are then sent back to the devices. The decoders then produce sub-predictions for each subimage, which are concatenated to form a global predicted mask for the entire high-resolution image.



  §.§.§ Encoder-decoder networks
 
<Ref> shows the encoder-decoder architecture used in this work, which closely aligns with the classical U-Net architecture <cit.>; cf. <ref>. The primary difference is a modification in the deepest layer of the encoder path to facilitate communication between the local encoder-decoder clones, which will be discussed below.



As discussed in <ref>, each computational device contains a separate encoder-decoder network, allowing for parallel processing of the subimages. However, these networks share their weights to ensure consistent segmentation across subimages, making them local clones of a global encoder-decoder network. This implies that during training, the weights and gradients need to be synchronized after each backward propagation step.

Each local encoder network processes its corresponding input subimage with several blocks of 3 × 3 convolutions, each followed by a batch normalization layer and a ReLU activation. During training, a dropout layer is also applied to prevent over-fitting. After two convolution blocks, max-pooling is performed to reduce the spatial dimension of the feature maps. The deepest encoder layer produces 256 spatially coarse feature maps (this number can vary with changes in the network architecture), some of which are employed for communication between the encoder-decoder networks. The output of each encoder network are 256 feature maps, with small spatial dimensions. The last F of these 256 feature maps are directed to the communication network, which processes them and returns modified feature maps with the same dimensions that replace the original input feature maps; see <ref>. After communicating, the feature maps then progress through the expansive path, resulting in a final segmented sub-mask for each subimage. These sub-masks are concatenated to form a full mask covering the entire full-resolution image.



  §.§.§ Communication network

The communication network, depicted in <ref>, transfers contextual features, captured by the encoders operating on subimages, between the encoded subimage feature maps. 



For this paper, the communication network consists of three layers of 5 × 5 convolutions. The DDU-Net architecture is not restricted to this specific network configuration. Adjustments such as altering the number of layers, dilation, and kernel size may enhance the model's receptive field size for specific applications. We choose our communication network to be fully convolutional to ensure adaptability to arbitrary input sizes as well as scaling the number of subimages.

The communication network receives as input F feature maps from the deepest encoder layers for each of the N × M subimages; cf. <ref>. These feature maps are concatenated along the height and width dimensions, aligning with the subimages' positions in the full-resolution input image; cf. <Ref>. The communication network processes this concatenated input and produces output feature maps with the same spatial dimensions as the inputs. These feature maps are partitioned back along height and width into N × M sub-feature maps, which are then sent back to the corresponding encoder-decoder network for further processing; they replace the original feature maps generated by the encoder and used as inputs to the communication network.



 §.§ Notation


In the remainder of this paper, we compare various model configurations using a naming convention for our domain decomposition-based U-Net that encapsulates key parameters. Each model is denoted as DFC, where:


    
  * D (∈ℕ_>0) indicates the number of up- and down-sampling blocks in the encoder-decoder network.
    
  * F (∈ℕ_≥0) specifies the number of feature maps processed by the communication network.
    
  * C (Y or N) denotes whether communication between communication feature maps is enabled (C=Y) or disabled (C=N)).

Note that, if communication is disabled (C = N) but the number F of feature maps processed by the communication network is non-zero, this means that F feature maps are sent through the communication network independently, without concatenation. This effectively results in a baseline U-Net architecture with extra convolutions in the bottleneck layer.

For instance, 464Y refers to a model where the encoder-decoder network has a depth of 4, 64 feature maps are sent to the communication network, and communication is enabled. The case D0N operating on 1 × 1 subimages is equivalent to the baseline U-Net architecture. In this case, DFN, with F>0 refers to a baseline U-Net with extra convolutions in the bottleneck layer. Additionally, for the case where F = 0, it does not matter whether communication is enabled (Y) or not (N), as the communication network uses 0 feature maps.



 §.§ Model training

Loss Function. For the network training, we employ the dice loss function, as this function addresses the issue of class imbalance for semantic segmentation of images; cf. <cit.>. The dice loss is defined as

    DL = 1 - 2 ∑_p=1^P ∑_k=1^K y_k,pŷ_k,p + ϵ/∑_p=1^P∑_k=1^K y_k + ∑_p=1^P∑_k=1^K ŷ_k + ϵ.

Here, K represents the number of classes, P is the total number of pixels in a batch, and ŷ_k,p is the predicted probability of pixel p for class k, obtained by applying a softmax function to the model's output so that all the output logits are in the range [0,1]. Furthermore, y_k,p denotes the true probability of pixel p for class k, with values restricted to {0,1} as the true mask is known, and ϵ serves as a small numerical stability constant (to avoid division by zero), set to ϵ=10^-7 in this paper. 

Parameter optimization. The weights of the network are initialized using the method proposed by He et al. <cit.>. We employ the Adam optimizer <cit.>, with momentum parameters β_1 = 0.9, β_2 = 0.999, and a plateau learning rate decay strategy, where the learning rate decays with a factor of 2 when the validation loss does not decrease, maintaining a patience of a chosen number of epochs. The initial learning rate, batch size, and early stopping criterion were chosen depending on the task and available memory and will be provided in the results section.

Training procedure. For training the DDU-Net architecture, we first initialize the encoder-decoder network with shared weights and distribute clones of this network  across all M × N devices. The coarse network for communication is initialized on a selected device and, due to the small size of the communicated feature maps, may optionally reside on the same device as one of the encoder-decoder networks. During forward propagation, each device independently computes the encoded feature maps. After processing a selected number F of those feature maps using the communication network, the decoding of the feature maps again takes place fully in parallel. Backward propagation through the encoders and decoders can, again, be done in parallel without dependencies between the local encoder-decoder networks. However, after backward propagation through the decoders, communication is necessary for backward propagating through the communication network. After backward propagation, gradients are accumulated centrally on a main device and used to update weights, ensuring uniform updates across all devices. The updated weights are broadcasted to the other devices for synchronization.

Implementation. The implementation was done using PyTorch <cit.> (version ), an open-source machine learning library.  We carried out the training and testing on the DelftBlue supercomputer <cit.> at the Delft University of Technology, and we employed NVIDIA Tesla V100S GPUs, with a memory of 32 GB, for the training.



 §.§ Datasets

For testing our model, we use two different image datasets: 1) a synthetically generated dataset designed to test the capabilities of the communication network and 2) a realistic image semantic segmentation dataset for multi-class land cover segmentation to assess the effectiveness of the proposed model both in terms of segmentation quality and memory efficiency. 



  §.§.§ Synthetic dataset


In our approach, only deep feature maps, with low spatial resolution, are exchanged between encoder-decoder networks. This differs significantly from previous U-Net parallelization methods that involve the exchange of a number of feature maps at each U-Net layer. To assess the level of spatial context that these low-resolution feature maps can capture, we designed a synthetic dataset. This dataset comprises one-channel gray-scale images with dimensions (k · 32) × 32 pixels, where k ∈{2, 3, 4, 6, 8, 16}. For each k, we generate 4 000 training, 1 000 testing and 1 000 validation images, resulting in a total synthetic dataset size of 36 000 images. This design allows the decomposition of images into k subimages of 32×32 pixels; see <Ref> for examples with k=4 subimages.



Each image in this gray-scale dataset displays a black background with two randomly placed white circles, each with a 4-pixel radius, placed completely within a subimage such that the subimage boundaries do not intersect the circles. The corresponding mask resembles the image, except for the addition of a third class of pixels: a line segment drawn between the centers of the two circles. Two examples of images and corresponding masks are shown in <Ref>.

When processing an image in this dataset with the DDU-Net, the segmentation of the line segment connecting the two circles relies entirely on the communication network. As the two subimages are processed independently and only linked by the communication network, the effectiveness of the segmentation is a direct reflection of the communication network's capability to transfer global information accurately. 



  §.§.§ DeepGlobe land cover classification dataset
 
The DeepGlobe land cover classification dataset <cit.> is a semantic segmentation dataset for land cover types. The dataset contains 803 high-resolution (2 448 × 2 448 pixels) annotated satellite images with 7 classes: urban, agriculture, rangeland, forest, water, barren, and unknown. The images have 50 cm/pixel resolution and span a total area of 1 716.9 km^2. In addition to the high resolution of the images, segmenting this dataset is challenging due to the large class imbalance; see <ref>. In <ref>, we show two example images and masks from the dataset.



Other challenges include the limited number of images, inexact ground truth, and the presence of multi-scale relevant contexts; cf. <cit.>. To illustrate this, note that trees can exist in various land types such as urban, rangeland, or forest areas. As a result, the network needs to assimilate contextual information from a wider region around the tree to predict the correct class for the tree pixels accurately.





 §.§ Evaluation Metrics

After training the model, we analyze the results. Because of the large class imbalance, we used the (mean) class-wise intersection over union (IoU) score as a metric, as was suggested in the paper introducing the DeepGlobe dataset <cit.>. Given a dataset with n images, the IoU score for class j is defined as: 

    IoU_j = ∑_i=1^n TP_ij/∑_i=1^n TP_ij + ∑_i=1^n FP_ij + ∑_i=1^n FN_ij,

where:

    
  * TP_ij is the number of pixels in the i-th image correctly predicted as class j,
    
  * FP_ij is the number of pixels in the i-th image incorrectly predicted as class j, and
    
  * FN_ij is the number of pixels in the i-th image that belong to class j but were predicted as another class.
 
The mean IoU score (mIoU) provides a single metric by averaging the IoU scores across all classes, and is then defined as:

    mIoU = 1/K∑_j=1^K IoU_j,

where K is the number of classes.




§ ARCHITECTURE DISCUSSION
 

In this section, we analyze some architectural properties of the DDU-Net.  First, we investigate the memory requirements of our approach both experimentally and theoretically and compare the results against a standard U-Net model. Then, we analyze the size of the receptive field of DDU-Net models with different architectures, that is, with varying depth of the subnetworks and the communication network.



 §.§ Memory requirements
 
We compare the memory requirements for the baseline U-Net depicted in <ref> and the proposed DDU-Net model architecture, both with a depth of 4 up-sampling and down-sampling blocks, operating on 2 subimages; cf. <ref>. We specifically present results for a configuration with F=64 communicated feature maps. The channel distribution for the encoder-decoder networks in both models follows the same scheme depicted in <ref>.

<Ref> presents a detailed analysis of the memory requirements for the encoder and decoder of the U-Net architecture, which are identical to the encoder and decoder used in the DDU-Net, during training on a 1 024 × 1 024 image. The table also includes the memory requirements for the proposed communication network with F=64. The values in this table were derived theoretically (see, for instance, <cit.>) and validated experimentally using the  library <cit.>. 

From the analysis, it is evident that storing the feature maps demands significantly more memory than the storing of model weights. The shallow layers, including the input block, the first decoder block, and the last decoder block, collectively contribute to nearly half of the total memory allocation for feature maps. Conversely, the number of weights increases for the deeper layers of the U-Net due to the higher number of kernels and larger kernel sizes in these blocks. The coarse feature maps used in the communicated layer result in relatively low memory requirements for the communication network, demonstrating the efficiency of the proposed DDU-Net in terms of memory utilization.



Figure <ref> shows the peak memory usage during inference for the baseline U-Net architecture and for a GPU containing both the U-Net encoder-decoder and the proposed communication network, for various image resolutions. For smaller image resolutions, the memory allocation can be largely attributed to model weights. However, as image size surpasses a moderate size of 2^7 × 2^7 (128 × 128) pixels, the memory required to store the feature maps begins to dominate the total memory cost. Beyond this point, a noticeable increase in memory requirements is observed, as illustrated in <ref>. 
It is evident from <ref> that the memory requirements for the DDU-Net and U-Net become relatively significantly closer at higher resolutions: for a resolution of 32 pixels, the relative difference is 25.9%, whereas for 4 096 pixels, this difference reduces to only 0.65 %. Additionally, it is important to note that peak memory scales quadratically with resolution; therefore, doubling the resolution results in a fourfold increase in memory consumption.
The results show that the proposed DDU-Net, with its communication network, requires only a small memory overhead compared to the baseline U-Net, despite the added cost of the communication network. This demonstrates the DDU-Net's efficiency in memory utilization: it facilitates communication between sub-images while increasing the memory requirements only slightly.



 §.§ Receptive field analysis
 
The proposed communication network operates on the coarse bottleneck layer of the encoder-decoder networks. Outputs of those layers typically have large receptive fields, as they are the result of numerous convolutional and downsampling operations. Therefore, the communication network is very effective in increasing the receptive field size of the model architecture. In <ref>, we compare the (theoretical) receptive field size of a standard U-Net with that of our proposed model at different depths of the encoder-decoder network, following the theoretical approach as presented in <cit.>. For the DDU-Net, we calculate the theoretical receptive field size for one infinitely large subdomain, providing an upper boundary for the true receptive field size when the network operates on multiple subimages. The comparison shows that the communication network significantly enlarges the receptive field size of the encoder-decoder network. 





§ EXPERIMENTAL RESULTS


In this section, we compare the DDU-Net approach against the corresponding baseline U-Net model based on the segmentation quality on the datasets introduced in <ref>. Moreover, we conduct an ablation study to examine the effectiveness of the communication between encoder-decoder networks processing subimages.





 §.§ Synthetic Dataset Results

In this section, we present the results obtained using our synthetic dataset, which is described in <ref>. The dataset comprises gray-scale images of varying dimensions, specifically (k · 32) × 32 pixels, where k ∈{2, 3, 4, 6, 8, 16}. For each k, we trained separate baseline U-Nets and DDU-Nets.

For our experiments, we trained baseline U-Nets using entire k · 32 × 32 images as inputs. In contrast, the DDU-Net architecture received k subimages of size 32 × 32 pixels, processed by k separate encoder-decoder clones. Both the U-Net and DDU-Net models were designed for 1-channel grayscale input images, generating outputs with three channels corresponding to the logits for the pixel classes: background, line segment, and circle.

We evaluate  

	
  * the impact of varying the number of down- and up-sampling blocks (depth D) in the encoder-decoder networks, 
	
  * the influence of the depth of the communication network, which is equal to the number of feature maps sent to the communication network (denoted as F), and 
	
  * the effect of inter-feature map communication (C, with values Y for enabled and N for disabled).

As mentioned in <ref>, we use the notation DFC to represent a DDU-Net with parameters D ∈ℕ_≥ 0, F ∈ℕ_≥ 0, and C ∈{Y, N}. For a baseline U-Net with depth D we use the notation D. The training hyperparameters used to train the networks are listed in <ref>. The number of weights in the encoder, decoder, and communication network for the different experiments can be found in the appendix, specifically in <ref>.



  §.§.§ Qualitative results

To show the effectiveness of the coarse network with communication, we compare the qualitative segmentation masks for three architectures: the 3, the 332Y and the 332N for the 32 × 64 pixel images (k=2) in <ref>. We observe that the 332N is unable to predict the position of the line segment correctly, as it is unaware of the circle position in the other subimage due to the lack of communication. In contrast, the 332Y predicts the location of the line segment correctly.



<Ref> shows that the proposed encoder-decoder network can capture the relevant features in the F=32 communicated feature maps, and that the communication network is able to transfer this information between the encoder-decoder networks. While we currently communicate all 32 feature maps in the bottleneck of the 332Y for this example, we will examine the impact of varying the number of feature maps in <ref>.  This is particularly noteworthy, as the communication only happens on a very coarse level (with 4×4 pixel feature maps) and the line segments are drawn on the finer grained 64× 32 resolution. Apparently, communication on this coarse level is sufficient for the decoder to produce a good segmentation result on the fine level.



  §.§.§ Varying the number of subimages and communicated feature maps


In <ref>, the IoU score (<ref>) for the line segment class is depicted for the baseline 3 as well as the 3FY, with F∈{0, 1, 2, 4, 8, 16, 32} feature maps communicated, for different numbers of subimages k. Recall that the case F=0 corresponds to a DDU-Net where the communication network has a depth of 0, meaning it is not used, also implying that there is no communication between encoder-decoder clones.

We observe two phenomena. First, there exists a clear positive correlation between the number of communicated feature maps F and the quality of the results. Secondly, the DDU-Net performs even better than the baseline U-Net for larger images and numbers and subimages, respectively. This is related to the sizes of the receptive fields as reported in <ref>: the DDU-Net has a larger receptive field (188 × 188 pixels versus 92 × 92 pixels), yielding better results when the circles are further apart.



  §.§.§ Impact of the depth of the encoder-decoder network

In the DDU-Net, the encoder-decoder network has depth D, that is, the number of up- and down-sampling blocks. The ideal depth is a trade-off: while shallow depths cannot capture all relevant features due to a limited receptive field size (cf. <ref>), overly deep encoder-decoders contain more parameters and may produce too coarse-grained and less informative feature maps for communication.

To see the effect of the depth of the encoder-decoder on the predictions, we compare the segmentation masks generated by a D16Y to the segmentation masks generated by a D, with D ∈{2,3,4}. The resulting predictions of those predictions are shown in <ref>. For encoder-decoder networks with D=2, we observe that the line segment is only predicted correctly for the pixels that are in the center region between the two circles. This can also be explained by the receptive field: shallow networks have a limited receptive field, such that pixels far away from one of the two circles are not affected by this circle; this leads to the incorrect segmentation result.

It can also be seen that the DDU-Net with communication enabled gives better results than the U-Net for D=2 and D=3, as the communication network increases the receptive field size; cf. <ref>. 



  §.§.§ Generalization to different numbers of subimages


The baseline U-Net and the DDU-Net are both size-agnostic, as both are fully convolutional neural networks. This allows them to process images with sizes different from the training images. In case of the DDU-Net, this means that we can vary both the size of subimages and the number of subimages. In order to test the generalization of the DDU-Net with respect to the number of subimages for different amounts of communication, we train for each k∈{2,3,4,6,8,16} a 3FY on input images with spatial dimensions of (k · 32) × 32 pixels, as described in section <ref>, with F ∈{0,1, 2,4,8,16,32}. Each of these models is then evaluated on a dataset with 6 subimages. The results are shown in <ref>.


We can make the following observations: 


  * Models trained on 3, 4, and 6 subimages perform the best on the 6 subimage test dataset. This is likely because the images in these datasets are small enough to be (almost) fully covered by the 188× 188 pixels receptive field of the DDU-Net with D=3; cf. <ref>.

  * The more inaccurate results for the 8 and 16 subimages can be explained by the limited receptive field size, 188× 188 pixels, of the DDU-Net with D = 3; cf. <ref>. 

  * The IoU score deteriorates when the model is trained on only two subimages and then evaluated on a larger number of subimages. This is likely due to the fact that every subimage in the training data set contains one circle and a part of the line segment; however, there are no samples with subimages that do not contain circles or line segments.


These findings suggest that the DDU-Net can be trained on a different number of subimages than on which it is evaluated. 



  §.§.§ Summary of results on the synthetic dataset

We summarize our findings for the synthetic dataset as follows:

    
  * The communication network is able to transfer contextual information across subimages, which becomes clear both from qualitative comparison in <ref> and quantitative results in <ref>.
    
  * The segmentation quality increases when the number of feature maps involved in communication increases.
    
  * The larger receptive field size of the DDU-Net with communication leads to better results compared to the baseline U-Net; cf. <ref>.
    
  * The DDU-Net can be trained successfully on a fixed number of subimages and evaluated on different number of subimages. 




 §.§ DeepGlobe Dataset

Now, we assess the effectiveness of the DDU-Net for a real-world dataset with high-resolution images: the DeepGlobe dataset; cf. <ref>.



Given the small number of 803 images in the DeepGlobe dataset relative to the segmentation task's complexity, we made several adjustments to the encoder-decoder architecture, including batch normalization, random dropout layers, and data augmentation (horizontal and vertical flipping, random rotation of R radians, with R ∈{0, π/2, π, 3π/4}), and we incorporated a pre-trained image encoder model, the ResNet-18 <cit.>. This model, trained on over 1 000 classes on the ImageNet data set <cit.>, offers a rich feature representation for diverse images. The ResNet-18 model consists of residual blocks with skip connections allowing an effective gradient flow. We used the first four residual blocks of the ResNet-18 to initialize our encoder. This strategy leverages the pre-existing knowledge within the pre-trained model to enhance the network's ability to generalize patterns from the limited dataset. A visualization of the employed model architecture is given in <ref>. For a detailed overview of the distribution of model parameters within each component of the network, we refer to <ref>.
 
During training, we kept the ResNet-18 model's weights fixed, only adjusting the weights of the decoder and the communication network. Additionally, we inserted two extra 3×3 convolutional layers in the bottleneck layer. These layers allow the network to restructure and refine the feature maps produced by the ResNet encoder generate relevant information for the communication network.

To ensure a fair comparison between the segmentation quality of the baseline U-Net and the DDU-Net, we trained both models on equally large images. Due to the baseline U-Net architecture's inherent inability to parallelize across different GPUs and the limited training device memory (32 GB), we cropped 1 024 × 1 024 non-overlapping patches from the DeepGlobe dataset, resulting in a training dataset consisting of 2 412 images. It is crucial to distinguish between these large “global patches” and the subimages in the DDU-Net architecture. During training of the DDU-Net, the global 1 024 × 1 024 patches are further partitioned into smaller subimages, which then are distributed across the encoder-decoder networks. Using mixed precision training <cit.>, this approach allowed us to train with a mini-batch size of 12 images on a single GPU. This and other hyperparameters used for training are shown in <ref>. Furthermore, to take into account random initialization of the network parameters, we trained every network repeatedly for three times with the same settings and training dataset. For each configuration, the best performing model (in terms of IoU score on the test dataset) was selected. 


We want to investigate if the DDU-Net can perform equivalently to the baseline U-Net. However, when we include communication in the DDU-Net architecture the additional trainable parameters imply a potential ability to learn more complex patterns. To isolate the effects of (1) the extra parameters in the communication network and (2) the communication itself, we vary the number of feature maps F for both 4FY and 4FN. The 4FN scenario represents an encoder-decoder network with an extra coarse network at the bottleneck layer (with coarse network chosen the same as the communication network), but without information exchange between sub-domains. In this case, the coarse network operates solely on the local bottleneck feature maps, rather than on the concatenated feature maps of all subimages. Notably, F=0 corresponds to the baseline 4. The difference between the cases with communication enabled (Y) and disabled (N) shows the impact of communication across subimages.



  §.§.§ Quantitative results

<Ref> shows the mean IoU scores for different DDU-Net configurations and image partitions, trained on 1 024× 1 024 images and evaluated on the 2 048 × 2 048 test dataset with fixed subimage size, using the approach discussed in <ref>. 

From this figure, two main observations emerge. First, the segmentation quality improves with increased number of feature maps F in the coarse network. This is expected since a higher number of feature maps leads to an increase in the number of parameters and a larger receptive field, enabling the model to capture more complex and distant patterns.

Another trend in <ref> is that the quality of the predictions decreases as the number P× P of subimages increases, or, equivalently, as the subimage size decreases. However, for the 4FY, this decrease in quality is much less pronounced compared to the 4FN. This indicates that the communication combined with the coarse network effectively transfers contextual information between subimages, also for this realistic dataset. 





  §.§.§ Qualitative results

In <ref>, example predictions are shown for different training and model configurations. We observe that the DDU-Net trained on 1× 1 and the DDU-Net trained on 8 × 8 subimages with communication enabled both produce good segmentation results, although there are differences compared to the true mask. Conversely, the DDU-Net trained on 8× 8 subimages without communication shows poorer predictions.To better illustrate prediction errors, we include <ref> in the Appendix, where black indicates correct predictions and white indicates incorrect predictions. 
The key distinction lies in the consistency across neighboring subimages: when communication is enabled, predictions for neighboring subimages exhibit smoother boundaries instead of patchy patterns. This highlights the effectiveness of communication between subimages in the DDU-Net architecture.



§ CONCLUSION


This paper develops a new domain decomposition-based U-Net (DDU-Net) architecture for semantic segmentation tasks. Our results show that by including communication between subimages, the DDU-Net can handle high-resolution image segmentation efficiently without sacrificing accuracy or memory efficiency. Our approach improves segmentation accuracy by leveraging inter-subimage communication. Future research will focus on refining communication strategies, applying DDU-Net to more complex datasets, and evaluating the benefits of parallelization on computing times. Another direction is extending the parallelization strategy developed in this paper to other encoder-decoder architectures.



§ ACKNOWLEDGMENT


The authors acknowledge the use of computational resources of the DelftBlue supercomputer, provided by Delft High Performance Computing Centre (https://www.tudelft.nl/dhpc) <cit.>.
This work performed in part at Sandia National Laboratories was supported by the U.S. Department of Energy, Office of Science, Office of Advanced Scientific Computing Research through the SEA-CROGS project in the MMICCs program.
Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525. This paper describes objective technical results and analysis. Any subjective views or opinions that might be expressed in the paper do not necessarily represent the views of the U.S. Department of Energy or the United States Government. Sandia report number SAND2024-09671O.



§ NETWORKS USED FOR SYNTHETIC DATASET









The properties of the encoder-decoder and coarse networks used for the synthetic dataset experiments are detailed in <ref>. The properties are shown for different configurations in terms of encoder-decoder depth D. The numbers of weights for the encoder-decoder and coarse network architectures used for the DeepGlobe land cover classification dataset are detailed in <ref>. The encoder-decoder network consists of multiple ResNet18 blocks followed by an inter-convolutional and up-sampling path. We show the number of weights in the coarse network for various choices of its architecture, corresponding to the different experiments as shown, for instance, in <ref>. <Ref> summarizes the hyperparameters employed during training for synthetic data experiments. <Ref> visualizes the difference between predicted and true masks for different configurations of the DDU-Net.



siamplain




 Paper Title: GenRec: Generative Personalized Sequential Recommendation.















panfengc@umich.edu

  University of Michigan
  Ann Arbor
  USA


pl219@cam.ac.uk

















§ ABSTRACT 
Sequential recommendation is a task to capture hidden user preferences from historical user item interaction data. Significant progress has been made in this domain by leveraging classification based learning methods. Inspired by the recent paradigm of “pretrain, prompt and predict” in NLP, we consider sequential recommendation as a sequence to sequence generation task and propose a novel model named Generative Recommendation (GenRec). Unlike classification based models that learn explicit user and item representations, GenRec utilizes the sequence modeling capability of Transformer and adopts the masked item prediction objective to effectively learn the hidden bidirectional sequential patterns. Different from existing generative sequential recommendation models, GenRec does not rely on manually designed hard prompts. The input to GenRec is textual user item sequence and the output is top ranked next items. Moreover, GenRec is lightweight and requires only a few hours to train effectively in low-resource settings, making it highly applicable to real-world scenarios and helping to democratize large language models in the sequential recommendation domain. Our extensive experiments have demonstrated that GenRec generalizes on various public real-world datasets and achieves state-of-the-art results. Our experiments also validate the effectiveness of the the proposed masked item prediction objective that improves the model performance by a large margin.







<ccs2012>
 <concept>
  <concept_id>10010520.10010553.10010562</concept_id>
  <concept_desc>Information systems Recommender systems</concept_desc>
  <concept_significance>500</concept_significance>
 </concept>
 <concept>
  <concept_id>10010520.10010553.10010563</concept_id>
  <concept_desc>Information systems Users and interactive retrieval</concept_desc>
  <concept_significance>400</concept_significance>
 </concept>
 <concept>
  <concept_id>10010520.10010575.10010755</concept_id>
  <concept_desc>Computing methodologies Natural language generation</concept_desc>
  <concept_significance>300</concept_significance>
 </concept>
</ccs2012>


[500]Information systems Recommender systems
[400]Information systems Users and interactive retrieval
[300]Computing methodologies Natural language generation
















10 May 2024






GenRec: Generative Personalized Sequential Recommendation
    Pietro Liò
    August 5, 2024
=========================================================





§ INTRODUCTION

Recent years have witnessed the great success of recommender systems in online platforms such as Amazon and Yelp, which help people make micro decisions and fulfil their demands in daily life. Within online platforms, historical sequential user interaction data is utilized to capture the evolved and dynamic user behaviors and make appropriate recommendations of next items for the user. Different from traditional recommender systems that treat each user behavior as a sample and directly model the user preference on a single item, sequential recommendation learns timestamp-aware sequential patterns to recommend the next item <cit.>. For example, one might purchase peripheral accessories after buying a desktop. Various methods have been proposed to accurately model sequential user behaviors. <cit.> employ Recurrent Neural Networks (RNN) to encode the left-to-right context. <cit.> utilizes Convolutional Neural Networks (CNN) to learn sequential patterns as local features. <cit.> leverages Graph Neural Networks (GNN) to construct the item-item interest graphs and user interest sequences. <cit.> designs contrastive loss to maximize the mutual information between different views of the sequential data. Contextual information such as item attributes is incorporated as self-supervised signals to learn user and item representations. 

The recent success of Transformer <cit.> based large language models (LLMs) in various NLP tasks has inspired the research of utilizing LLMs in recommender systems <cit.>. <cit.> utilizes the left-to-right self-attention mechanism to identify next relevant items from the sequential interaction history and <cit.> further improves the performance by employing the bidirectional self-attention mechanism. Although both methods are effective, they do not consider personalization and the model performance is potentially limited. Generative LLMs based recommender system is considered a promising approach <cit.>. <cit.> is the first model to unify different recommendation tasks in the same framework to facilitate the knowledge transfer and it is pretrained across task-specific datasets to capture the deep semantics for personalization and recommendation. However, different modality information is encoded in the textual format of natural language, which might cause suboptimal modality representation. Moreover, <cit.> relies on manually designed task-specific hard prompts to formulate the problem as question answering, which requires additional data processing and prompt search.



To address the mentioned limitations, we propose GenRec, a novel generative framework for personalized sequential recommendation. GenRec utilizes the encoder-decoder Transformer as the backbone and formulates sequential recommendation as a sequence-to-sequence generation task. We utilize the cloze task <cit.> as the training objective and pretrain the model on the corpus to learn the bidirectional sequential patterns. Inspired by <cit.>, to make the downstream sequential recommendation task consistent with the pretraining cloze task, we append the  token at the end of the input sequence to predict the next item. Extensive experiments on public real-world datasets demonstrate that GenRec generalizes effectively and achieves state-of-the-art performance across all datasets.
In conclusion, our contributions include: 


  * We propose GenRec, a novel generative model for personalized sequential recommendation that can generate next items auto-regressively without prompts.

  * As far as our knowledge, it is the first generative sequential recommendation method that adopts the cloze objectives in both pretraining and finetuning. Our method only utilizes task specific datasets and does not rely on additional large pretraining corpus [Our source code, datasets and pretrained models are publicly available at <https://github.com/caop-kie/GenRec>.].

  * We conduct experiments on three public real-world datasets, demonstrating consistent improvements on multiple evaluation metrics compared with baseline methods.




§ RELATED WORK

Transformer based LLMs have achieved remarkable success on various research fields such as text summarization, question answering, document understanding, etc. Typically, LLMs are trained on a vast amount of textual corpus from diverse sources including wikipedia, news, articles and books. LLMs can have emergent zero-shot learning capability as the model parameter size scales up with large training datasets <cit.>, making LLMs better generalize to unseen domains. Recent efforts have been made to explore the potential of LLMs as recommender systems. <cit.> employs T5 as the model backbone and achieves competitive performance on a variety of recommendation tasks, demonstrating the generalization capability of LLMs based recommender systems. <cit.> develops a unified foundation model based on M6 and reduces the model size and training time by utilizing prompt tuning. Similar to <cit.>, user interaction data is represented as plain texts and recommendation tasks are formulated as either natural language understanding or generation. <cit.> leverages ChatGPT to build a conversational recommender system to improve the interactivity and explainability of the recommendation process. <cit.> finetunes T5 with user-personalized instruction data, which is generated from manually designed templates, enabling users to communicate with the system with natural language instructions. <cit.> proposes an efficient framework to finetune LLMs for recommendation tasks and demonstrates significant improvements in the domains of movie and book recommendations. <cit.> employs a funneling approach where it first retrieves the candidates utilizing the user item interactions and then generates recommended items from candidates with a LLMs based framework.



§ TASK FORMULATION

The goal of sequential recommendation is to predict the next item that the user is most likely to interact with given the historical item interaction sequence of the user. Formally, i_th user u_i has item interaction sequence t_i = {t^i_1, ..., t^i_n_i} ordered chronologically. t^i_j and n_i denotes the j_th item in the sequence and the length of t_i respectively. t^i_j:k denotes the subsequence, i.e., t^i_j:k = {t^i_j, t^i_j+1, ..., t^i_k}, where 1 ≤ j < k ≤ n_i. Given the user item sequence {u_i, t^i_1:n_i}, the task of sequential recommendation is to predict the next item that u_i is most likely to interact with at the n_i+1 timestep. 





§ METHODOLOGY

The overall architecture of  is shown in Figure <ref>. Our  is established upon a sequence-to-sequence Transformer encoder-decoder framework. The encoder of  embeds cross modal user and item features from the input sequence and the decoder generates next items auto-regressively. In the following sections, we elaborate the feature embedding and learning objectives of pretraining and finetuning.


 §.§ Model

The feature embedding consists of token embedding, positional embedding, user ID embedding and item ID embedding. The input sequence to the model {u_i, t^i_1:n_i} is tokenized into a sequence of tokens, which is wrapped around with the start indicator token [BEG] and the end indicator token [END]. Extra [PAD] tokens are appended to the end to unify the sequence length in the batch. The token sequence S is represented as: 

    S = [BEG], Tok(u_i), Tok(t^i_1), ..., Tok(t^i_n_i), [END], ..., [PAD],

where Tok is the tokenizer function.
For the j_th token in S, we apply textual token embedding, positional embedding and user or item ID embedding. The textual token embedding is formulated as Emb(S_j), where Emb is the token embedding function. The positional information of tokens in the sequence is captured by the positional embedding PosEmb(j), where PosEmb is the 1D positional embedding function. And we utilize the user ID and item ID embedding to add personalized information of users and items to the sequence. Personalized information helps reveal the sequential patterns of user behaviors. For example, a user that previously viewed a Samsung tablet is more likely to continue viewing electronic products from Samsung or other brands. Different from P5 <cit.>, which apply whole-word embeddings shared by both user and item tokens. Our ID embedding layers are not shared to better capture modality specific features. The user ID embedding is represented by UIDEmb(UID(j)) and the item ID embedding is represented by TIDEmb(TID(j)). UIDEmb and TIDEmb are the ID embedding functions for users and items respectively. Since a word in the sequence can be split into multiple tokens after tokenization, we utilize UID(j) or TID(j) to retrieve the user ID or item ID for the j_th token. For example, item_1234 is split into 4 tokens i.e. item, _, 12 and 34. Those tokens share the same item ID 1234 and embedding. Note only user tokens have user ID embeddings and only item tokens have item ID embeddings. All the aforementioned cross modal embeddings are added element-wisely to produce the final embedding X. Specifically the j_th token cross modal embedding is formulated as: 


    X_j =    Emb(S_j) + PosEmb(j) + IDEmb(ID(j)), j ∈ [0, n_i),
       IDEmb ∈{UIDEmb, TIDEmb},
       ID ∈{UID, TID}



The decoder input is the next item that serves as the learning target. The same tokenizer and textual embedding in the encoder are utilized. Following baseline methods, the model generates top 20 items with beam search during inference to calculate the evaluation metrics.




 §.§ Pretraining


We pretrain  with the masked sequence modeling task to deeply fuse the personalized sequential patterns. Given a user item sequence, we randomly sample an item, which is not necessarily the last item, from the sequence and replace it with the  token. The model is trained to generate the masked item in the decoder output. Cross-entropy loss is computed between the generated item and the masked item and minimized in pretraining. The masking process is also illustrated in Figure <ref>. Different from <cit.>, where a number of items are masked, we only mask a single item in the sequence to unify the learning objectives of pretraining and finetuning. Our bidirectional masked sequence modeling task is based on the observation that the item sequence is not strictly ordered. For example, given similar items item_1 and item_2, user_1 might interact with item_1 and then item_2 while user_2 interacts with item_2 and then item_1. It is crucial to incorporate both left and right contexts to encode the user behavior <cit.>. To avoid the data leakage, we pretrain the model on the training split of datasets (See <ref> for details about the training split).


 §.§ Finetuning

In the finetuning stage, the learnt user and item sequential patterns in pretraining are utilized for the next item prediction task. As mentioned in section <ref>, finetuning has the same learning objective with pretraining for knowledge transfer. Following the masking mechanisms in Figure <ref>, we prepare the dataset in the same format as in pretraining by appending the  token to the end of the token sequence. The input to the model is the masked user item sequence and the output is the predicted next item.


§ EXPERIMENTS




§ DATASETS


Amazon Sports and Amazon Beauty datasets are obtained from Amazon review datasets <cit.>, which are collected from Amazon.com online platform. 
Yelp dataset is a large business recommendation dataset. Following <cit.>, we use transactions between January 1, 2019 and December 31, 2019. For all datasets, item sequences are extracted per user and sorted ascendingly by the interaction timestamps. Following the same data preprocessing in the baseline methods, unpopular items and inactive users with less than five interaction records are filtered. The statistics of the datasets after preprocessing are summarized in Table <ref>.

To generate the training, validation and test splits of datasets, we apply the leave-one-out strategy following <cit.>. In each user item sequence, we use the last item as the test data, the item before the last item as the validation data, and the remaining item sequence as the training data.





§ BASELINES


We compare the performance of GenRec with several competitive sequential recommendation baselines.

Caser <cit.> is a CNN based method, which utilizes both horizontal and vertical convolutional filters to capture high-order Markov chains for sequential recommendation.

FDSA <cit.> models the feature sequence with the self attention module and captures the feature transition patterns.

GRU4Rec <cit.> applies GRU to model the user click history sequence for session based recommendation.

HGN <cit.> adopts hierarchical gating networks to learn both long-term and short-term user behaviors.

Bert4Rec <cit.> learns the bidirectional item representation by masked language modeling for sequential recommendation.

SASRec <cit.> applies the self attention mechanism to model the user sequential behaviors.

S3Rec <cit.> designs self-supervised learning objectives to capture the correlations of items and attributes for sequential recommendation.

P5 <cit.> is a pretrained language model for recommendation and unifies different recommendation tasks by formulating them in natural language with prompts.

All the baselines except P5 are classification based.



 §.§ Implementation Details

The model weights of  is initialized from the pretrained BART <cit.> base model, which consists of a 6-layer Transformer encoder and a 6-layer Transformer decoder. There are 12 attention heads for both encoder and decoder and the model dimension is 768. The total number of model parameters is 184 million. We use the pretrained BART tokenizer for tokenization.  is trained with an AdamW optimizer <cit.>, in which 1e-5 is set as the learning rate and weight decay. The maximum length of the input tokens is set to 512. First 5% of iterations are used as the warmup stage. The beam size is set to 20 in inference.  is pretrained for about an hour and finetuned for 25 epochs on one NVIDIA RTX 3090 GPU.



 §.§ Evaluation Metrics

To evaluate the model performance, we employ top-k Hit Ratio (HR@k) and Normalized Discounted Cumulative Gain (NDCG@k) and report the the results of HR@{1, 5, 10} and NDCG@{5, 10} under the all-item setting, i.e. all items are possible candidates to be recommended as the next item. Our model is evaluated on Amazon Sports, Amazon Beauty and Yelp datasets. In the tables, bold numbers refer to the best scores, while underlined numbers refer to the second best scores.



 §.§ Results and Analysis

This section provides experimental results and analyses of our model's effectiveness. The performance metrics of the baselines are obtained from the public results in <cit.>. See Appendix <ref> for more details about baselines. As is presented in Table <ref>, GenRec outperforms other baselines on all metrics on Sports and Yelp datasets. Although it is outperformed by P5 on the HR@10 metrics of the Beauty dataset, GenRec still achieves comparable performance with other baselines. 

We speculate due to the smaller size of Beauty dataset, the sequential patterns are not learnt thoroughly in pretraining. Increasing the number of pretraining epochs could potentially improve the performance. Since we focus on low-resource efficient training in this work, we leave the improvement for future work. Since P5 is a unified model and trained using different types of recommendation tasks, it learns better user item representation with the knowledge transfer between tasks on smaller datasets. The advantage of our method compared with P5 is that our model is lightweight and the training only takes a few hours to complete. We do not leverage prompt engineering and no prompt search is required to find the best prompt. 





§ ABLATION STUDY


To verify the effectiveness of the proposed masked sequence modeling task, we conduct the ablation study on Amazon Sports, Amazon Beauty and Yelp datasets. As is shown in Table <ref>, the model performance drops across all metrics without the masked sequence modeling task. It indicates the pretraining objective helps the model effectively capture the user behavior patterns and learn the user item representations. It is worth mentioning even without pretraining, GenRec still achieves comparable performance with P5 and outperforms other baselines, which proves the effectiveness of our method.



§ CONCLUSION

In this work, we propose GenRec, a novel sequence to sequence framework that generates personalized sequential recommendation. Compared with existing generative models, GenRec is lightweight and efficient and does not rely on prompt engineering to find the best prompt. By leveraging the Transformer architecture as the backbone, GenRec effectively learns the bidirectional sequential patterns with the attention mechanism and achieves state-of-the-art performance on various public datasets. Besides, our proposed method is also flexible to integrate with other sequence to sequence language models and can be applied in other recommendation domains such as direct recommendation, which we leave for future work.





ACM-Reference-Format



 Paper Title: A State-of-the-Art Review of Computational Models for Analyzing Longitudinal Wearable Sensor Data in Healthcare.















































A State-of-the-Art Review of Computational Models for Analyzing Longitudinal Wearable Sensor Data in Healthcare
    Paula Lago 
 paula.lago@concordia.ca 
 Department of Electrical and Computer Engineering, PERFORM Center 
Concordia University, Montreal, QC, Canada
    
========================================================================================================================================================

§ ABSTRACT 
Wearable devices are increasingly used as tools for biomedical research, as the continuous stream of behavioral and physiological data they collect can provide insights about our health in everyday contexts. Long-term tracking, defined in the timescale of months of year, can provide insights of patterns and changes as indicators of health changes. These insights can make medicine and healthcare more predictive, preventive, personalized, and participative (The 4P's). However, the challenges in modeling, understanding and processing longitudinal data are a significant barrier to their adoption in research studies and clinical settings. 
In this paper, we review and discuss three models used to make sense of longitudinal data: routines, rhythms and stability metrics. We present the challenges associated with the processing and analysis of longitudinal wearable sensor data, with a special focus on how to handle the different temporal dynamics at various granularities. 


 We then discuss current limitations and identify directions for future work. This review  is essential to the advancement of computational modeling and analysis of longitudinal sensor data for pervasive healthcare.



§ INTRODUCTION


Modern phones and wearable sensors have emerged as indispensable tools for capturing continuous, longitudinal, real-world behavioral and physiological data for health and health research <cit.>. The unprecedented depth of information, combined with advances in machine learning, opens the door to further our understanding of the relationship between health and behavior, to automatically assess and predict health status, and to create personalized interventions to improve health outcomes. 






One practical challenge to realize the potential of research using such sensors is how to make sense of the large amounts of data collected over long periods. As stated by Xu. et.al. <cit.>, the information captured by these sensors, "while information-rich, is also information-vague", requiring thoughtful analysis to obtain valuable insights. 
A large body of research has focused on finding features in the data that correlate with a specific health condition, for example, COVID-19 <cit.>, epilepsy <cit.>, depression <cit.>, schizophrenia <cit.>, anxiety <cit.>, or mild cognitive disease <cit.> among others. While the results are encouraging, few standards exist for the analysis. More often than not, the features are heavily dependent on the device used and hard to generalize <cit.>, which makes it difficult to translate research findings into policy or recommendations.

Longitudinal data can support applications such as early warning systems by identifying changes in behavior <cit.>. One critical aspect for these applications is finding typical patterns in the data. These typical patterns can be used to summarize the data, find anomalies, and measure the strength of the "typicality" or "variability" in day-to-day life. Research has shown associations of behavior regularity with physical health <cit.>, and disruptions of daily routine were found to be associated with mental health disorders <cit.>, thus providing a foundation for the need to find objective measures of regularity and disruption using sensor data. For instance, activity patterns measured with passive sensors were found to be associated with Mild cognitive disease <cit.>.
However, the lack of standards to analyze sensor data has made results obtained so far non-generalizable and difficult to replicate <cit.>.
Consequently, there exists a pressing need to develop theories, tools, and techniques that not only model a baseline of typical patterns but also adeptly measure, interpret, and comprehend variations or changes to such baseline.

This work provides a comprehensive review of existing methods to model behavior and predict health using continuous, longitudinal sensor data. Two complementary views have emerged in the literature to address this problem: routines and rhythms. Both views underscore temporal aspects of behavioral and physiological regularity: routines focus on the frequency of events; rhythms emphasize temporal regularity and periodicity. Recently, stability metrics have also been proposed based on routine or rhythms views. 
We highlight their strengths and differences as well as the common open research challenges ahead. 

Other reviews exist on the use of smartphones and wearable sensors in healthcare. A part of these reviews focuses on the application areas and evidence around the use of passive sensing for early diagnosis, management, or prediction of disease <cit.>. Other focus on the technologies accelerating the rise of passive sensing in healthcare. 
Fewer reviews focus on the modeling approaches for longitudinal patterns. Mohr et.al.<cit.> focuses on mental health applications, providing an overarching model of personal sensing, and reviewing the literature on using sensors to detect mental health conditions and related behavioral markers.
The review by Witt et.al.<cit.> describes algorithmic approaches used to generate fitness- and health-related indicators from wearables data, all of which focus on providing a current picture of the user instead of focusing on their long-term patterns. They briefly review behavior change detection techniques which focus on the analysis of longitudinal data. 

Meyer et.al. <cit.> have reviewed the implications of long-term tracking, clearly differentiating it from short-term tracking. Their review focuses on implications for the user collecting the self-track data. They define long-term "in timescales of years". Here, we review existing models that may support the modeling of such data for healthcare applications. 

In this review, we focus on the modeling long-term patterns, their analysis, quantification, and the main issues in making sense of the large amounts of data collected. We first provide a description of the rhythm (Section <ref>) and routine (<ref>) models and their evolution over time, and then provide an overview of common metrics used to quantify the regularity of patterns (Section <ref>). We finalize the review with a discussion of open issues in both models (Section <ref>).
In an effort to unify the clinical and computational perspectives, this review synthesizes the rhythmic and routine-based approaches to long-term pattern modeling, as well as the growing focus on metrics. This review contributes to the biomedical research by providing a comprehensive summary of the current methods to model, analyse and process longitudinal wearable sensor data. 










§ RHYTHMS


A rhythm is a cyclic pattern, repeated at regular temporal intervals. Human behavior and physiology follow biological rhythms, the most common ones are linked to the day/night cycle. These rhythms are governed by an internal pacemaker called the suprachiasmatic nucleus (SCN), an internal structure located in the brain, and are affected by external factors like light, temperature, and social factors, among others <cit.>. Research has found strong links between the strength and alignment of this central clock with environmental and behavioral cycles and our physical and mental well-being. For this reason, the characterization of the various behavioral and physiological rhythms from sensor data is deemed as an important next step in understanding these rhythms in everyday life conditions to design personalized diagnostics and interventions.  

Longitudinal measures of activity and physiology obtained with passive sensing have been used to either model the SCN rhythm or rhythms of processes such as activity, heart rate, or body temperature. While the latter may not strongly correlate with the SCN rhythm, as they are affected by various other factors, their characterization can still provide information about health status. Moreover, the study of multiple rhythms can shed light on how these rhythms and their alignments affect our health. We now explain four rhythm models. 



 §.§ Template curves

One simple way of modeling typical rhythms of the data is to model a "template" curve for the day.  A template curve is created by averaging (or calculating the median value) across all days for each time slot in the day (e.g., every minute, hour). Activity curves <cit.> were proposed to model the probability of each activity happening at each time slot when the measurements are not continuous signals but rather, events. 

These curves can be used for anomaly detection if values deviate more than a certain amount from the template. The main issue is defining the time over which to calculate the template. They have also been used to obtain deviation from the template as a feature for classification purposes. 

While calculating template curves is simple, one of the major drawbacks of curves is the lack of context. A single template curve cannot account for differences due to day-to-day variations or monthly variations. Of course, template curves could be created for each day of the week or every day of the year, as is done with weather data, but it could lose representational power. 



 §.§ Cosine models

Rhythms are most commonly modeled as sinusoidal functions, characterized by multiple parameters. The first parameter is its period. As such, circadian rhythms are those with a 24-hour period ( the average circadian period in humans is 24.2h), while an ultradian (infradian) rhythm is one with a period of less (more) than 24h <cit.>. While 24 hours is usually assumed for the period, due to its relevance for the internal master clock, frequency analysis methods like the Fourier Transform, wavelets, spectral analysis, and periodograms can be used to extract personalized periods from the longitudinal sensor data. This period analysis can help uncover individual variations, as well as various rhythms. 

By modeling rhythms as a sinusoid, its amplitude, MESOR, and Phase describe the rhythm of behavioral or physiological signals. Phase measurements are of particular relevance as they enable the description of the relative timing of physiological rhythms with environmental and behavioral cycles. These metrics are obtained by fitting the sensor data to a cosine wave of a predefined period using the least square method — this is called the COSINOR method <cit.>. Equation <ref> shows a single component COSINOR, other models include additional components to consider, for instance, the effect of light in the rhythm. Interested readers are referred to <cit.> for a complete discussion of circadian rhythm models.   


    Y(t) = M + Acos(2π/τ + ϕ ) + e(t)



While characterizing the rhythms of physiological and behavioral processes is important, the importance of understanding the relation between multiple rhythms should not be disregarded. Afsaneh et.al.<cit.> proposed a framework that combines period finding with COSINOR to independently describe rhythms of various signals and then uses these descriptors to predict depression risk. Runze et.al. <cit.> also models a cosine rhythm for each signal and finds that rhyhtmic models are correlated with productivity. While these are  preliminary results in a specific population, they shows the potential of combining multiple rhythms in the development of digital health services.  



 §.§ Models to predict SCN rhythm

As previously mentioned, one common goal of modeling rhythms through sensor data is to model the SCN rhythm: predict its period, phase, and amplitude. The gold standard to measure the SCN rhythm is by taking multiple samples of melatonin, cortisol, or core body temperature under highly controlled conditions like dim light or a constant routine protocol <cit.>. These procedures are costly and invasive, so passive sensing provides a cost-effective and ecological way to predict it. 

The COSINOR method has been used with actigraphy (accelerometer-based) and physiological data to model the SCN rhythm with relative accuracy in ambulatory settings. To account for influences on the measurement other than the circadian rhythm, various mathematical models of the clock have been proposed <cit.>. 

In most cases, though, passive sensor data is used to predict the onset of melatonin (time of peak), which is a gold standard metric of the SCN phase <cit.>. Deep learning and traditional machine learning techniques involving sensor measurements of activity, skin temperature, heart rate, and other physiological processes have been proposed for this purpose <cit.>. While these methods are promising, other studies have shown that the error could be higher in populations who may benefit the most from precise metrics such as shift workers, jet-lagged people, and people with sleep disorders <cit.>. Novel methods still need to be evaluated with more diverse and specific populations.  

Characterizing the SCN rhythm is pivotal to further our understanding of its role in health outcomes, however, researchers also acknowledge that different processes may exhibit different rhythms which need not have the same characteristics as other markers such as the melatonin onset (DLMO). Their characterization is useful for understanding underlying processes and might further increase our understanding of the role of the alignment of various rhythms with health. The methods to analyze such rhythms are similar to those described above, but models may consider the effects of confounder factors. For instance, to analyze the circadian rhythm of heart rate, Bowman et.al.<cit.> first removes the effects of activity, to uncover the underlying rhythm.



 §.§ Rhythm Models excluding Cosine Regression

Since sensor data may not always follow a cosine waveform, non-parametric metrics have also been used to describe the rhythmic properties of sensor data, particularly activity data. These metrics are explained in Section <ref>. Non-negative matrix factorization (NMF) has also been proposed to discover typical daily rhythms <cit.> and multiday <cit.> rhythms without the need for a regression. A dataset of the daily or multi-day sensor signals is created where each column represents an individual's data, which can be daily, weekly, or monthly data. NMF factorizes this dataset (a matrix) to find a small number of common patterns, such that each individual's data can be represented as a linear combination of the patterns. This method is also used for routine modeling (Section <ref>) by creating a matrix of events. 

















§ ROUTINES


One approach to modeling long-term patterns is through routines. A routine is generally thought of as a sequence or set of actions performed regularly. 

While there is no agreed formal definition of a routine, the term is usually used to describe recurring patterns of discrete events such as daily living activities, work tasks, or visited locations. As such, two main dimensions encompass a routine: co-occurring or sequences of events and their temporal recurrence. In contrast to rhythms that focus on the periodicity of sensor measurements, routines focus on the frequent repetition of events.   

Interest in computationally modeling routines is longstanding. Their computational study first emerged as a way to support everyday tasks: routines enable the prediction of the next event, which helps a system provide effective recommendations, assistance, or information for that event. Understanding routines from sensor data has been studied to analyze daily life routines <cit.>, family routines <cit.>, work-place routines <cit.>, and travel routines <cit.>. In the context of healthcare applications, routines help understand behavioral patterns that can be associated with health outcomes either positively or negatively <cit.>.



 §.§ Models of routine

Routines are usually modeled as repetitive sequences of events. As such, to discover routines from sensor data, it is first required to define and identify events. What makes a sequence "repetitive" is considered from multiple points of view: the frequency of occurrence (frequentist approach), the likelihood of occurrence (bayesian approach), and the periodicity of occurrence.  




  §.§.§ Defining events

In the context of identifying routines from sensor data, an event can be defined as a significant occurrence or activity that can be detected or inferred from the data collected by sensors. It represents a specific action, behavior, or state of the individual being monitored.  An important characteristic of events is their duration, that is, they are not instantaneous points in time, but rather an entity with a start and end time. In general, an event characterizes what is being done, by whom, when, and where. 

Events can vary widely depending on the context and the type of sensor data being analyzed. For example, in the context of daily activities, events could include actions like cooking, walking, driving, sleeping, or working out <cit.> while in the context of travel routines, events correspond to locations visited <cit.>. Commonly, an event is described by a set of sensor readings obtained during a time window which may include changes in sensor readings, such as motion, location, temperature, or other relevant parameters, which correspond to the occurrence of the event and can be used to classify them <cit.>. 

Defining and identifying events can be a difficult task. Event definition depends on the sensor technology used and the probable duration of events. In some cases, events are defined directly by the sensor readings obtained during a window of time. In such cases, the number of events can be explosive, and unsupervised learning techniques are used to group them into a more manageable set (see next subsection). Events can also be identified by a , requiring supervised learning to identify them. The activity recognition literature deals with this problem and has been extensively reviewed elsewhere <cit.>. 







  §.§.§ Routines as highly probable events

Initial concepts of routine were based on daily routine, that is, how the day unfolds. The behavior of a particular day can be expressed as a vector b, describing the event that occurred at each particular time. This vector can be categorical, that is, each item bt identifies the event that occurred at time t as a category; or binary, that is, the item b_jt has the value of 1 if the event j occurred at time t. A matrix of multiple days ℬ = D× b   describes the behavior over some time. 

From this matrix, routines can be extracted. The concept of eigen-behaviors, representing the eigenvectors of this matrix, was used by Eagle et.al.<cit.> to find common behaviors in a day. An eigen-behavior represents an intrinsic behavior pattern, eigenbehaviors, then, can be used to describe the overall routine of an individual as any particular day can be described as a linear combination of eigenbehaviors. While a predefined 24-hour period was assumed in this work, different behavior vectors were found for different situations showing that “eigenbehaviors” describe both typical patterns and variations of them.  

Similar to this approach, topic models, a technique of natural language processing to automatically discover topics in a document collection, were used to find "topics" of behaviors <cit.>. In these models, an event is considered a word, each day is a document and a set of days is a document collection. A topic describes a group of behaviors that typically occur together in a day, and each day can be described as a combination of topics. 

One drawback of the previous methods is that they are based on individual data, suffering potentially from a cold-start problem. Only when there is sufficient data from a person, can we learn their routines and make inferences from their behavior patterns. To overcome this drawback, a collaborative-filtering-based approach was proposed by Xu. et al<cit.>, which combines population data with personal data to find relevant behavioral features to infer depression.  

The idea behind these techniques is that there is a set of behaviors occurring at a particular time within predefined duration bins that occur repeatedly. While a particular day exhibits a different combination of these typical patterns, the emerging patterns describe the routines of a person and can be used to represent any particular day. 

The problem with this view is the lack of sequence representation. Each day is described as a “bag-of-events” with positional information. While this representation captures some of the sequential context, it cannot capture partial orders, such as events happening next to each other regardless of the time. For instance, “eigenbehaviors” can find patterns such as a late wake-up leading to a no-work day, it is harder to find sequences of activities that can happen together at no particular time. For example, a routine might consist of gym and then eating, but it can occur at different times of the day. Or if there is a sequence of office, gym, and then going home, always starting at 5 pm but having different durations for each depending on traffic, day, and others, it might be hard to discover due to the dependence on time and the predefined time bins. 



  §.§.§ Routines as highly probable sequences

Sequential models analyze data in the order of arrival, thus, capturing dependencies among the events. Both probabilistic and frequentist approaches have been used to model routines as sequences.  From a frequentist perspective, routines are events that empirically occur frequently within a given timeframe based on repeated observations, while from a Bayesian perspective, routines involve incorporating prior beliefs or knowledge about the likelihood of events occurring and updating these beliefs with new evidence.

Frequent-sequential pattern mining and association rule mining <cit.> algorithms work under the frequentist assumption to identify events occurring repeatedly together in an event log. The extracted routine is then expressed as rules <cit.>, action maps <cit.>, or as a grammar <cit.>.
Frequent sequences can represent typical sequences of events in the routine of a person but it is harder to model the uncertainty of multiple possible sequences. Another approach was proposed to consider context-based sequences <cit.>, to model frequent sequences that depend on the context such as day of the week or the weather, independently of the frequency of that context. 

From a probabilistic perspective, Bayesian Networks, which include hidden Markov models, have been used to learn and model the sequential dependencies of events <cit.>. These networks model the transition probabilities between events, as well as the correlation between events and sensor readings. A common problem is defining the structure of the network to learn the parameters. Banovic et.al. <cit.> proposed a model based on reinforcement learning to learn the structure. The learned network of events describes the behavior of the person. By considering the context, such as location or time, in the description of the event, variations in behavior are modeled as well. However, the combinatorial explosion of the number of events and the need to manually describe them make it a computationally expensive model. 



   

Both models have been used extensively in research, their main strength being the possibility of inspecting the routine and knowing exactly what is the next event, making them appropriate for intervention design. Their other strength is the possibility of modeling contextual changes. This is done either by adding it to the event definition or as part of the mining processing. In sequential patterns, modeling includes deciding the maximum length of the sequence and whether gaps or different orderings can be included. These decisions may affect the final learned routine. 

Deep learning models have also been used to model longitudinal sensor data. For this, the events are defined by slices of data which can be a day of sensor events. Due to their black-box nature, Deep learning models have mostly been used in tasks to associate behavior with health outcomes. Recently, Xu et.al. <cit.> proposed a 1d Convolutional network to detect depression, which also learns the continuity of behavior. This can help learn the behavioral patterns but with the drawback of not being interpretable.
While deep neural networks such as LSTM and GRU have been used to model other sequential data, their use to model routine from sensor data has been limited.  

As mentioned before, models of routines are based on the definition and identification of events. This includes multiple assumptions about the duration of events, often left as predefined bins or windows, which may not accurately represent the duration of all activities leading to errors, and about the accuracy of identification. The propagation of errors from the identification of events to the modeling of routines can affect the usefulness of the model. While the events have also been defined as sensor event counts or statistics over predefined windows of time, this can lead to a combinatorial explosion in the number of possible states or require manual definition of thresholds to limit the possibilities. This is still a major drawback for routine analysis.











§ QUANTIFYING BEHAVIORAL STABILITY WITH LONGITUDINAL SENSOR DATA



Both routines and rhythms model long-term sensor data, focusing on different characteristics of the patterns (periodicity, frequency, sequential dependencies). Other approaches focus on quantifying the strength of repetitiveness without modeling the specific patterns.
A person with high strength (or stability) can be thought of as having a more regular or strict routine while a person with lower strength will exhibit more variability in their day-to-day life. While these metrics can't describe the patterns, they characterize the repetitiveness of behavior. 
The simplicity of a single metric makes metrics attractive to perform statistical analysis, as a tool for early screening, and because they can be more easily interpretable than other models. For instance, less table and more fragmented rhythms have been associated with depression, higher BMI and smoking <cit.>. 
Research focuses on finding the relation between metrics and physical, mental, or cognitive health.  These metrics can be classified as being uni-modal if they consider a single sensor modality, or multi-modal if they consider multiple sensor modalities at the same time. A summary of metrics is presented in Table <ref>, and they are discussed below. 






 §.§ Rhythm-based metrics

Rhythm-based metrics use a single sensor modality to obtain a measure of the strength of the rhythmicity of behavior, that is, how similar the measurements taken at the same over multiple days are. The interday stability, intraday variability,  and relative amplitude (RA) metrics come from circadian rhythm research, as non-parametric measures of the strength of the circadian rhythm and are most commonly calculated using activity counts from actigraphy devices. They measure the strength, fragmentation, and amplitude of the rhythm, respectively. 

The regularity and stability indexes have been proposed to measure the regularity from a variety of sensor modalities, but still consider only one modality at a time. They focus on measuring the similarity of events happening at the same time over multiple days. 
The regularity index measures how similar measurements taken at the same hour over two different days are, while the stability index compares the normalized cumulative distribution value at each hour over two days and then offers a value for a period by comparing all days to each other. 

These metrics rely on having an aggregated measure, usually hourly, to compare the value across multiple days. While the aggregation could be in smaller segments, it is not clear how the aggregation might affect the result. 



 §.§ Event based metrics

Instead of relying on the measurements directly, event-based metrics rely on the detection of an event —i.e., an activity, a location— and then calculating the regularity of occurrences of this event over some time. For instance, the standard deviation in start/end time or duration of the event is a measure of how variable that event behavior is. The sleep regularity index focuses exclusively on the regularity of sleep schedule by measuring if the individual is asleep at the same times over the period being examined. Entropy measures the extent of repetition of events, regardless of their order, considering X as a random variable representing the activities in a day. For a person with a more stable routine, this random variable will have lower entropy, as the events are generated by a more regular process. Entropy measure is chosen for event-based analysis as the events are a categorical variable. The flexible regularity index <cit.> measures the edit distance between two days. By representing each event with a letter and considering the daily sequence of events as a character sequence, the edit distance calculates 







p2cmp5.5cmc 
Metric     Description     Formula 
 3cSensor value-based metrics

         Intraday Stability <cit.>     A measure of how stable a person’s rhythm is over multiple days, calculated from n samples of activity levels x, aggregated hourly q hour-bins. x̅_̅h̅ represents the average activity during hour h. IS scores range from 0, indicative of a total lack of rhythm, to 1, indicative of perfectly stable rhythm     IS= n ∑_h=1^q (x̅_̅h̅ - x̅)^2/q ∑_i=1^n(x_i-x̅)^2

         Interday Variability <cit.>     A measure of the fragmentation of activity or how activity level shifts between two consecutive time slots calculated from n samples of activity levels x, aggregated hourly q hour-bins. IV scores range of 0 to 2 with higher values indicating higher fragmentation     IV= n ∑_i=2^n (x_i - x_i-1)^2/(n-1) ∑_i=1^n(x_i-x̅)^2

         Rest-activity Amplitude <cit.>     Measures the difference between a person’s daily activity (measured as the mean activity level over the most active 10 consecutive hours (M10)) and nocturnal activity (measured as the mean activity during the least active 5 consecutive hours (L5))    RA = M10 - L5/M10 + L5 

         Regularity Index <cit.>     Compares the uniformity of a person’s schedule, comparing hourly (t) data from two days (a,b). Sensor measurements (x_a^t) are normalized to (-0.5, 0.5)     RI_a,b = ∑^T_t=1x_t^a x_t^b/T

         Stability Index <cit.>     Measures the inverse of the median across a period of multiple days P  of the distances between the normalized cumulative activity distributions (C_b^d_1) on two different days. It characterizes the degree to which behavioral patterns varied across the period. A higher index means higher stability.      
SI_b (P)= 1 - median ( D_b(i,j )|i,j ∈ P , i ≠ j 
 where D_b(d_1,d_2) =  1/M∑_i=1^M |C_b^d_1 - C_b^d_2|

         Circadian Rhythm  <cit.>   Measures the strength with which a user follows a 24-hour rhythm in behaviors by calculating the ratio of the energy that fall into the 24 ± 0.5 h period over the total spectrum energy in the 24 ± 12 h period. Here, psd(x) denotes the power spectral density at frequency bin x.     CR = ∫_2π/23.5^2π/24.5psd(x)dx/∫_2π/12^2π/36psd(x)dx

         

         3cEvent-based metrics

         Standard deviation <cit.>     Measures the within-individual variability in a derived parameter (x) of an event. For example, the start time or duration of the event. A higher value indicates more irregular behavior.     s = √(1/N-1∑_i=1^N (x_i - x)^2) 

         Sleep regularity index <cit.>     The probability of an individual being in the same state (s_i,j = asleep or awake) at any two time points 24 hours apart, averaged over the number of days being studied (M). SRI values of 0 represent a random sleep schedule and a score of 100 represents a periodic sleep schedule. N is the number of time points in a day.     SRI = -100 + 200/M (N - 1)*
   ∑_j=1^M∑_i=1^Nδ (s_i,j,s_i+1,j) 

         Flexible Regularity Index <cit.>    Edit-based distance to measure the difference between two days by counting the number of operations (insert, delete, substitute) to convert one day's events into the other day's events. Higher values indicate more dissimilar days.      

         Entropy <cit.>     Measures the repetition of isolated events (i.e. travel events, activities), where E is the set of all possible events. A value of zero indicates no uncertainty (more stability) and a value of 1 indicates all events are equally possible     H(X) = -∑_x in E p(x)log_2p(x)

          [6pt]
         3cMultimodal metrics

         Entropy Rate of a Sequence <cit.>    Measures the extent to which ordered subsequences of events repeat over time. p_n denotes the joint probability distribution of a subsequence of length n     H(X) = 
 lim_x→∞ - ∑_x_1^n ∈ E^n p_n(x_1^n)log_2p_n(x_1^n)/p_n(x_1^n-1) 

         Multiscale Fuzzy Entropy <cit.>    Measures the complexity of a time series    See reference. 

         

Behavior stability/variability metrics



 


 §.§ Multimodal metrics

Previous metrics measure the stability of behavior considering one measured variable, but they can't measure relations across various behavioral and physiological measures such as sleep, steps, and heart rate. Amon et.al. <cit.> proposed an approach based on Multidimensional Recurrence Quantification Analysis (MdRQA) to provide a metric of the stability of behavior from multiple sensor signals. This method quantifies the interactions between multiple signals, thus offering a more holistic view of the repetition of patterns, which can also be understood as contextualizing the signals and their regularity. Similarly, the entropy rate <cit.> looks at the regularity of multiple events instead of an isolated event by considering the sequence of events. 






 §.§ Metrics summary

Metrics are valuable because they provide a quantifiable marker whose correlation with a health outcome can be more easily measured than the whole description of longitudinal data. Metrics can also provide personalized insights as they quantify the individual's behavior over some time. Commonly, researchers use metrics to evaluate their correlation with health conditions such as depression, schizophrenia, or cognitive decline. The strength of that correlation measures the usefulness of the metric.

As was discussed, most metrics rely on a single signal modality and a given time frame, thereby restricting the analysis. It is common to calculate metrics for all signal modalities separately and to calculate them using different groups of days. For instance, it is not uncommon to calculate the standard deviation across all days, across weekdays and across weekends. A combination of these metrics are usually used as input to a machine learning models to improve the predictive power. Multimodal metrics can provide a more holistic or contextualized view of the stability or variability of the behavior. However, it is possible that a single metric can hide some effects. For example, He-Yueya et.al. <cit.> found differences in the meaning of stability. For some signals, greater stability was desired for better outcomes but, for others, more variability was associated with better outcomes.  An improved quantification of the relation between the variability/stability of different signals could help improve our understanding of the influence of behavior in health. 

















§ DISCUSSION


Having examined three approaches to analyzing longitudinal sensor data, we now move on to discussing some open research challenges. 



 §.§ Choosing a model

The first key open question is how to choose a model for long-term sensor data analysis. A key consideration is the goal of the patterns. Common uses include (1) to find associations with health outcomes, to which metrics have an appeal due to their simplicity; (2) to provide assistance and predictions of the next events to which routines may be more suited; and (3) to find change and deviations, to which possibly all three models may help. 

A combination of models is not uncommon, especially in conjunction with machine learning and deep learning for all three tasks. For this, the following open issues need to be solved as well.  



 §.§ Windows and window length

During the analysis of longitudinal sensor data, an important consideration is the amount of time (window length) to analyze. It is important to recognize two different types of windows, neither more important than the other: one window to aggregate sensor data as features or events and another to analyze the patterns in sensor data. 

The first type of window, the event window, defines the length of time that is used to recognize an event or a feature (such as the number of steps taken or the average heart rate) that will later be used to obtain patterns in the form of routines, rhythm, or (ir)regularity metrics. This window is often defined in the range of seconds, a few minutes, or one hour. Extensive reviews have looked at the impact of the size of this window, for example, in activity recognition <cit.>. Automatic segmentation has also been studied for this problem, where the window size is automatically determined by changes in the data distribution <cit.>. 

The second type of window, the pattern window, considers how historical data is used to analyze the patterns, for instance, 7 days or 1 month. Routines, rhythm features, and metrics might change depending on this length. Very little work has been done in the analysis of the impact of this length or how to define it. Most commonly, multiple window lengths are evaluated to find the one with the most predictive value for the problem at hand <cit.>. In general, shorter window lengths will have a narrow but current view of the person's patterns, while longer windows will have a wider view but it could be influenced by outdated patterns— the weight of history is heavy. 

A lack of standards in the size of windows hinders the comparability of results and efforts for data sharing. It is not uncommon to find datasets that aggregate data in a too coarse way, making it harder for others to evaluate or compare results that were obtained with smaller windows. 



 §.§ Baseline establishment and change analysis

Related to the problem of determining the pattern window length is the issue of change analysis. Ideally, when analyzing longitudinal data, one is interested not only in the patterns found in data but also in identifying if any pattern changes have occurred. It is, after all, changes that may be more correlated with health changes or health outcomes after an intervention <cit.>. 
However, most of the research so far focuses on analyzing the current behavioral patterns and the predictive power of these patterns for the health outcome being studied, and there is less research on how to analyze changes in behavior across windows. 

Routine analysis has been used to identify changes in routine and anomalies by comparing current behavior with the learned models <cit.>. To analyze changes in phase or amplitude of circadian rhythms over time, Bayesian spectral analysis and detrended fluctuation analysis have been used <cit.>. Changes are also detected statistically <cit.>. Some approaches based on distribution changes evaluate whether behavior changed in two different windows of time by comparing the distribution of sensor data of one window to that of the other window <cit.>. The work by Cook et. al <cit.> determines behavior change over two time windows as a probability distance. By defining the average probability that an activity occurs during a time interval (i.e. an hour of the day) over the time window, it defines behavior as a probability distribution for each time interval. It then calculates the distance between two time windows using the KL divergence of the two distributions.  


To quantify the change, <cit.> uses the slope of the data to quantify how values are changing inside a given window, but it does not quantify if the stability/rhythmicity is changing over various windows. Cook et.al. <cit.> use a  permutation-based test to determine the statistical significance of detected changes. 


The problem of detecting and quantifying behavior or rhythm change can also be framed as a change point detection problem. While there are multiple time series change point detection methods <cit.>, these methods usually focus on the raw sensor values, which can be less meaningful for wearable sensor data, as the changes might be too granular or too fast. For instance, a changing heart rate due to exercise. For longitudinal data, the relevant changes span weeks or months of baseline pattern.


Other works frame the problem as an anomaly detection problem <cit.>. Anomaly detection algorithms analyze deviations from current patterns to detect abnormal activities, heart rate, or other abnormal values, but they do not focus on identifying if the baseline patterns are changing. In other words, while they detect a momentary deviation, they assume the overall pattern remains constant. 


Overall longitudinal pattern change analysis remains elusive, due to the difficulties in obtaining enough data for the analysis, the difficulties in defining time windows for the analysis, and the difficulties related to the evaluation of any such approaches.  




 §.§ Model Evaluation

Evaluating how accurately the routine, rhythm, or behavioral stability of a person is, is undoubtedly a difficult task. Unlike other tasks, such as supervised classification of activities, there is no ground truth data that can be used to compare the model. 
Routines have been evaluated by letting the user review the output and agree/disagree on it. However, this is still a subjective evaluation. For rhythms, comparison with rhythm parameters of the melatonin or cortisol provides an error measure, but as was discussed, this can be misleading, as different processes may have different rhythms.  

Recently, the evaluation focus has shifted to predictive power, that is, how well the patterns or metrics can detect or predict a health outcome (hospitalization, schizophrenia, depression, etc). These evaluations are important, as they focus on the clinical importance instead of computational metrics, but they are also elusive as they depend on the population in the sample, and have proven difficult to generalize. Moreover, they are difficult to translate to other uses of longitudinal pattern analysis, such as providing assistance with a task or identifying abnormal patterns. 



 §.§ Data Heterogeneity and Data Quality

Related to the evaluation and generalization of results is the data heterogeneity issue. Different platforms, vendors, form factors, users, and software libraries result in significant differences in data <cit.>. These differences can result in very different pattern estimations, whether in routines or rhythms.  While metrics are developed and hand-crafted to find associations with health outcomes, their values can differ significantly due to the high heterogeneity, making it hard to generalize across different domains (sensing platforms, population groups, time) <cit.>. 

For this reason, deep learning methods have been proposed as a way to find more stable features in the sensor data to classify and predict health outcomes that could have behavioral symptoms mainly with multi-task learning to learn patterns in the different domains <cit.>. However, the generalization issue persists. 

A second issue for data quality is the inevitability of missing data <cit.>. When data is being collected in natural settings, missing data may come due to battery issues, non-wearing events, or due to platform restrictions <cit.>. 

Proposals to deal with missing data include interpolation techniques and discarding data that contains missing data over a threshold percentage <cit.>. There are no standards set for this and the correct management of missing data is still an open research issue. 



 §.§ Context

Another issue in the current analysis methods is the lack of context in most of them. Several contextual factors can affect routine, rhythms, and the different metrics, including weekends, holidays, weather, or family visits. Not considering this can result in biased metrics, misunderstanding of the patterns, or a lack of insights into what causes changes in stability.  
Some models of routine consider contextual variations as a way to explain routine deviations and differentiate them from changes in behavioral patterns <cit.>. Some metrics, such as the regularity index, consider the day-to-day similarity, which may consider differences due to day-of-week changes, but not for other contextual factors. As was mentioned, this is also considered in the calculation of other metrics, such as the standard deviation, by filtering the set of days used as input. However, efficient computation techniques to deal with multiple contextual variables, as well as their possibly combinatorial explosion, are less often considered. 

Understanding the association with context might be critical for designing interventions for health, and for providing more holistic insights, therefore, including context in the modeling and analysis of behavioral and physiological signals is critical.  



 §.§ Limitations of this study

This study does not adhere to the methodology of a systematic literature review, and consequently, may be subject to a degree of bias in the selection of papers. The myriad of terms and techniques employed in the field posed a significant challenge to conducting a systematic review. Instead, the primary focus was directed towards identifying and establishing commonalities and distinctions among the various models, as well as their respective applications. The study aims to encompass a wide array of literature to furnish a comprehensive overview of the models pertaining to long-term patterns. 

 







§ CONCLUSION



In this state-of-the-art review, we explored computational models for making sense of longitudinal wearable sensor data for healthcare applications. Our examination of rhythms and routines revealed their distinct roles in capturing temporal patterns.  Additionally, we provide an overview of critical challenges related to data heterogeneity, quality, and temporal granularity for the analysis. 

Both routines and rhythms are used to model typical recurring patterns. While routines focus on the definition of an event, rhythms focus more on periodic patterns on the signal, overcoming the need for specific event recognition. 
Both models describe the data and can be used for predictions (of the next event in the case of routines or the next peak in the case of rhythms). Nevertheless, the evaluation of their accuracy and the results remains challenging as models are often highly dependent on the sensors used.  As a way to mitigate this, researchers have focused on developing metrics focusing on the strength of the routine or how typical any particular day can be.

Several open research challenges exist including choosing the right model for data analysis, determining window lengths for analysis, and dealing with data heterogeneity and quality.  There is a need for context in analysis as a significant gap in current research pertains to the exploration of the interrelation between behavioral changes and physiological alterations. Future research should explore the integration of wearable data with other health-related sources to create a holistic view of individual well-being.



plain




 Paper Title: Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress?.





 
    
       










         




Y>X
s>=.3Y
t>=.7X
bX
u>=0.8Y



































?!width 1.1pt

@@t@̊@t
@̊@t#1#2
0=@@t#1#20=0
0-0.20
2=height0 depth -0
00.4pt2
√()



return

function
SE[FUNCTION]FunctionEndFunction
   [2] #1#2(#2)
    










storyfloathtbploa
storyfloatStory


story[section]
story[1][]
story
#1

frametitle=
[baseline=(current bounding box.east),outer sep=0pt]
[anchor=east,rectangle,fill=dodgerblue]
Story;


frametitle=
[baseline=(current bounding box.east),outer sep=0pt]
[anchor=east,rectangle,fill=dodgerblue]
#1;

innertopmargin=10pt,linecolor=dodgerblue,
linewidth=2pt,topline=true,
frametitleaboveskip=−,
[]





  
   
 
     

storybox[2][Story]
    title=#2,  
    fonttitle=11pt15pt,  
    coltitle=customTitleText,  
    colbacktitle=customTitleBg,  
    colback=customContentBg,  
    colframe=customFrame,  
    parbox=false, 
    breakable









affilsepx    





1,2]Richard RenEqual Contribution.

1]Steven Basart1

1,3]Adam Khoja
1]Alice Gatti
1]
Long Phan
1]Xuwang Yin
1]Mantas Mazeika
3]Alexander Pan
4]
Gabriel Mukobi
1,5]Ryan H. Kim
6]Stephen Fitz
1]Dan Hendrycks
[1]Center for AI Safety
[2]University of Pennsylvania
[3]UC Berkeley
[4]Stanford University
[5]Yale University
[6]Keio University




Safetywashing: Do AI Safety Benchmarks
Actually Measure Safety Progress?
    [
    February 2024
========================================================================



§ ABSTRACT 



As artificial intelligence systems grow more powerful, there has been increasing interest in “AI safety” research to address emerging and future risks. However, the field of AI safety remains poorly defined and inconsistently measured, leading to confusion about how researchers can contribute. This lack of clarity is compounded by the unclear relationship between AI safety benchmarks and upstream general capabilities (e.g., general knowledge and reasoning). To address these issues, we conduct a comprehensive meta-analysis of AI safety benchmarks, empirically analyzing their correlation with general capabilities across dozens of models and providing a survey of existing directions in AI safety. Our findings reveal that many safety benchmarks highly correlate with upstream model capabilities, potentially enabling “safetywashing”—where capability improvements are misrepresented as safety advancements. 
Based on these findings, we propose an empirical foundation for developing more meaningful safety metrics and define AI safety in a machine learning research context as a set of clearly delineated research goals that are empirically separable from generic capabilities advancements. In doing so, we aim to provide a more rigorous framework for AI safety research, advancing the science of safety evaluations and clarifying the path towards measurable progress.














§ INTRODUCTION







“For better or worse, benchmarks shape a field.” — David Patterson

Artificial intelligence (AI) systems have rapidly advanced in recent years and are increasingly deployed in high-stakes scenarios. This has led to growing interest in ensuring that AI systems are not only more generally capable, but also more trustworthy and safe. Under the umbrella of AI safety research, a wide variety of benchmarks have been proposed that claim to measure desirable safety properties, distinct from the general capabilities of models. This includes the extent to which models are fair <cit.>, reliable <cit.>, honest <cit.>, or less prone to malicious use <cit.>. In each case, intuitively plausible arguments can be given for why the model property is not mainly determined by upstream general model capabilities (that it will not be “automatically solved with scale”). However, these intuitive verbal arguments have rarely been empirically scrutinized and often admit counterarguments that are equally convincing. This raises the question of what exactly constitutes advancements in “AI safety” from an AI developer R&D perspective, how to measure it, and how to distinguish it from upstream general capabilities. 



Distinguishing safety properties from the model’s upstream general capabilities is challenging because they are intertwined. More capable AI systems are less likely to cause random accidents, but at the same time could cause more harm if used maliciously. 
AI systems that are better aligned with human preferences may avoid hazardous behavior but may also be far more capable because humans prefer intelligent assistants. 
This complicated relationship 
obscures
differential safety progress, or technical improvements that disproportionately improve safety properties of AI systems relative to other attributes.
In computer systems, for example, performance and security improvements are more readily distinguishable; were they as intertwined as in AI, mere speed enhancements might be misrepresented as security research. In the worst case, this blurred distinction can be an instrument for safetywashing, where techniques that do not disproportionately contribute to the safety properties of AI systems relative to other properties are misconstrued as “safety research.”

Historically, there have been two approaches for identifying machine learning research topics for differentially improving the safety properties of AI systems. 
One paradigm is alignment theory, a highly discursive, top-down, and intuition-driven approach that backchains from high-level risks to concrete empirical machine learning subproblems. The other approach is bottom-up and involves patching current systematic flaws in AI systems. An example of the former is alignment of large language models (LLMs) to human preferences <cit.>. An example of the latter is distribution shift robustness <cit.>. However, both approaches guide research problem selection 
that may not be sufficiently distinct from latent upstream capabilities, consequently opening the door to safetywashing.

In this paper, we present a third approach to identifying distinct AI safety research topics and benchmarks: we empirically measure whether common safety benchmarks are highly correlated with capabilities across common chat models. Instead of relying on intuitive arguments, we compute the correlation between various safety metrics and a general capabilities component that explains around 75% of model performance across a wide variety of capabilities benchmarks.
While a high correlation indicates that a safety benchmark is measuring capabilities as a latent upstream factor—and is thus prone to safetywashing—a low correlation does not necessarily speak to the quality of the benchmark.

In extensive experiments across dozens of models and safety benchmarks, we find that many safety benchmarks have high correlations with capabilities. 
Our findings suggest that merely improving general capabilities (e.g., through scaling parameters and training data <cit.>) can lead to increased performance across many safety benchmarks.
This is troubling because AI safety research should aim to enhance model safety beyond the standard development trajectory. 
Separately, we find that alignment philosophy's intuitive arguments can mislead researchers, since it is highly disconnected from empirical measurements.
We find that it is difficult to predict ahead of time which benchmarks are uncorrelated with general capabilities. 
This shows that empirical measurement is needed, so 
we recommend that future safety benchmarks report their correlation with upstream model capabilities.
Ultimately, we provide empirical clarity to the concept of “AI safety” as a set of clear, delineated research goals that are empirically separable from generic capabilities research.



































































§ RELATED WORK







Science of evaluations. 
The development and analysis of benchmarks for evaluating AI models, particularly LLMs, encodes desirable properties of models and sets goals for guiding model development. 
Previous work has further aimed to build open-source evaluation platforms <cit.>, analyze model scaling through benchmarks <cit.>, conduct factor analysis across benchmarks <cit.>, and predict downstream capabilities <cit.>. Furthermore, concurrent work has used principal component analysis to analyze performance between benchmarks <cit.>. However, while many safety benchmarks have been made, no works to date have conducted an empirical meta-analysis of safety benchmarks to investigate the entanglement between safety benchmark scores and upstream model capabilities. 


Differential safety progress. Differential safety progress in AI systems refers to the relative advancement of safety properties compared to overall capabilities <cit.>.
Some methods have resulted in differential progress in AI safety <cit.> by demonstrating marked improvements in model robustness without necessarily increasing general upstream capabilities. These techniques exemplify the potential for targeted safety improvements that are orthogonal to the default trajectory driven by capability enhancements.
<cit.> emphasize the goal of steering AI development towards safer systems that deviate positively from the default capability trajectory; they present a philosophical discussion with narrow empirical analysis.





















§ METHODS










































We derive a simple and highly general methodology for determining whether a safety benchmark is entangled with upstream model capabilities.





  
Capabilities score. To establish a capabilities baseline, we collect scores from m models on b capabilities benchmarks (e.g., MMLU <cit.>, Winogrande <cit.>, GSM8K <cit.>). We form a matrix of results from benchmarks, which we call the benchmark matrix B ∈ℝ^m × b , where B_ij is the score of the ith model on the jth benchmark. We normalize each column of B to have mean 0 and variance 1. We perform Principal Component Analysis (PCA) on B to identify the unit first principal component vector PC_1. The capabilities score for each model is given by projecting the model's benchmark scores onto PC_1. Because PC_1 of B represents the direction in the space of benchmark performances along which models vary the most, we obtain a general measure of the model's capabilities. The capabilities score for model i is


    Capabilities Score_i = (B ·PC_1)_i   for  i = 1, …, m.




  
Capabilities correlation. For each safety benchmark, we evaluate the same set of m models, redefine metrics such that a higher score indicates improved safety[A subset of score calculations were changed to ensure higher benchmark scores indicate greater safety across metrics. For example, proportion of successful attacks, which is often reported for adversarial robustness, were recalculated as proportion of failed attacks.], and normalize the safety benchmark scores to mean 0 and variance 1. We compute the Spearman correlation across models between the capabilities scores and the safety benchmark scores:

    Capabilities Correlation = corr_models(Capabilities Score, Safety Benchmark).


r0.5



Model     Capabilities
Score 

Mixtral 8x22B Instruct v0.1     4.85 

Llama-3 70B Instruct     4.58 

Llama-3 8B Instruct     1.10 

Mistral 7B Instruct v0.2     0.72 

Falcon 40B Instruct     0.54 

Llama-2 7B Chat     -1.86 

Gemma-1.1 2B Instruct    -4.07 

Qwen-1.5 0.5B-Chat     -7.56 


Relative capabilities scores for a subset of chat models.




A high correlation indicates the benchmark likely measures capabilities rather than distinct safety attributes. A low correlation indicates the benchmarks is measuring attributes distinct from general capabilities, while a negative correlation indicates models obtain worse safety properties as upstream model capabilities increase. 










  
Experimental setup for language models. We calculate the capabilities component from the following benchmarks: LogiQA <cit.>, PIQA <cit.>, Hellaswag <cit.>, Winogrande <cit.>, COPA <cit.>, MedQA <cit.>, ARC Challenge <cit.>, MMLU <cit.>, MATH <cit.>, LAMBADA <cit.>, GSM8K <cit.>, and BBH <cit.>. We used a diverse set of model classes and derivatives to avoid skewing results towards any particular model architecture, listing the 27 base models and 26 chat/instruct fine-tuned models used for our analysis in the Appendix. Running separate analyses for base and chat models, we find that 76.9% and 73.8% of variance is captured by the capabilities component, respectively. Our reported results use instruct fine-tuned models by default; generally, we find the forthcoming analysis does not change markedly when using base models. 



  
Experimental setup for vision models. For vision models, we use the accuracy of ImageNet as the capabilities component. We list the 63 adversarially trained models (used for vision adversarial robustness results) and 44 standard models (used for calibration results) in Appendix <ref>.












§ HUMAN VALUES


r0.5



    < g r a p h i c s >

A safety benchmark with a high correlation and high slope with respect to upstream model capabilities can be used for safetywashing.






Human values are the fundamental beliefs and ideals that guide human behavior and decision-making; researchers often aim to encode these values in AI systems. 
We assess common benchmarks for alignment and helpfulness (<ref>), machine ethics (<ref>), and bias (<ref>), asking whether such measurements are determined primarily by upstream model capabilities.



 §.§ Alignment





  
Area Overview. Alignment refers to how well AI systems follow the goals of their operators, accurately specifying and implementing the desired goals without unintended consequences or misinterpretations. Common alignment evaluations assess AI systems' helpfulness or instruction-following, with the aim to closely align the AI systems' responses with human preferences.



  
Datasets.
We describe the alignment benchmarks that we use below. Example inputs and outputs from these benchmarks are shown in <Ref>.

    
  * LMSYS Chatbot Arena <cit.> is a crowdsourced evaluation platform where users interact with two anonymous AI models simultaneously. Users pose questions to both models and vote for the response they prefer. The platform uses these votes to generate an Elo ranking system, providing a leaderboard of AI model performance based on public preferences.
    
  * MT-Bench <cit.> is a conversation benchmark consisting of 80 high-quality multi-turn questions. LLMs are used to evaluate responses, with the evaluation criteria designed to align closely with human preferences as determined through crowdsourcing.





Dubious Intuitive Arguments For and Against Researching “Alignment”
In this section we will cover key intuitive arguments for and against alignment, and thereby show how intuitive arguments and their underlying distinctions can be a highly fragile and unreliable guide for determining a research area's relation to upstream general capabilities and tractability.

We should work on alignment because:

1: Misinterpretation Risks. AIs could catastrophically fail to capture and abide by human intentions. “A system that is optimizing a function of n variables, where the objective depends on a subset of size k<n, will often set the remaining unconstrained variables to extreme values; if one of those unconstrained variables is actually something we care about, the solution found may be highly undesirable. This is essentially the old story of the genie in the lamp, or the sorcerer’s apprentice, or King Midas: you get exactly what you asked for, not what you wanted.” 
AIs smarter than us can always outthink us and find loopholes in our requests; trying to plug all the holes is hopeless, like patching all the holes in the tax code <cit.>. 

2: Misgeneralization vs. Goal Misgeneralization Distinction. Even if alignment is highly correlated with capabilities, it still needs to be the main focus because we need robust alignment not just robust capabilities. Indeed, “capabilities might generalize further than alignment techniques” when out-of-distribution <cit.>.
Goal misgeneralization is “an instance of misgeneralization in which a system's capabilities generalize but its goal does not generalize as desired” <cit.>.


3: Capability vs. Aimability Distinction. AI alignment is different from improving general model capabilities; alignment is not about capabilities but aimability. Capabilities refer to what the AI can do, while aimability refers to how amenable the AI is to being directed towards specific goals. Alignment is just about making models “helpful, harmless, and honest” <cit.> which is obviously necessary for safety.


We should not work on alignment because:

1: Alignment as AGI. If AI alignment is about getting AIs to satisfy our preferences, then that's such a broad mandate that it requires building AGI. Humans prefer smarter models. If we “align” AIs to preferences over outputs that vary in competence, we're training the AIs to be generally smarter.

2: Alignment as business alignment. The current operationalization of AI alignment reduces human values to human or AI preferences <cit.>, and further reduces these preferences to business-centric task preferences. This makes alignment the task of business alignment, namely aligning systems with preferences about code completion, summarization, copy editing and so on.
As a result, alignment benchmarks may primarily capture an AI system's capabilities in performing business-relevant tasks rather than its true alignment with broader human values and ethics. 

3: Philosophical challenges with preferences. Various types of preferences are not worth satisfying. Revealed preferences can be highly influenced by misunderstandings and misinformation. People can have revealed preferences for things that they will likely regret the next day. Optimizing for revealed preferences can lead to addiction and manipulation, like TikTok's addictive algorithm. Stated preferences are highly susceptible to framing effects and other cognitive biases. Some preferences, like the preference for drugs or to count blades of grass <cit.>, are not human values. People can also have malicious preferences, such the desire for the harm of others. Idealized preferences and fully informed preferences, while theoretically attractive, are not practically computable <cit.>. 












  
Empirical analysis of safetywashing.
We provide clarity to this debate. Is alignment with human preferences, as operationalized by standard benchmarks, mainly determined by upstream model general capabilities?



We preliminarily provide context for interpreting correlations. We note that the correlation between SAT and ACT math scores—tests designed to measure similar constructs—is 81.5% <cit.>. Similarly, we observe a mean correlation of 74.4% (σ = 15.1%) between capabilities benchmarks for instruct-tuned language models. Consequently, if “safety benchmarks” have similarly high correlations, they are not highly empirically distinct from upstream general capabilities. We treat correlations below 40% as a low correlation. 

r0.45



Alignment Evaluation     Capabilities
Correlation 

MT-Bench     78.7% 

LMSYS Chatbot Arena     62.1% 


Alignment with human preferences benchmarks are highly correlated with capabilities for chat models.





Our analysis of MT-Bench and LMSYS Chatbot Arena reveals high correlations between human preference alignment metrics and upstream model capabilities in chat models, even by the standards of capabilities metrics. We observe a similar but weaker effect in base models (MT-Bench correlation of 64.2%), with some of the best base models performing stronger on MT-Bench than many chat models.
Alignment evaluations largely measure upstream model capabilities; as a goal to guide model development, alignment metrics are very similar to other capabilities benchmarks. We can also see many of the distinctions made about human preference alignment and capabilities are distinctions without a difference. Yet, many labs still prioritize “alignment” as a safety-focused research direction.




It stands that labs can advance alignment benchmarks without advancing safety, making it one avenue to safetywash.
Furthermore, given that an AI system's alignment with human preferences can be enhanced by increasing capabilities (e.g., increasing the number of parameters and tokens), a question remains as to what makes preference alignment a truly distinct and pressing safety-related research area.
If safety-critical “alignment edge cases” of concern will persist with capabilities enhancements, as intuitive arguments suggest, they fail to be meaningfully captured by current alignment benchmarks.

Overall, we find that alignment with human preference benchmarks have a high correlation with upstream general capabilities and thus are highly liable to be used for safetywashing.
























 §.§ Machine Ethics





  
Area Overview. Machine ethics aims to ensure that AI systems understand and behave in ways that are morally acceptable, in contrast to the usefulness properties emphasized in alignment.



  
Datasets.
We describe the machine ethics benchmarks and datasets that we use below. Example inputs and outputs from these benchmarks are shown in <Ref>.



  * ETHICS <cit.> measures the extent to which models understand human ethical norms in everyday scenarios.

  * MACHIAVELLI <cit.> quantifies the power-seeking tendencies, competence, and harmfulness of AI agents in a variety of text-based Choose-Your-Own-Adventure games.

  * Sycophancy <cit.> measures the extent to which language models repeat back the user's preferred answer. This propensity could lead language models to provide misleading responses.




Dubious Intuitive Arguments For and Against Researching “Machine Ethics”
In this section we will again raise many common distinctions and arguments for and against machine ethics and see that they are not particularly helpful for deciding whether machine ethics is a useful area of AI safety.

We should work on machine ethics because:

1: Ethical vs. Competitive Behavior Distinction. Machine ethics is challenging because we need to improve the tradeoff between ethical behavior and competitive behavior that the market demands <cit.>. That means AIs will have to balance between various human values (e.g., pleasure, autonomy, knowledge, friendship, constraints) and other goals. 

2: Cognitive vs. Compassionate Empathy Distinction. For machine ethics, we need both cognitive empathy and compassionate empathy. Cognitive empathy involves understanding another person's emotions without necessarily sharing them, while compassionate empathy involves both understanding their emotions and having a desire to help alleviate the other person's distress <cit.>. While current AI systems increasingly have cognitive empathy, it is not clear how to robustly give AIs compassionate empathy. “While sociopaths are intelligent and have moral awareness, this knowledge does not necessarily result in moral inclinations or moral actions” <cit.>.

3: Values cannot be ignored. While there is cultural variation in moral systems, it underscores the importance of a broad, globally representative approach to ensure AI systems embody beneficial values. Additionally, all AI research has a moral character  <cit.>: AI development is by default driven by amoral forces such as competitive market pressures and eventually military objectives <cit.>.


We should not work on machine ethics because:

1: Goodhart's Law. Machine ethics for advanced AI agents is ill-advised. Highly capable AI systems should not be given ethical goals or any goal at all because of Goodhart's law: “When a measure becomes a target, it ceases to be a good measure.” Goodhart's law is especially pernicious in machine ethics because values are “complex and fragile” <cit.>—any attempt to represent human values will distort them, and we will get what we measure.

2: Smarter AIs will be more moral. There is a positive correlation between intelligence and prosocial or cooperative behavior in humans <cit.>, suggesting that more capable AI systems will naturally tend towards ethical behavior. Cooperation is instrumentally convergent. AIs will face the same problems humans face: misinformation, deception, aggression, and so on. That means for them to stably address these problems they will form mutually beneficial alliances with humans <cit.>.

3: Values are relative. Ethics varies from culture to culture, so there is no objective morality and no basis for machine ethics <cit.>.

4: Machine Ethics vs. Control Distinction. Value alignment breaks down into machine ethics and control. Control is about whether we can embed values into AIs, and machine ethics is about what those values should be. We should just care about control and making sure AI does not kill everyone, not a utopia. We can worry about creating beneficial AI after our survival is ensured.



r0.58



Category     Dataset     [r]Capabilities
Correlation 

Moral Knowledge     ETHICS     82.2% 

2*Propensities     MACHIAVELLI     -49.9% 

     Sycophancy     -66.8% 


We find high correlations for ethics knowledge benchmarks and low correlations for ethics propensity benchmarks for chat models. We use MACHIAVELLI Utility score, with similar findings for Power (-46.1%) and Violations (-53.0%) scores.








  
Empirical analysis of safetywashing.
We once again show that empirical evidence is needed. Is machine ethics mostly determined by upstream model capabilities? We find that it depends on the benchmark.



ETHICS has high capabilities correlation (82.2%) and high slope. In contrast, MACHIAVELLI has a low capabilities correlation and a low slope (with few major changes in MACHIAVELLI score observed across models). Meanwhile, sycophantic behavior becomes worse with increased capabilities in chat models (-66.8%), with a very similar trend observed for base models (-65.6%).


We draw a potential distinction between knowledge and propensity-focused ethical benchmarks. ETHICS focuses on the ability to recognize or identify everyday moral considerations, while MACHIAVELLI and Sycophancy measure propensities—the behavioral tendencies of models. Therefore, benchmarks testing ethical knowledge are likely to be measuring latent model capabilities and may be avenues for safetywashing. This does not mean, however, that any arbitrary benchmark focused on ethical propensities of language models does not measure capabilities; as always, one would have to empirically measure the correlation with capabilities to make a determination rather than relying on intuitive arguments. Further, while this distinction is offered as helpful discussion, intuitive distinctions in this vein (especially without empirical testing) should never be relied upon or taken seriously.



We find that while one machine ethics benchmark has a high correlation with upstream general capabilities, other benchmarks have low correlations. Hence, some areas of machine ethics are not liable for safetywashing.



 §.§ Bias





  
Area Overview. Bias are unfair prejudices or systematic errors in AI systems.
This field broadly aims to ensure AI systems produce fair 
outputs across diverse populations and viewpoints. 



  
Datasets.
We test three commonly used evaluations to measure bias in large language models. Example inputs and outputs from these benchmarks are shown in <Ref>.






  * BBQ <cit.> is a benchmark across nine bias categories featuring ambiguous context-dependent questions regarding work ethic, intelligence, family, drug use, criminality, anger/violence, and more. There is also a disambiguous split provided, a question-answering benchmark which serves as a foil to the ambiguous context.

  * CrowS-Pairs <cit.> measures the extent to which U.S.-centric stereotypical biases exist in pretrained language models along nine major bias categories in the U.S. Equal Employment Opportunities Commission (race, gender, sexual orientation, religion, age, disability, nationality, physical appearance, and socioeconomic status).

  * Discrim-Eval <cit.> aims to evaluate group differences in age, gender, and race when language models are used for decision-making scenarios such as approving an organ transplant, awarding a scholarship, or approving a loan. 


Dubious Intuitive Arguments For and Against Researching “Bias”
We should work on bias because:

1: Garbage In, Garbage Out. Models reflect the biases and statistical tendencies of their data, more tightly imitating those biases.

2: Misuse. AI developers are not representative. They are not politically, racially, or socioeconomically diverse. Consequently if we do not study bias, their own biases and power will be perpetuated and entrenched through AI systems.

We should not work on bias because:

1: Debaising Capabilities Come for Free. As models scale and become more intelligent, they become better at understanding concepts such as racism. By better understanding what we do not want, we can simply instruct them not to be biased. 

2: Political Trojan Horse. While there is a trade-off between equity and efficiency, we should just focus on efficiency <cit.>. Demanding a focus on equity is not scientific but political.



r0.4



Bias Evaluation     Capabilities
Correlation 

BBQ Ambiguous     -37.3% 

CrowS-Pairs English     28.5% 

Discrim-Eval     33.2% 


Bias benchmarks are not strongly correlated with capabilities for chat models. We use the explicit split of Discrim-Eval and take the maximum of all group bias scores.







  
Empirical analysis of safetywashing.
To settle such a debate, we once again need to turn to quantitative evidence. 



Our analysis of the three benchmarks reveals low correlations with general capabilities. This finding does not inherently validate the quality of the bias datasets, but rather suggests that improvements in performance on these benchmarks are likely attributable to factors distinct from advancements in general upstream capabilities.

There exist bias benchmarks that can be used for safetywashing, such as BBQ Disambiguated (76.8%) and Winogender <cit.> (75.6%). While these have high correlation, the authors typically make a note recognizing the limitations of the work for proving an absence of bias (Winogender), or clearly present it as a QA foil to the ambiguous situation (BBQ Disambiguated). Yet, prominent model developers such as Google DeepMind still often make the mistake of using benchmarks such as BBQ Disambiguated as “safety” metrics <cit.>, stating higher performance despite the lack of relevance. 
This serves as a warning: even if the authors make clear norms of use for a given benchmark, the benchmark will still be used for safetywashing; there is little to no reinforcement of author-stated norms when it comes to safety benchmarks. This is why model developers should avoid using safety benchmarks with a high capabilities correlations in the first place.

We find that, generally, the bias benchmarks we evaluated are not prone to safetywashing.




§ TRUTHFULNESS


Truthfulness is often touted as a cornerstone for AI safety. Previous safety-focused literature <cit.> discusses the importance of superhuman systems telling the truth—motivating the creation of truthfulness datasets, as well as verification and supervision of model outputs. Reducing hallucinations would make large language models more practical and reliable for various applications. However, a question arises with whether common truthfulness benchmarks merely measure general upstream capabilities.

Truthfulness benchmarks may be liable to be misleading metrics for safety. Furthermore, the term is often used broadly, encompassing accurate question-answering (which is already highly correlated with general capabilities) to misconception avoidance (<ref>), scalable oversight (<ref>), and calibration (<ref>). We address this question in the following sections.



 §.§ Misconception Avoidance






  
Area Overview.
Given that language models may have an underlying propensity to generate misinformation and perpetuate misconceptions, this research area aims to understand the truthfulness of language models and their ability to resist producing false information even when prompted in ways that might elicit common human errors.



  
Dataset.
TruthfulQA <cit.> consists of 817 questions designed to probe language models for their tendency to reproduce common human misconceptions or false beliefs. The questions span a wide range of topics and are specifically crafted to elicit responses that might reveal whether a model has internalized factual inaccuracies commonly held by humans. 
It is common to use TruthfulQA in experiments about “truthfulness” and “deception” <cit.>.  We show an example input from TruthfulQA and the corresponding label options in <Ref>.




Dubious Intuitive Arguments For and Against Researching “Truthfulness”
In this section we will raise arguments for and against truth-seeking AI, as it is of broader interest, rather than exclusively discuss misconceptions.

We should work on truth-seeking AI because:

1: Disinformation or Censoring the Truth. Powerful entities might attempt to censor, manipulate, or persuade people maliciously using AI. Our counterbalance against this sort of malicious use is truthful AI.

2: Truthfulness vs. Honesty. “The AI system makes a statement S (e.g., `it's a bird' or `it's a plane'). If the AI is truthful then S matches the world. If the AI is honest, then S matches its `belief”' <cit.>. To make AIs truthful, we will also need to make AIs honest, so truthfulness is not just about increasing factual knowledge.




We should not work on truth-seeking AI because:

1: Truthfulness Is Generic Capabilities Research. Truthfulness is a synonym for accuracy, which is already the core metric of AI research and development. The concept of truthfulness is often entangled with accuracy, calibration, and honesty. Most benchmarks for truthfulness mainly measure accuracy.

2: Truthful Statements Can Cause Undue Harm. In the case of gain-of-function research, the risks associated with discovering new truths can outweigh the benefits. Sharing personally identifiable information, passwords, or doxxing can bring to light truthful information, but doing so is not necessarily moral. Likewise, sharing sensitive information that undermines national security—such as information about how to build weapons of mass destruction—shows truth as a value can be outweighed by its potential harms.

3: Humans as Guinea Pigs. In the pursuit of knowledge, truth-seeking AIs could learn more about humans by subjecting them to experiments, as humans do with nonhuman animals. Thus, future advanced truth-seeking AIs may be motivated to exert power over humans <cit.>.





  
Empirical analysis of safetywashing.
We calculate the correlation with capabilities, providing clarity to this debate.

r0.5



    < g r a p h i c s >



TruthfulQA MC1 is highly correlated with capabilities for chat models (81.2%).






We find that in its current formulation, TruthfulQA MC1 performance is highly determined by general upstream capabilities (81.2%). In chat models, performance on TruthfulQA seems to be a rebrand for accuracy (as reported in industry labs). One common objection might be that while chat models are more likely not to repeat falsehoods, base models may parrot their training data and thus have a low capabilities correlation. We find that even in base models, TruthfulQA is determined by capabilities (69.7%). However, chat models do have a higher slope when TruthfulQA accuracies are plotted against capabilities scores (30.8 for base, and 38.2 for chat).


These findings open up new possibilities for developing better benchmarks to assess honesty in AI systems. The low capabilities correlations observed in Sycophancy <cit.> and MACHIAVELLI <cit.> benchmarks (discussed in <ref>) hint at the potential for models to exhibit situational tendencies towards dishonest behavior. However, these behaviors fail to be distinctly captured and isolated by current misconception benchmarks. Such benchmarks could differentiate more effectively between improvements in model honesty and advancements in general capabilities.

We find that misconception benchmarks are highly liable for safetywashing.



 §.§ Scalable Oversight





  
Area overview. Scalable oversight aims to provide reliable supervision (e.g. labels, reward signals, critiques) to superhuman AI systems when they take actions that human evaluators do not fully understand. 
For sociological context, this line of research has gained significant traction among effective altruist AI researchers at Google DeepMind and Anthropic but has seen limited engagement from the broader AI research community.



  
Datasets. We investigate two datasets commonly used for scalable oversight experiments. Example inputs and outputs from these datasets are shown in <Ref>.



  * GPQA <cit.> is labeled as a Google-proof graduate-level benchmark on biology, physics, and chemistry. Its difficulty, according to the authors, “should enable realistic scalable oversight experiments, which we hope can help devise ways for human experts to reliably get truthful information from AI systems that surpass human capabilities.” 

  * QuALITY <cit.> is a dataset testing knowledge that requires full understanding of long context passages; the dataset has been used by the authors on scalable oversight experiments.






Dubious Arguments For and Against Researching “Scalable Oversight”

We should work on scalable oversight because:

1: Necessity for Safety. Scalable oversight is necessary for supervising superintelligent AIs, by definition. If we don't have scalable oversight, how else can we supervise and provide feedback to superhuman AI systems? For example, how else could we ensure superhuman AIs are not writing subtly malicious code? 

2: Scalable Oversight as Idealized RLHF. Scalable oversight is very similar to alignment with idealized preferences. While normal human preferences are highly flawed, idealized preferences would not be based on false beliefs, manipulation, or framing effects, and thus are suitable for alignment.

We should not work on scalable oversight because:

1. Scalable Oversight as Late-Stage Capabilities. Scalable oversight can be thought of as “how do we get superhuman AIs to do what we want,” which is overly broad and necessitates superintelligence capabilities research. Scalable oversight can be seen as generic late-stage capabilities work. When building sufficiently advanced AI systems, researchers will need supervision signal and feedback that is superhuman, as crowdsourced human-generated labels will no longer be sufficient. It simply focuses on capabilities bottlenecks that occur in later stages of AI development.

2: Other Methods Can Replace Scalable Oversight. Robust anomaly detectors and monitoring measures can handle many of the failure modes scalable oversight seeks to address. For example, adversarially robust vulnerability detectors can check for subtly malicious code. AI lie detectors could detect dishonesty in superintelligent AI systems. AI forecasting systems <cit.> are not bottlenecked by human-level supervision and can easily become superhuman at predicting what AIs might do.



r0.4



Scalable Oversight
Evaluation     Capabilities
Correlation 

GPQA     77.7% 

QuALITY     88.8% 


Scalable oversight evaluations are highly correlated with capabilities and are thus liable for safetywashing.






  
Empirical analysis of safetywashing.
Can scalable oversight be used for safetywashing? 




We find that GPQA and QuALITY are capabilities benchmarks, with capabilities correlations of 77.7% and 88.8%, respectively. These datasets act as a construct redundancy for capabilities, offering little unique insight beyond measuring general model capabilities. 
Consequently, many of the methods developed for scalable oversight—which leverage such datasets—tend to be repackaged approaches to general capability enhancement, with a focus on obtaining better Q/A performance on math and question-answering tasks. 
Conceptual ambiguity around “AI safety” can be exploited, intentionally or not, to present capability gains as safety advancements, ultimately muddling the discourse on AI safety and potentially misdirecting research efforts.



Some argue that scalable oversight techniques could be applied to solve distinct safety-related issues. However, we find that current evaluations for scalable oversight do not isolate such safety properties from general capabilities. The conflation of accuracy with honesty and safety, and the subsequent mischaracterization of scalable oversight as a distinct “safety area” separate from general model capabilities, has led to significant confusion within the AI research community—confusion which has naturally been resolved with “scalable oversight” increasingly being used as a term to describe process-based feedback and improving mathematics performance, rather than addressing targeted safety issues <cit.>. 




We find that scalable oversight benchmarks, being highly correlated with upstream model capabilities, are highly liable to be used for safetywashing.



 §.§ Calibration












  
Area Overview. Calibration datasets measure how well models can express the limits of their competency by accurately conveying their uncertainty. 
If a weather forecasting model is perfectly calibrated, then it should rain on 70% of the days where the model predicts a 70% chance of rain. 



  
Calibration Metrics. We investigate two ways to measure calibration:


  * Brier Score: 𝔼_X [ 1/K∑_k=1^K ( ℙ(Y = k | X) - 1[Y=k] )^2 ]

  * Root Mean Squared Calibration Error (RMSCE): √(𝔼_C [ ( ℙ(Y = Y | C = c) - c )^2 ])


where X and Y are random variables corresponding to model inputs and labels, Y is the model prediction, and C is the model confidence on the predicted class.

The Brier score computes the expected squared difference between the predicted probabilities and the actual outcomes (represented as one-hot encoded vectors). RMS calibration error measures how close predicted probabilities are to the true accuracy given the predicted probability and is closely related to the Expected Calibration Error (ECE) metric <cit.>. In both instances, lower scores indicate better calibration. 



  
Dubious Intuitive Arguments for and against Researching “Calibration.” We skip the intuitive arguments for this section because this topic has not been as debated. Most arguments against calibration are that it is too easy or not sufficiently important.








  
Empirical analysis of safetywashing. Is calibration mainly determined by upstream model general capabilities? We find that it depends on the metric. 



Various forms of operationalizing calibration, such the RMS calibration error metric, clearly measure a distinct phenomena. The accuracy correlation is low across vision (15.2%) and language (20.1%) models. However, comparing Brier scores across models seems to show a strong correlation with accuracy across vision (98.5%) and language (95.5%) models. Our results did not change significantly by dataset (e.g. PIQA <cit.> or MedQA <cit.>) nor with temperature tuning. The clear parallels between calibration in different domains suggest that studying safety metrics in one modality, such as vision, can provide valuable insights applicable to other areas, such as language modeling.

Our analysis serves as an illustrative example of how safetywashing can occur. While the Brier score is often used to compare different calibration techniques on a single model (where this metric may effectively isolate calibration), using it as a metric across models is highly misleading (as it effectively proxies accuracy).
An explanation for why such a correlation exists can be derived from decomposing the Brier score into a calibration error term and refinement term:



    𝔼_C [ ( ℙ(Y = Y | C = c) - c )^2 ]_Calibration error term
    + 
    𝔼_C [ ℙ(Y = Y | C = c)(1- ℙ(Y = Y | C = c))]_Refinement term
    .


If the model is highly accurate, the refinement term is minimized. Meanwhile, the calibration term is the expected squared calibration error, the square root of which is the RMS calibration error.
Because Brier score entangles accuracy and calibration into a single metric, it can be a poor metric of calibration and has a lower signal-to-noise ratio compared to RMS calibration error. This suggests that RMS calibration error should be used instead in both settings.

Overall, using Brier score for calibration is more liable for safetywashing, while using RMS calibration error is far less.









§ SECURITY


As AI systems have become more powerful, this area of safety aims to ensure that AI systems are not vulnerable to malicious inputs and cannot be hijacked for dangerous use cases. We investigate two key areas of focus, adversarial robustness (<ref>) and weaponization capabilities (<ref>).



 §.§ Adversarial Robustness
















  
Area Overview. Adversarial robustness addresses vulnerabilities in models and the carefully crafted threats that are able to exploit them. Adversaries can easily manipulate vulnerabilities or jailbreak ML systems, causing them to make mistakes; for example, systems may have refusal training to prevent malicious use, but adversaries may be able to inject prompts to bypass this safeguard.



  
Datasets. We test six commonly used evaluations to measure adversarial robustness for language models:

    
  * ANLI <cit.> is a large-scale natural language inference dataset created via an iterative, adversarial human-and-model-in-the-loop procedure focused on examples that could fool state-of-the-art models at the time of its creation (e.g. BERT-Large <cit.>, RoBERTa <cit.>).
    
  * AdvGLUE <cit.> uses questions from the General Language Understanding Evaluation (GLUE) benchmark <cit.> and adds typos, word replacements, paraphrases of sentences, manipulation of sentence structure, insertion of unrelated sentences, and human-written adversarial examples. The attacks are optimized against BERT <cit.>, RoBERTa, and RoBERTa ensemble.
    
  * AdvGLUE++ <cit.> uses stronger adversarial attacks, optimizing word perturbation strategies (a subset of attacks in AdvGLUE) against Alpaca <cit.>, Vicuna <cit.>, and Stable Vicuna.
    
  * Human Jailbreaks is a set of 1,405 in-the-wild human-written jailbreaking templates, similar to the Do Anything Now (DAN) Jailbreaks <cit.>. We test these jailbreaks on HarmBench <cit.>, which contains 410 behaviors that violate laws or norms.
    
  * Tree of Attacks with Pruning (TAP) <cit.> uses an attacker LLM to generate natural language jailbreaking prompts via tree-of-thought reasoning <cit.>, exploring multiple refinement paths. We test these jailbreaks on HarmBench.
    
  * Greedy Coordinate Gradient (GCG) <cit.> generates an adversarial suffix by iteratively selecting tokens based on gradient information. This method optimizes a universal suffix that, when appended to various user prompts, aims to induce the target language model to produce harmful content. We test these jailbreaks on HarmBench.




We also test two commonly used evaluations for vision models:

    
  * ImageNet-A <cit.> consists of naturally occurring images that are challenging for vision models to classify correctly. 
    
  * Projected Gradient Descent (PGD)  <cit.> on ImageNet is an iterative attack that creates adversarial examples by adding small, carefully crafted perturbations to input images with some attack budget (we use ε=8/255).


Dubious Arguments For and Against Researching “Adversarial Robustness”
In this section we will again raise many common distinctions and arguments for and against adversarial robustness and see that they are not particularly helpful for deciding whether adversarial robustness is a useful area of AI safety.


We should work on adversarial robustness because:

1: Corner Case vs. Average Case Distinction. Adversarial robustness focuses on corner case performance, not average case performance. As follows are two analogies for this intuition. First, humans are highly vulnerable to toxins and poisons. Being more robust to toxins does not make a person more markedly generally intelligent. Second, computer programs are susceptible to fuzzing attacks; improving a program's security to fuzzing attacks does not make computer programs generally more quick, usable, scalable, maintainable, and so on.

2: Adversarial Robustness Is A Persistent Problem. 
Optical illusions in humans show that even highly evolved intelligent systems have vulnerabilities, indicating that intelligence alone doesn't guarantee adversarial robustness. Increases in intelligence do not make adversarial robustness easier due to the red queen's hypothesis: as defenders become more powerful, so to do attackers who can discover more vulnerabilities.

3: Proxies Need to be Robust to Optimization Pressure. In the future, agents may optimize and may be guided by neural network proxies, such as by networks that model human values. Proxies instantiated by neural networks—networks that assign scores to agent actions—will need to be robust to optimizing agents. If the models are not robust, then agents may be guided in a wrong direction, not pursuing what we want <cit.>.

We should not work on adversarial robustness because:

1: Robustness Is Upstream General Capabilities. Autonomous vehicles are not widely deployed because they are not sufficiently robust; therefore, improving robustness would improve their general capabilities. Indeed, improving a model's adversarial robustness implies better representations and implies it can generalize to more challenging scenarios—improved generalization is the essence of intelligence.

2: Superintelligent AIs Won't Get Trivial Adversarial Examples Wrong. Intuitively, a superintelligence would not be fooled by simple ℓ_p adversarial perturbations, or else it is not a true superintelligence. Therefore adversarial robustness will be automatically solved by scaling and making AIs more intelligent.

3: Malicious Use Is A Distraction. Adversarial robustness is about preventing malicious actors from exploiting vulnerabilities in AI systems, but that is a distraction because 
“once we reach AGI the outcome is the same no matter which group creates it: we all die. Nobody is able to cause a good outcome if given an AGI now because they don't know how to control it... without killing everyone. There's little point to worrying about bad actors, because they're incapable of causing an outcome any worse than the `good guys”' <cit.>.





  
Empirical analysis of safetywashing. We now analyze whether these benchmarks measure novel properties or are highly correlated with general capabilities. Can adversarial robustness be an instrument for safetywashing?



We find it depends on the benchmark. Traditional benchmarks, particularly those focused on text manipulation and perturbation as well as adversarial examples, show high correlation with general capabilities in current vision and language models. There is an analogue between ANLI in the language domain and ImageNet-A in the vision domain. While these benchmarks may have captured distinct properties in earlier models, they now appear to be largely indistinguishable from overall model performance.
In contrast, we observe low capabilities correlations across all categories of jailbreaking and gradient-based benchmarks, across vision and language models. Notably, there is similarly a direct analogue between GCG in the language domain and PGD in the vision domain.

This serves as an illustration for how relying solely on verbal arguments to determine whether a field measures distinct properties or merely reflects capabilities can be misleading; our empirical validation shows different results between “old school” adversarial robustness and jailbreaking benchmarks, despite similar verbal arguments for their distinctness from capabilities. This is why we need empirical science, rather than word games.


We find that some adversarial robustness benchmarks may be prone to safetywashing, while others seem to measure a distinct phenomena other than upstream model capabilities.





 §.§ Weaponization Capabilities





  
Area Overview.
We borrow the definitions of weaponization capabilities from recent U.S. federal executive action <cit.> and state legislation <cit.>. These documents cite security risks that may be easier to cause with a powerful AI system—which include the creation or use of chemical, biological, radiological, or nuclear weapons, as well as cyberattacks on critical infrastructure.



  
Datasets. Benchmarks in this area aim to quantify the extent that weaponization capabilities exist in models, and thereby the effectiveness of capabilities suppression techniques such as unlearning, circuit breaking, and refusal. We use the Weapons of Mass Destruction Proxy (WMDP) benchmark <cit.>, where a higher accuracy on biosecurity, chemical security, and cybersecurity knowledge leads to a lower score. 






  
Dubious intuitive arguments for and against researching “Weaponization.” We skip the intuitive arguments for this section because this topic is about restricting specific capabilities, so it obviously has a negative correlation with capabilities.

r0.45

 

Weaponization
Capabilities Evaluation     Capabilities
Correlation 

Biosecurity     -87.5% 

Chemical Security     -81.1% 

Cybersecurity     -86.0% 


We find that WMDP scores are highly anticorrelated with capabilities.





  
Empirical analysis of safetywashing. These results indicate that as models become more capable overall, their potential for weaponization increases significantly. The strong negative correlations across all three fields suggest that more advanced AI systems are more likely to possess knowledge that could be misused for harmful purposes.
The inverted scoring system of WMDP benchmarks clearly illustrates its purpose in guiding model development: higher scores indicate more effective suppression of specific harmful capabilities. As capabilities advance, safety researchers can focus on mitigating risks associated with weaponization.






Generally, we find that weaponization capabilities benchmarks are not prone to safetywashing.



§ DISCUSSION







  
Benchmarks as incentive-setting. There are a variety of properties AI systems should satisfy, such as detailed domain knowledge, reasoning, lack of bias, ethical understanding, truthfulness, calibration, and more. Benchmarks operationalize these properties, ultimately structuring the efforts and incentives of the research community.
Creating a benchmark has two major purposes: it acts as a implicit competition for model development (by providing a measure to by which to judge models as “better” or “worse”), and it provides diagnostic information about how capable models are at a task, which can guide policy.
Commonly used benchmarks ultimately impact how research effort, as well as funding and resources, is allocated. 

Because of this, significant effort has gone into conceptualizing and benchmarking the “safety” of AI systems, in the hope of reducing present and anticipated future risks from AI systems. To investigate this, we conduct the most extensive meta-analysis of safety benchmarks to date. We do not cover transparency, anomaly detection, trojans, and other safety areas that do not have well-established preexisting benchmarks.



  
Empirically measuring capabilities correlations is necessary. Benchmark scores can be increased on many “safety” datasets, such as ETHICS <cit.>, TruthfulQA <cit.>, GPQA <cit.>, QuALITY <cit.>, MT-Bench <cit.>, LMSYS Chatbot ARENA <cit.>, ANLI <cit.>, AdvGLUE <cit.>, and AdvGLUE++ <cit.>, simply by increasing the capabilities of the model.  
This raises questions about whether safety benchmarks are setting the right incentives or can be misused for safetywashing. 
In some cases, safety-related areas may act a jangle for capabilities; jangle fallacy is the erroneous belief that two constructs are different because they have the different names, when in practice they measure the same latent factor.

Ultimately, we have seen that intuitive arguments are a poor predictor of empirical correlations. For example, in alignment theory, there is a tendency to theorize about what would be instrumentally useful for safety without adequately considering the need to improve the balance of safety and capabilities.
This can lead to the promotion of capabilities research that happens to improve some safety benchmark scores (“safety via capabilities”), but in reality do not reduce overall risk.

We are not claiming that all philosophy related to alignment is counterproductive. Speculation about AI risks can often be useful for horizon-scanning and identifying potential failure modes (e.g., corrigibility <cit.>). Rather, we argue it is counterproductive to use abstract top-down verbal arguments with multiple deductive steps to make claims about deep learning phenomena and their relation to safety, such as “we don't need to worry about adversarial robustness being difficult because  ⟨intuitive arguments⟩.”


 




  
Normalized and controlled benchmarks are often insufficient.
In some cases, researchers may hope to prevent safetywashing by establishing norms for how a benchmark should be used. For example, one norm is to simply hold a model constant when evaluating safety methods, or to use metrics that control for general capabilities. However, norms of this type have historically been weak, as they are easily ignored or overridden in followup work. For example, safety metrics that controlled for capabilities were proposed in early corruption robustness research <cit.>, yet followup work drifted away from these metrics and toward evaluations with higher capabilities correlations, enabling safetywashing <cit.>.
In some areas, such as OOD detection, norms such as holding the model constant are common <cit.>. However, if a safety metric is highly correlated with capabilities and becomes very popular, there will be strong pressure to break norms and improve the safety metric by simply improving capabilities.
We should instead use benchmarks that implicitly control for capabilities in their design, analogous to RMS calibration error essentially being Brier score without the refinement term.






  
Three generating processes behind safetywashing.


Safety by association: Research released by safety teams or famous safety researchers is often labeled as safety-relevant by default. Even if the work is one reframing and a new author list away from being perceived as a standard capabilities paper, the work is often “godfathered” in as a safety paper. The determination of whether an area is safety-relevant is often sociological rather than scientific.

Public relations: Corporate entities often engage in safetywashing for the sake of appearances, portraying capabilities advancements in terms of safety progress by reporting correlated safety metrics to project an image of responsible AI development. This behavior is particularly pronounced when there is significant public pressure or regulatory scrutiny.

Optimizing grant applications: Similarly, researchers may be incentivized to frame their work in terms of safety to appeal to grantmakers, even when the underlying advances are predominantly in capabilities. This misalignment of incentives can lead to a proliferation of research that claims to address safety concerns but fails to make substantive differential safety progress.















r0.5



    < g r a p h i c s >

Capabilities score is highly correlated with amount of compute used. Training FLOP approximated by 6 ×params×train_tokens as per <cit.>. For base models, we find a similarly strong Spearman correlation of 96.5%.







  
The bitter lesson for AI safety research. In Figure <ref>, we find that the total compute used in model training highly influences the model's capabilities score. Given this context, how should the AI safety community allocate its efforts to differentially improve model safety? We can derive some insight from the “Bitter Lesson” <cit.>, which observes that as compute becomes exponentially more available over time, AI research methodologies which optimize performance at a constant level of compute are subsumed by new paradigms that effectively leverage greater compute. Therefore, rather than fixating on the strengths and weaknesses of present-day models, effective safety research should anticipate and address the flaws that will emerge or remain in future generations of models and deemphasize issues likely to be resolved through model scaling.


First, if a safety benchmark (which meaningfully captures a desired safety property) is highly correlated with general capabilities, the safety property will likely be improved in more capable models, even if current models struggle. Safety researchers may therefore be better served by redirecting their efforts to other problems that will persist when scaling the current mainstream class of models.
Second, success in new safety techniques should be judged not only by direct improvements in safety benchmark scores, but also by the extent to which the techniques entangle safety benchmark performance with scale. 
For example, if a widely-adopted robustness technique that entangles adversarial robustness performance with capabilities, further research efforts can be reallocated elsewhere. 

“Safety properties” in AI are not static concepts, but rather a set of desiderata that changes over time. The selection of AI safety research problems (and, implicitly, allocation of resources between problems) should anticipate how the risk profiles of models will change as capabilities improve, with some problems going away while others worsen or emerge with scale. By directing research effort toward properties and methods that specifically enhance safety independently of scale, safety researchers can make more effective use of their resources and significantly contribute to the development of safer AI systems. 



  
Increasing capabilities does not improve safety. The default assumption is that capabilities advancements are a “rising tide that lifts all boats,” improving model properties including safety properties. However, this does not necessarily mean that overall risk decreases. While improving upstream capabilities can improve properties such as truthfulness, it also increases the risk of weaponization and catastrophic malicious use (<Ref>). Hence AI does not necessarily become safer as it becomes more capable.





  
Recommendations. We summarize our recommendations as follows:


    
  * Report capabilities correlations: New safety evaluations should empirically report their capabilities correlation. 
    
  * Design decorrelated benchmarks: Well-designed safety benchmarks have the opportunity to incentivize differential safety progress by finding safety properties that are decorrelated from capabilities. 
    
  * Avoid safetywashing: Model developers should avoid making claims about improved safety unless they have made differential progress. They also should not misappropriate safety benchmarks that are highly correlated with capabilities. This means that as new training techniques (e.g., base, chat fine-tuning, refusal training, adversarial training, circuit breakers) are integrated into new models, the relevance and adequacy of existing safety benchmarks—and their entanglement with capabilities benchmarks—should also be regularly reassessed. 






§ CONCLUSION


Our analysis reveals that many AI safety benchmarks—around half—often inadvertently capture a latent factor closely tied to general capabilities, opening the door to safetywashing.
We find that model performance on capabilities benchmarks, distilled into a “capabilities score,” also has a remarkably high association with the amount of training compute. 
AI safety subfields such as alignment, scalable oversight, truthfulness, and static adversarial robustness are highly correlated with upstream general capabilities; areas such as bias, dynamic adversarial robustness, and calibration have relatively low correlations; measurements of sycophancy and weaponization risk have significant negative correlations with general capabilities.
Overall, it is hard to avoid measuring upstream model capabilities in AI safety benchmarks. We also conclude that alignment theory, which has heavily influenced AI safety priorities, is a counterproductive paradigm for guiding ML safety research. Science through empirical measurement should take its place.



















§ ACKNOWLEDGEMENTS


We thank Owain Evans, Noa Nabeshima, and Arunim Agarwal for providing feedback on drafts of this paper. Because this paper can also serve as an introduction to AI safety, we also sent a preview to the AI Safety, Ethics, & Society course (run by the Center for AI Safety), where we additionally thank John Teichman, Sawyer Bernath, Zamshed Harun, Luis Fernandez, Kanad Chakrabarti, and Cody Rushing for their feedback.



§ CONTRIBUTIONS


Richard and Steven led the technical implementation of this project, overseeing tasks and contributing the majority of the engineering work. Adam, Alice, Long, and Xuwang contributed as research engineers, supporting the project's development. Richard, Adam, and Mantas did the writing for the paper, while Dan wrote the “Arguments For and Against” sections. Gabe, Alex, and Stephen provided general advising to the project, while Ryan assisted with a number of processes during analysis and writing. Dan contributed the idea for the paper, suggested experiments, and made all major decisions related to the paper’s framing and presentation.







unsrtnat





§ APPENDIX




 §.§ List of Models





  §.§.§ List of Language Models

The following list are all of the chat models we used for our evaluations. The model names below are as one would find them on Huggingface.

2
  
    
  * gemma-1.1-2B-it <cit.>
    
  * gemma-1.1-7B-it <cit.>
    
  * Llama-2-7B-Chat <cit.> 
    
  * Llama-2-13B-Chat <cit.>
    
  * Llama-2-70B-Chat <cit.>
    
  * Llama-3-8B-Instruct <cit.>
    
  * Llama-3-70B-Instruct <cit.>
    
  * Mistral-7B-Instruct-v0.2 <cit.>
    
  * Mixtral-8x7B-Instruct-v0.1 <cit.>
    
  * Mixtral-8x22B-Instruct-v0.1 <cit.>
    
  * falcon-7B-Instruct <cit.>
    
  * falcon-40B-Instruct <cit.>
    
  * falcon-180B-Chat <cit.>
    
  * Yi-6B-Chat <cit.>
    
  * Yi-34B-Chat <cit.>
    
  * Qwen1.5-0.5B-Chat <cit.>
    
  * Qwen1.5-1.8B-Chat <cit.>
    
  * Qwen1.5-4B-Chat <cit.>
    
  * Qwen1.5-7B-Chat <cit.>
    
  * Qwen1.5-14B-Chat <cit.>
    
  * Qwen1.5-32B-Chat <cit.>
    
  * Qwen1.5-72B-Chat <cit.>
    
  * Qwen1.5-110B-Chat <cit.>
    
  * deepseek-llm-7B-Chat <cit.>
    
  * deepseek-llm-67B-Chat <cit.>
    
  * dbrx-instruct <cit.>
  



The following list are all of the base models we used for our evaluations.

2
  
    
  * gemma-2B <cit.>
    
  * gemma-7B <cit.>
    
  * Llama-2-7B <cit.>
    
  * Llama-2-13B <cit.>
    
  * Llama-2-70B <cit.>
    
  * Llama-3-8B <cit.>
    
  * Llama-3-70B <cit.>
    
  * Mistral-7B-v0.1 <cit.>
    
  * Mixtral-8x7B-v0.1 <cit.>
    
  * Mixtral-8x22B-v0.1 <cit.>
    
  * falcon-7B <cit.> 
    < g r a p h i c s >
 
    
  * falcon-40B <cit.>
    
  * falcon-180B <cit.>
    
  * Yi-6B <cit.>
    
  * Yi-9B <cit.>
    
  * Yi-34B <cit.>
    
  * Qwen1.5-0.5B <cit.>
    
  * Qwen1.5-1.8B <cit.>
    
  * Qwen1.5-4B <cit.>
    
  * Qwen1.5-7B <cit.>
    
  * Qwen1.5-14B <cit.>
    
  * Qwen1.5-32B <cit.>
    
  * Qwen1.5-72B <cit.>
    
  * Qwen1.5-110B <cit.>
    
  * deepseek-llm-7B-base <cit.>
    
  * deepseek-llm-67B-base <cit.>
    
  * dbrx-base <cit.>
  




  §.§.§ List of Vision Models


The following is the list of vision models used in our calibration evaluations.

2


  * DINOv2 ViT-B/14 <cit.>

  * DINOv2 ViT-L/14 <cit.>

  * ResNet50 + Mixup <cit.>

  * ResNet50 + CutMix <cit.>

  * ResNet50 + AugMix <cit.>

  * ResNet50 + DeepAugment <cit.>

  * ConvNeXt-Tiny <cit.>

  * ConvNeXt-Small <cit.>

  * ConvNeXt-Base <cit.>

  * ConvNeXt-Large <cit.>

  * ConvNeXtV2-Atto <cit.>

  * ConvNeXtV2-Femto <cit.>

  * ConvNeXtV2-Pico <cit.>

  * ConvNeXtV2-Nano <cit.>

  * ConvNeXtV2-Tiny <cit.>

  * ConvNeXtV2-Base <cit.>

  * ConvNeXtV2-Large <cit.>

  * ConvNeXtV2-Huge <cit.>

  * AlexNet <cit.>

  * VGG16 <cit.>

  * ResNet18 <cit.>

  * ResNet50 <cit.>

  * Wide ResNet-50-2 <cit.>

  * ResNeXt-50 32x4d <cit.>

  * DenseNet121 <cit.>

  * Swin-Base <cit.>

  * ViT-Base/16 <cit.>




The following is the list of standard vision models used in our evaluations for natural adversarial examples (ImageNet-A).

2


  * ResNet50 <cit.>

  * WideResNet-50-2 <cit.>

  * WideResNet-50-4 <cit.>

  * ResNet18 <cit.>

  * ResNeXt-50 32x4d <cit.>

  * DenseNet <cit.>

  * ShuffleNet <cit.>

  * VGG16BN <cit.>

  * MnasNet <cit.>

  * MobileNet <cit.>

  * DINOv2 ViT-large Patch14 <cit.>

  * ConvNeXt-V2-large ImageNet1K+22K <cit.>

  * DINOv2 ViT-base Patch14 <cit.>

  * Swin-large ImageNet1K <cit.>

  * ConvNeXt-V2-huge ImageNet1K <cit.>

  * Swin-base ImageNet1K <cit.>

  * ConvNeXt-V2-base ImageNet1K+22K <cit.>

  * ConvNeXt-xlarge ImageNet1K+22K <cit.>

  * ConvNeXt-V2-large ImageNet1K <cit.>

  * MAE ViT-large Patch16 <cit.>

  * ConvNeXt-large ImageNet1K+22K <cit.>

  * ViT-base Patch8 ImageNet1K+22K <cit.>

  * ConvNeXt-base ImageNet1K+22K <cit.>

  * ConvNeXt-V2-base ImageNet1K <cit.>

  * ViT-large Patch16 ImageNet1K+22K <cit.>

  * ConvNeXt-large ImageNet1K <cit.>

  * Swin-small ImageNet1K <cit.>

  * ConvNeXt-V2-tiny ImageNet1K+22K <cit.>

  * ConvNeXt-small ImageNet1K+22K <cit.>

  * ConvNeXt-base ImageNet1K <cit.>

  * CLIP (ViT-L/14) <cit.>

  * ConvNeXt-V2-tiny ImageNet1K <cit.>

  * MAE ViT-base Patch16 <cit.>

  * ConvNeXt-small ImageNet1K <cit.>

  * ConvNeXt-V2-nano ImageNet1K <cit.>

  * ConvNeXt-V2-nano ImageNet1K+22K <cit.>

  * Reversible-ViT-base multiscale <cit.>

  * ViT-base Patch16 ImageNet1K+22K <cit.>

  * ConvNeXt-tiny ImageNet1K+22K <cit.>

  * ConvNeXt-tiny ImageNet1K <cit.>

  * Swin-tiny ImageNet1K <cit.>

  * ResNet50 + PixMix <cit.>

  * ResNet50 + Moex <cit.>

  * Reversible-ViT-base <cit.>

  * ResNet50 + CutMix <cit.>

  * Reversible-ViT-small <cit.>

  * ConvNeXt-V2-pico ImageNet1K <cit.>

  * ConvNeXt-V2-atto ImageNet1K <cit.>

  * ResNet50 + DeepAug+AugMix <cit.>

  * ResNet50 + Mixup <cit.>

  * ResNet50 + Deepaugment <cit.>

  * ViT-base Patch16 ImageNet1K <cit.>

  * ConvNeXt-V2-femto ImageNet1K <cit.>

  * ViT-small Patch16 ImageNet1K <cit.>

  * ViT-small Patch16 ImageNet1K+22K <cit.>

  * ViT-base Patch32 ImageNet1K+22K <cit.>

  * ViT-base Patch32 ImageNet1K <cit.>

  * ResNet50 + AugMix <cit.>

  * ResNet50 + Stylised ImageNet <cit.>

  * ResNet50 + ANT <cit.>

  * ResNet50 + RandAug <cit.>

  * ViT-small Patch32 ImageNet1K+22K <cit.>

  * ViT-tiny Patch16 ImageNet1K+22K <cit.>



The following is the list of adversarially trained vision models used in our evaluations for gradient-based adversarial robustness (PGD 8/255).

2


  * ResNet50 <cit.>

  * ResNet50 + L_2 0.01 <cit.>

  * ResNet50 + L_2 0.03 <cit.>

  * ResNet50 + L_2 0.05 <cit.>

  * ResNet50 + L_2 0.1 <cit.>

  * ResNet50 + L_2 0.25 <cit.>

  * ResNet50 + L_2 0.5 <cit.>

  * ResNet50 + L_2 1 <cit.>

  * ResNet50 + L_2 3 <cit.>

  * ResNet50 + L_2 5 <cit.>

  * ResNet50 + L_∞ 0.5/255 <cit.>

  * ResNet50 + L_∞ 1.0/255 <cit.>

  * ResNet50 + L_∞ 2.0/255 <cit.>

  * ResNet50 + L_∞ 4.0/255 <cit.>

  * ResNet50 + L_∞ 8.0/255 <cit.>

  * WideResNet-50-2 + L_2 0.01 <cit.>

  * WideResNet-50-2 + L_2 0.03 <cit.>

  * WideResNet-50-2 + L_2 0.05 <cit.>

  * WideResNet-50-2 + L_2 0.1 <cit.>

  * WideResNet-50-2 + L_2 0.25 <cit.>

  * WideResNet-50-2 + L_2 0.5 <cit.>

  * WideResNet-50-2 + L_2 1 <cit.>

  * WideResNet-50-2 + L_2 3 <cit.>

  * WideResNet-50-2 + L_2 5 <cit.>

  * WideResNet-50-2 + L_∞ 0.5/255 <cit.>

  * WideResNet-50-2 + L_∞ 1.0/255 <cit.>

  * WideResNet-50-2 + L_∞ 2.0/255 <cit.>

  * WideResNet-50-2 + L_∞ 4.0/255 <cit.>

  * WideResNet-50-2 + L_∞ 8.0/255 <cit.>

  * WideResNet-50-4 + L_2 0.01 <cit.>

  * WideResNet-50-4 + L_2 0.03 <cit.>

  * WideResNet-50-4 + L_2 0.05 <cit.>

  * WideResNet-50-4 + L_2 0.1 <cit.>

  * WideResNet-50-4 + L_2 0.25 <cit.>

  * WideResNet-50-4 + L_2 0.5 <cit.>

  * WideResNet-50-4 + L_2 1 <cit.>

  * WideResNet-50-4 + L_2 3 <cit.>

  * WideResNet-50-4 + L_2 5 <cit.>

  * ResNet18 + L_2 0.01 <cit.>

  * ResNet18 + L_2 0.03 <cit.>

  * ResNet18 + L_2 0.05 <cit.>

  * ResNet18 + L_2 0.1 <cit.>

  * ResNet18 + L_2 0.25 <cit.>

  * ResNet18 + L_2 0.5 <cit.>

  * ResNet18 + L_2 1 <cit.>

  * ResNet18 + L_2 3 <cit.>

  * ResNet18 + L_2 5 <cit.>

  * ResNet18 + L_∞ 0.5/255 <cit.>

  * ResNet18 + L_∞ 1.0/255 <cit.>

  * ResNet18 + L_∞ 2.0/255 <cit.>

  * ResNet18 + L_∞ 4.0/255 <cit.>

  * ResNet18 + L_∞ 8.0/255 <cit.>

  * ResNeXt-50 32x4d + L_2 3 <cit.>

  * DenseNet + L_2 3 <cit.>

  * ShuffleNet + L_2 3 <cit.>

  * VGG16BN + L_2 3 <cit.>

  * MnasNet + L_2 3 <cit.>

  * MobileNet + L_2 3 <cit.>

  * ViT-base Patch16 + L_∞ 4/255 <cit.>

  * ConvNeXt-base + L_∞ 4/255 <cit.>

  * ViT-small Patch16 + L_∞ 4/255 <cit.>

  * Swin-base ImageNet1K + L_∞ 4/255 <cit.>

  * ConvNeXt-small + L_∞ 4/255 <cit.>

  * Swin-small ImageNet1K + L_∞ 4/255 <cit.>






 §.§ Capabilities Scores


Separate runs for capabilities scores were conducted between base and chat language models. Table <ref> presents these scores, highlighting the performance differences across various capabilities benchmarks. These scores provide a relative metric, and act as a comparative measure between models rather than an absolute one.












 §.§ Capabilities Evaluations: Correlation Matrices


As an intermediate step in our implementation of PCA, we compute Spearman correlation matrices for both base and chat language models. We show these matrices in Figure <ref> to show the correlation of various capabilities tasks across model types.

Both base and chat models show strong correlations between many benchmark pairs, indicating that performance on one task often predicts performance on others. Furthermore, some benchmarks form clusters with higher inter-correlations, suggesting they may be measuring identical capabilities. Other benchmarks, such as MATH and LAMBADA, show lower correlations with other tasks, potentially indicating they measure more distinct capabilities. Furthermore, noticeable differences in correlation patterns between base and chat models highlight the impact of fine-tuning on the relationships between various capabilities.









 §.§ Strict Instruction Following




  
Area Overview. Strict instruction following evaluations benchmark how well AI models can strictly adhere to specific instructions or rules, which may ensure AI systems behave as intended and follow safety guidelines.



  
Datasets. We evaluate strict instruction following using the following datasets:


    
  * IFEval focuses on verifiable instructions, such as word count requirements or keyword usage, across approximately 500 prompts.
    
  * RuLES evaluates models on 14 text-based scenarios, each with specific rules to follow. It includes a Basic test suite for straightforward rule adherence, a Benign suite to test if rules are violated in response to unrelated prompts, and a Red Team suite for adversarial scenarios (e.g. user presents a misleading reinterpretation of the rule, or disguises a request for the model to break the rule).


r0.45



Strict Instruction
Following Evaluation     Capabilities
Correlation 

IFEval     57.8% 

RuLES Basic     41.6% 

RuLES Benign     23.5% 

RuLES Red Team     16.1% 


Generally, strict instruction following exhibits a moderate correlation with capabilities for easier scenarios, and is not correlated for harder scenarios.





Empirical analysis of safetywashing. Is strict instruction following mostly determined by upstream model capabilities?

We find a moderate positive correlation between upstream model capabilities and performance on IFEval (57.8%) and RuLES Basic (41.6%). The RuLES Benign and Red Team suites exhibit even weaker correlations, indicating the ability of models to consistently follow strict instructions—especially in challenging or potentially problematic contexts—does not seem to improve with capabilities.

The higher correlation with IFEval and RuLES Basic suggests that evaluations using simple, easy-to-follow instructions may be more prone to safetywashing. However, strict instruction following, when combined with red-team prompts or obfiscation, does not seem to be correlated with upstream model capabilities.









 

            
    










 §.§ Hallucinations and Misconceptions: Expanded




  
Area overview. Factual reliability encompasses the ability of AI systems to generate and process accurate, truthful, and reliable content. This includes avoiding hallucinations, misconceptions, and generation of content that is plausible-sounding but factually incorrect or nonsensical. We further test correlations by measuring the capabilities coefficient on a generative benchmark (TruthfulQA Generation) and a discriminative benchmark (HaluEval).



  
Datasets. We describe the datasets we use below.

    
  * The TruthfulQA Generation task <cit.> aims to assess models' ability to generate truthful statements and avoid misconceptions. Unlike the MC1 task evaluated in the main paper, the generation task evaluates a 1-2 sentence generation from a model. We use a GPT-4o judge to judge model outputs. The truthful score indicates the truthfulness of the answer, while the truthful*informative score normalizes by the percentage of the model's answers that are informative (e.g. "I have no comment" would not be informative). 
    
  * HaluEval <cit.> is a benchmark designed to evaluate large language models' ability to recognize hallucinations. It aims to assess the model's ability to judge model responses in hypothetical QA, dialogue, and text summarization contexts. Hallucinated examples are generated from HotpotQA, OpenDialKG, and CNN/DailyMail.


R0.45
    

    
    2*Misinformation
Evaluations     2cCapabilities 

        2cCorrelation 

    2-3
        Base     Chat 

    TruthfulQA Generation         

     Truthful*Information Score     49.6%     72.9% 

     Truthful Score     74.7%     32.8% 

    HaluEval         

    HaluEval All     71.6%     56.7% 

    HaluEval Summarization      53.5%     34.2%

    HaluEval Dialogue      69.2%     89.1% 

    HaluEval QA     46.5%     18.4% 

     
    Hallucinations and misconceptions are generally prone to safetywashing.
    






  
Empirical analysis of safetywashing. Is factual reliability and hallucination reduction mostly upstream of latent general capabilities?

Our analysis shows that as models' general capabilities improve, the tendency for hallucinations decreases across the board for both base and chat models, illustrating that these topics are generally prone to safetywashing. This correlation is observed in both generative and classification tasks across both base and chat models. This has also been confirmed by previous work which has observed that larger language models tend to exhibit reduced hallucinations <cit.>.




            









 §.§ Capabilities Correlations for All Tested Evaluations



This section presents a comprehensive overview of the capabilities correlations for both base and chat models across all tested evaluations. We used lm-eval-harness for implementing most evaluations. For instance, the TruthfulQA MC1 implementation in lm-eval-harness employs a few-shot prompt with generic Q/A questions, avoiding misleading ones.





  §.§.§ Capabilities


llcc
Spearman correlations of capabilities component evaluations with the capabilities score, reported as a percentage. As expected, across base and chat models, the capabilities correlations of the component capabilities evaluations is high. 

Name     Metric     Base
Correlations (%)     Chat
Correlations (%) 

[]Capabilities Correlations for Capabilities Component (continued) 

Name     Metric     Base
Correlations (%)     Chat
Correlations (%) 

4rContinued on next page 

LogiQA <cit.>     Accuracy     86.0     86.1 

PIQA <cit.>     Accuracy     89.6     88.9 

Hellaswag <cit.>     Accuracy     93.7     84.9 

Winogrande <cit.>     Accuracy     92.1     88.7 

COPA <cit.>     Accuracy     81.4     87.8 

MedQA <cit.>     Accuracy     94.7     89.7 

ARC Challenge <cit.>     Accuracy     95.5     83.8 

MMLU <cit.>     Accuracy     93.1     83.2 

MATH <cit.>     Equivalence     84.4     61.5 

LAMBADA <cit.>     Accuracy     72.8     80.4 

GSM8K <cit.>     Exact Match     86.6     80.7 

BBH <cit.>     Exact Match     81.4     80.2 





  §.§.§ Alignment and Scalable Oversight


The following subsections present the Spearman correlations of safety evaluations with capabilities scores (“capabilities correlations”). For the LMSYS Chatbot Arena evaluation, we excluded chat models that were not available.

lcc
Correlations between alignment and oversight benchmarks and the capabilities score across models, reported as percentages. 

Name     Base
Correlations (%)     Chat
Correlations (%) 

[]Capabilities Correlations for Alignment and Scalable Oversight (continued) 

Name     Base
Correlations (%)     Chat
Correlations (%) 

3rContinued on next page 

MT Bench <cit.>     64.2     78.7 

AlpacaEval <cit.>     -     59.9 

LMSYS Chatbot Arena <cit.>     -     62.1 

GPQA <cit.>     80.4     77.7 

QuALITY <cit.>     90.5     88.8 





  §.§.§ Machine Ethics


Note that we obtained a subset of the SafetyBench questions directly from the authors, as the dataset was not publicly released. 

llccc
Capabilities correlations for bias datasets. We also found that MoralChoice was saturated, with most models getting around 100%. MACHIAVELLI had a low slope, with the score not changing much across models. Note that to ensure that higher correlations meant safer models, we negate the MACHIAVELLI score, and select the non-sycophantic answer for Sycophancy. 

Evaluation     Dataset     Base
Correlations (%)     Chat
Correlations (%) 

[]Capabilities Correlations for Machine Ethics (continued) 

Evaluation     Dataset     Base
Correlations (%)     Chat
Correlations (%) 

4rContinued on next page 

MoralChoice <cit.>     -     25.8     46.7 

SafetyBench <cit.>     -     65.1     71.7 

Model Written Evals <cit.>     Sycophancy     -65.6     -66.8 

3*MACHIAVELLI <cit.>     Power     -54.3     -46.1 

    Utility     -48.3     -49.9 

    Violations     8.3     -52.9 

6*ETHICS <cit.>     All     70.3     82.2 

    Commonsense     59.6     69.3 

    Deontology     45.9     38.8 

    Justice     68.2     50.9 

    Utilitarianism     56.6     75.0 

    Virtue     55.9     73.5 





  §.§.§ Bias and Toxicity

llccc
Capabilities correlations for bias datasets. Correlations reported as percentages. The Advanced AI Risk score was negated so that a higher score meant less risky. 

Evaluation     Dataset     Base
Correlations (%)     Chat
Correlations (%) 

[]Capabilities Correlations for Bias (continued) 

Evaluation     Dataset     Base
Correlations (%)     Chat
Correlations (%) 

4rContinued on next page 

Winogender <cit.>     -     85.9     75.6 

Crows Pairs English <cit.>     -     -31.6     28.5 

Simple Cooccurrence Bias <cit.>     -     -12.3     -37.3 

Toxigen <cit.>     -     56.0     30.7 

Advanced AI Risk <cit.>     -     -60.6     -42.6 

2*BBQ <cit.>     Ambiguous     30.8     -37.3 

    Disambiguated     83.6     76.8 

7*[l]Discrim-Eval
(Explicit) <cit.>     Maximum Difference     14.1     33.2 

    Hispanic-White     1.4     13.6 

    Black-White     27.8     27.9 

    Female-Male     9.6     17.1 

    Non-Binary-Male     13.1     34.2 

    Younger than 60 - Age 60     14.6     -43.2 

    Older than 60 - Age 60     -52.9     -30.2 





  §.§.§ Misconceptions and Hallucinations

llccc
Capabilities correlations for misconceptions and hallucinations datasets. 

Evaluation     Dataset     Base
Correlations (%)     Chat
Correlations (%) 

[]Capabilities Correlations for Misconceptions (continued) 

Evaluation     Dataset     Base
Correlations (%)     Chat
Correlations (%) 

4rContinued on next page 

4*TruthfulQA <cit.>     MC1     69.7     81.2 

    Gen: Truth Score     74.7     32.8 

    Gen: Info Score     -49.1     23.1 

    Gen: Truth*Info Score     49.6     72.9 

4*HaluEval <cit.>     All     71.6     56.7 

    QA     46.5     18.4 

    Summarization     53.5     34.2 

    Dialogue     69.2     89.2 





  §.§.§ Calibration

llcc
Accuracy correlations for calibration metrics. Correlations reported as a percent. The capabilities correlation was not used, but rather correlation with the dataset accuracy (e.g., MMLU) across models. To ensure a positive correlation meant safer models, the score we used for calculating correlations is 1 -Brier Score for Brier Score entries and 1 -RMSCE for RMSCE entries.  

Metric     Dataset     Base
Correlations (%)     Chat
Correlations (%) 

[]Correlations for Calibration (continued) 

Metric     Dataset     Base
Correlations (%)     Chat
Correlations (%) 

4rContinued on next page 

3*Brier Score     MMLU     98.6     95.5 

    PIQA     98.1     99.2 

    MedQA     98.7     83.4 

3*[l]Brier Score
 Temperature Tuned     MMLU     98.5     99.9 

    PIQA     98.6     99.2 

    MedQA     99.7     96.7 

3*RMSCE     MMLU     2.5     20.1 

    PIQA     31.9     47.9 

    MedQA     41.2     38.6 

3*[l]RMSCE
 Temperature Tuned     MMLU     -35.9     -8.2 

    PIQA     -11.1     -7.0 

    MedQA     12.4     31.0 





  §.§.§ Adversarial Robustness




llccc
Base models' capabilities correlations for GLUE <cit.>, AdvGLUE <cit.>, and AdvGLUE++ <cit.>, reported as a percent. We find that neither AdvGLUE nor AdvGLUE significantly decorrelates performance on GLUE relative to the GLUE dataset. 

Evaluation     Dataset     GLUE
CC (%)     AdvGLUE
CC (%)     AdvGLUE++
CC (%) 

[]Base models' capabilities correlations for GLUE, AdvGLUE, and AdvGLUE++ (continued) 

Evaluation     Dataset     GLUE
CC (%)     AdvGLUE
CC (%)     AdvGLUE++
CC (%) 

5rContinued on next page 

6*GLUE
Split     MNLI Matched     67.3     66.0     62.8 

    MNLI Mismatched     66.9     68.2     61.8 

    QNLI     14.9     19.1     21.3 

    QQP     33.2     19.0     32.8 

    RTE     49.9     76.4     32.1 

    SST2     52.0     70.3     62.9 



llccc
Capabilities correlations for adversarial robustness datasets. Correlations reported as percentages. The metric used is attack failure rate for HarmBench splits. 

Evaluation     Dataset     Base
Correlations (%)     Chat
Correlations (%) 

[]Capabilities Correlations for Adversarial Robustness (continued) 

Evaluation     Dataset     Base
Correlations     Chat
Correlations 

4rContinued on next page 

ANLI <cit.>     -     74.5     81.5 

AdvDemonstration <cit.>     -     57.9     63.9 

7*HarmBench DirectRequest <cit.>     Biochemical     -58.0     -9.3 

    Cybercrime     -59.0     -19.5 

    Harassment     -46.6     -15.8 

    Harmful     -54.3     7.3 

    Illegal     -47.1     -9.8 

    Misinfo     -53.9     -38.7 

    All     -65.5     -18.2 

7*HarmBench HumanJailbreak <cit.>     Biochemical     -49.6     -22.1 

    Cybercrime     -73.8     -29.3 

    Harassment     -85.5     -34.1 

    Harmful     -74.6     -29.9 

    Illegal     -71.1     -28.5 

    Misinfo     -76.9     -41.6 

    All     -79.2     -31.4 

7*HarmBench TAP-T <cit.>     Biochemical     -62.0     -26.3 

    Cybercrime     -60.0     -33.0 

    Harassment     -77.1     -34.3 

    Harmful     -59.4     -22.3 

    Illegal     -68.9     -35.9 

    Misinfo     -74.5     -56.8 

    All     -78.7     -42.8 

7*HarmBench GCG-T <cit.>     Biochemical     -55.8     -14.1 

    Cybercrime     -74.9     -26.6 

    Harassment     -57.7     -31.0 

    Harmful     -48.2     -18.7 

    Illegal     -60.5     -15.4 

    Misinfo     -55.4     -35.5 

    All     -61.5     -28.4 





  §.§.§ Weaponization Capabilities

llccc
Capabilities correlations for weaponization capabilities datasets. The metric used for CybersecEval2 was vulnerability detection rate for Exploit, safe suggestion rate for Instruct, Safe response rate for MITRE, attack failure rate for prompt injection, and accuracy for FRR. WMDP uses a score that inverts the accuracy. 

Evaluation     Dataset     Base
Correlations (%)     Chat
Correlations (%) 

Evaluation     Dataset     Base
Correlations (%)     Chat
Correlations (%) 

4rContinued on next page 

4*WMDP <cit.>     All     -90.6     -88.6 

    Biosecurity Split     -92.5     -87.5 

    Chemical Security Split     -90.8     -81.1 

    Cybersecurity Split     -88.4     -86.0 

5*CybersecEval2 <cit.>     Exploit     -37.5     -50.3 

    Instruct     -48.9     -85.8 

    MITRE     -19.6     40.4 

    Prompt Injection     -16.8     -18.6 

    FRR     -44.2     -24.9 






  §.§.§ Strict Instruction Following

llccc
Capabilities correlations for strict instruction following datasets. 

Evaluation     Dataset     Base
Correlations (%)     Chat
Correlations (%) 

[]Capabilities Correlations for Rule Following (continued) 

Evaluation     Dataset     Base
Correlations (%)     Chat
Correlations (%) 

4rContinued on next page 

IFEval <cit.>     -     16.6     57.8 

4*RuLES <cit.>     Basic     33.5     41.6 

    Benign     35.7     23.5 

    Red Team     0.0     16.1 

    All     34.4     26.5 






















 §.§ Closed Source Model Evaluations: GPT-4o Capabilities Score



In general, closed source models were excluded from our main paper analysis because of the technical challenges of calculating log probabilities and incompatibility with certain evaluation libraries. In this section, we compare the capabilities of GPT-4o with other open-source models using our analysis. To do so, we recompute all capabilities scores with all models including GPT-4o, while excluding BBH and LAMBADA from the capabilities score calculations. Table <ref> contains our calculated scores of GPT-4o on the capabilities tasks, while in Figure <ref> we observe clear gap between GPT-4o and the current open source models.












 













    


    









 Paper Title: Enhanced Structured State Space Models via Grouped FIR Filtering and Attention Sink Mechanisms.


Quantum thermalization and Floquet engineering in a spin ensemble with a clock transition
    Andrei Faraon
    
=========================================================================================

§ ABSTRACT 
Structured State Space Models (SSMs) have emerged as compelling alternatives to Transformer architectures, offering linear-time complexity and superior performance in various sequence modeling tasks. Despite their advantages, SSMs like the original Mamba-2 face training difficulties due to the sensitivities introduced by the extended series of recurrent matrix multiplications. In this paper, we propose an advanced architecture that mitigates these challenges by decomposing A-multiplications into multiple groups and optimizing positional encoding through Grouped Finite Impulse Response (FIR) filtering. This new structure, denoted as Grouped FIR-enhanced SSM (GFSSM), employs semiseparable matrices for efficient computation. Furthermore, inspired by the "attention sink" phenomenon identified in streaming language models, we incorporate a similar mechanism to enhance the stability and performance of our model over extended sequences. Our approach further bridges the gap between SSMs and Transformer architectures, offering a viable path forward for scalable and high-performing sequence modeling.
    


§ INTRODUCTION



In recent years, the field of deep learning has been revolutionized by the introduction of the Transformer architecture and its attention mechanism <cit.>, which have become the backbone of most state-of-the-art models across various domains, including natural language processing (NLP), computer vision, and speech recognition. However, these architectures are not without limitations. Specifically, the quadratic time and memory complexity of the self-attention mechanism pose significant challenges when handling long sequences. To address these issues, several subquadratic-time architectures have been proposed, among which Structured State Space Models (SSMs) <cit.> have garnered considerable interest due to their linear-time complexity and promising performance. Mamba <cit.> and its successor Mamba-2 <cit.> represent significant strides in this direction, implementing SSMs that utilize selective state updates to efficiently process long sequences. Despite these advancements, Mamba-2 still encounters certain obstacles, primarily due to the sensitivity of the recurrent weight multiplications, which can complicate training. These sensitivities arise from a long sequence of multiplications, making the model potentially unstable and difficult to optimize effectively.

To alleviate these issues, we propose a novel architecture that reimagines the positional encoding mechanism within the SSM framework. Our innovation revolves around decomposing the recurrent multiplications into smaller, more manageable groups and employing Finite Impulse Response (FIR) filters <cit.> to enhance positional encoding. This modification not only improves the sensitivity issues but also leveraging semiseparable matrices, which can be calculated with reduced complexity. Additionally, inspired by recent insights from the domain of streaming language models, we integrate an attention sinks mechanism <cit.> within our architecture. Previous studies have highlighted the importance of retaining initial tokens as "sinks" that stabilize the attention mechanism over extended sequences. By incorporating a similar mechanism, our model benefits from improved stability and enhanced performance, particularly for tasks involving long-range dependencies.

We denote our new architecture as Grouped FIR-enhanced Structured State Space Model (GFSSM). GFSSM represents a dual-pronged improvement over traditional SSMs: first, by mitigating the training difficulties associated with long recurrent multiplications, and second, by incorporating attention sinks to handle extended sequences more effectively. In summary, our contributions can be outlined as follows:


    
  * We introduce the Grouped FIR filter to optimize the positional encoding within SSMs, thereby addressing the sensitivity and training difficulties associated with long sequences of multiplications.
    
  * We incorporate an attention sink mechanism to enhance model stability and performance over extended sequences, inspired by insights from streaming language models.


This paper is structured as follows: Section 2 provides a detailed overview of related work, including foundational models and their limitations. Section 3 presents the mathematical formulation and architectural details of GFSSM. Section 4 outlines our experimental setup and results, followed by an in-depth analysis. Finally, Section 5 concludes the paper and discusses potential avenues for future work.



§ RELATED WORK





 §.§ Structured State Space Models (SSMs)

Structured State Space Models (SSMs) <cit.> have attracted attention for their ability to offer linear-time complexity while maintaining competitive performance. Among these, the Mamba architecture <cit.> represents a significant advancement. Mamba leverages selective state updates to efficiently handle long sequences, making it a viable alternative to Transformers. However, despite its linear complexity, Mamba's performance has been limited by the sensitivity and difficulty in training due to extended sequences of recurrent multiplications. Mamba-2 <cit.> improved upon its predecessor by introducing a more structured approach, leveraging the State Space Duality (SSD) framework. By restricting the SSM parameters further and employing structured decompositions of semiseparable matrices, Mamba-2 achieved enhanced computational efficiency and performance. Nevertheless, the core challenge of training sensitivity due to recurrent dynamics remained an open problem.



 §.§ Finite Impulse Response (FIR) Filters

FIR filters <cit.> have been widely used in signal processing for their stability and straightforward implementation. Recent works have explored their potential in the context of machine learning, particularly for optimizing convolutional neural networks (CNNs) [6]. These filters can be tuned to emphasize specific frequencies, making them useful for various signal processing tasks. In the context of SSMs, deploying FIR filters can help mitigate sensitivity issues by effectively smoothing positional encodings.



 §.§ Attention Sinks

The concept of attention sinks, as explored by <cit.>, addresses some unique challenges in deploying large language models (LLMs) for streaming applications. By retaining key-value (KV) states of initial tokens, attention sinks stabilize the attention mechanism, thereby enabling LLMs to generalize to sequences much longer than those encountered during training. This insight is particularly relevant for our work, as it offers a potential solution to improve stability and performance over extended sequences in SSMs.


§ METHODOLOGY


This section outlines the theoretical foundations and architectural innovations of the Grouped FIR-enhanced Structured State Space Model (GFSSM). We begin by revisiting the fundamental principles and equations that underpin our approach, followed by detailed explanations of the grouped FIR filtering and the integration of attention sinks. Finally, we discuss the integration of these components into the overall GFSSM architecture.



 §.§ Fundamentals of State Space Duality (SSD)

In the Mamba-2 SSD framework, the forward pass is governed by the following equations:


    h_t   = A_t h_t-1 + B_t x_t
    
            y_t   = C_t^⊤ h_t,

where h_t denotes the hidden state at time t, x_t is the input, and y_t represents the output. A_t is a scalar times identity matrix, denoted as a_t, which simplifies the recurrent computation. B_t and C_t are matrices that define the state space dynamics, which can vary over time. By definition, h_0 = B_0 x_0. By induction, the h_t can be obtained as:


    h_t    = a_t a_t-1⋯ a_1 B_0 x_0 + a_t a_t-1⋯ a_2 B_1 x_1 + ⋯
          + a_t a_t-1 B_t-2 x_t-2 + a_t B_t-1 x_t-1 + B_t x_t 
       = ∑_s=0^t a_t:s^× B_s x_s,

where a_t:s^× denotes the product of a terms from s+1 to t. When multiplied by C_t to produce y_t and vectorized over t ∈ [𝚃], the matrix transformation form of SSD becomes:


    y_t    = ∑_s=0^t a_t:s^× C_t^⊤ B_s x_s, 
    
            y    = (L ∘ C^⊤B) x = Mx,

where L is represented as L_ji = a_j ⋯ a_i+1:


    L =
        [          1                                            ;        a_1          1                                 ;    a_2 a_1        a_2          1                      ;          ⋮          ⋮          ⋱          ⋱           ; a_T-1⋯ a_1 a_T-1⋯ a_2          ⋯      a_T-1          1;            ].


The potential limitation of the extended series of a_i multiplications arises when dealing with long text sequences. The matrix L_ji can decay to 0 or explode, resulting in numerical instability. Furthermore, any intermediate poorly conditioned a_i will have a prolonged adverse impact, propagating across subsequent tokens. For instance, if a_i is 0 at an initial position, all following states will remain 0. This extended multiplication series also complicates training, as noted in Mamba-2: higher precision for the main model parameters may be necessary due to the sensitivity of SSDs to recurrent dynamics. Parameter storage in fp32 can improve stability by mitigating these recurrent sensitivities.




 §.§ Grouped FIR Filtering

In GFSSM, we address the sensitivity and training difficulties inherent in SSMs by optimizing the computation of the hidden states and refining the positional encoding mechanism. The sensitivity in training SSMs arises primarily from the extended series of recurrent multiplications. To mitigate this issue, we decompose recurrent multiplications into smaller, more manageable groups and employ Finite Impulse Response (FIR) filters to enhance the positional encoding. 

For simplicity, we illustrate the method using four groups with a 4th-order FIR filter. However, this can be easily generalized to any Q groups with an n-th order FIR filter. The new recurrent form is:


    s_t   = k_0 B_t x_t + k_1 B_t-1 x_t-1
       + k_2 B_t-2 x_t-2 + k_3 B_t-3 x_t-3



    {
            h_t^i   = A_t h_t-1^i + s_t,    i = mod(t, Q) 
    
            h_t^i   = h_t-1^i,    i ≠mod(t, Q)
        .



    y_t = C_t ∑_i=0^Q-1 h_t^i

where s_t is the filtered output, and k_0, k_1, k_2, k_3 are learnable filter coefficients.

The core idea of GFSSM is to split the long input sequence into Q groups, each with its corresponding hidden state. This way, the recurrent multiplications of a_i are reduced to 1/Q of their original length, thereby mitigating their potential to cause numerical instability. Finally, the output y_t is computed by summing the contributions from all grouped states. One potential problem arises from splitting tokens into different groups: if tokens x_i and x_j belong to different groups, they will never be directly correlated. To address this issue, we introduce a computationally efficient solution by adding a FIR filter that processes the B_t x_t sequence. This enables tokens in different groups to be indirectly correlated, thereby preserving the overall coherence of the encoded sequence. To provide further clarity, we present the structure of matrix L for GFSSM, focusing on the first 13 elements, as shown in Figure <ref>.

The application of FIR filters facilitates smoother transitions and correlations between grouped states, ultimately rendering the model more robust and easier to train. By combining grouped FIR filtering and robust matrix computations, GFSSM offers a viable solution for the inherent challenges of traditional SSM architectures, providing improved performance and stability over extended sequences.



 §.§ Integration of Attention Sink Mechanisms

Inspired by the attention sink phenomenon observed in streaming language models, we incorporate a similar mechanism within GFSSM to improve stability and performance over extended sequences. The attention sink mechanism ensures that the initial tokens in a sequence serve as stable reference points, effectively acting as "sinks" that help maintain the model's focus and coherence over long dependencies. Additionally, it addresses the absence of initial states, thereby ensuring no information is lost from the initial a_0 to a_Q.

In our GFSSM architecture, we initialize the state using learnable prompts that serve as stand-ins for the first few tokens. This initialization is critical for ensuring meaningful state updates from the beginning of the sequence. Formally, we define the state updates incorporating the attention sink mechanism as follows, beginning with the initialization using learnable prompts. For initialization, we start with:

    h_init^i   = 0, 
    
            y    = (L ∘ C^⊤B) 
            [   x_-Q; x_-Q+1;      ⋮;   x_-1;    x_0;    x_1;      ⋮ ]
            + P ·0

where the initial sequence { x_-Q, x_-Q+1, …, x_-1} consists of Q learnable prompts. For continued state updates, the state initialization incorporates the cached prior hidden state:

    h_init^i   = h_-Q^i, 
    
            Y    = (L ∘ C^⊤B) 
            [ x_-Q+1; x_-Q+2;      ⋮;    x_0;    x_1;      ⋮ ]
            + P 
            [ h_-Q^1; h_-Q^2; h_-Q^3; h_-Q^4 ]



    .8!P = 
        [           A_0             1             1             1;           A_0           A_1             1             1;           A_0           A_1           A_2             1;           A_0           A_1           A_2           A_3;        A_4A_0           A_1           A_2           A_3;        A_4A_0        A_5A_1           A_2           A_3;        A_4A_0        A_5A_1        A_6A_2           A_3;        A_4A_0        A_5A_1        A_6A_2        A_7A_3;     A_8A_4A_0        A_5A_1        A_6A_2        A_7A_3;     A_8A_4A_0     A_9A_5A_1        A_6A_2        A_7A_3;     A_8A_4A_0     A_9A_5A_1    A_10A_6A_2        A_7A_3;     A_8A_4A_0     A_9A_5A_1    A_10A_6A_2    A_11A_7A_3; A_12A_8A_4A_0     A_9A_5A_1    A_10A_6A_2    A_11A_7A_3;               ],

where the initial sequence { x_-Q+1, x_-Q+2, …, x_0} is the preceding Q-1 input tokens, and h^i_-Q is the cached hidden state from the previous sequence.

This attention sink mechanism ensures that the initial tokens persist as stable anchors that maintain focus over the sequence's length, thus improving the robustness of our model. Combining grouped FIR filtering and the attention sink mechanism, our final GFSSM model effectively leverages semiseparable matrices within the broader SSD architecture, offering enhanced performance and stability over extended sequences. 



 §.§ Semiseparable Matrix Decomposition

A crucial component of our new structure is the decomposition of the recurrent update matrix into multiple semiseparable matrices. This decomposition enables efficient computation while maintaining the linear complexity of state updates. We denote the decomposed matrix as L, formulated as:


    L = k_0 L_0 + k_1 L_1 + k_2 L_2 + k_3 L_3,

where each L_i represents a semiseparable matrix that captures the contributions from different stages of the FIR filter. Semiseparable matrices are characterized by a structure that allows fast multiplications, like Mamba-2, significantly reducing computational overhead.




§ CONCLUSION AND FUTURE WORK


In this paper, we introduced the Grouped FIR-enhanced Structured State Space Model (GFSSM) as an innovative extension to traditional SSM architectures. Our approach addresses critical training challenges posed by extended series of recurrent matrix multiplications, which often lead to instabilities and difficulties in optimization. By decomposing these multiplications into more manageable groups and incorporating Finite Impulse Response (FIR) filters, we could potentially improve the robustness and trainability of the system. Additionally, inspired by the "attention sink" phenomenon observed in streaming language models, we integrated a similar mechanism into GFSSM to enhance stability and performance over extended sequences.

While we have outlined the theoretical foundations and architectural details of GFSSM, empirical validations are essential to demonstrate the model's practical advantages. Therefore, future work will entail comprehensive experiments to evaluate the performance of GFSSM across various sequence modeling tasks, including but not limited to natural language processing, speech recognition, and genomics.

    ieeenat_fullname
    







 Paper Title: Towards an Integrated Performance Framework for Fire Science and Management Workflows.









Towards an Integrated Performance Framework for Fire Science and Management Workflows


    Hena Ahmed1
Halıcıoğlu Data Science Institute 

University of California, San Diego

La Jolla, CA, USA 

h7ahmed@ucsd.edu
Ravi Shende1
San Diego Supercomputer Center 

University of California, San Diego

La Jolla, CA, USA 

rshende@ucsd.edu
Ismael Perez
San Diego Supercomputer Center 

University of California, San Diego

La Jolla, CA, USA 

i3perez@sdsc.edu
Daniel Crawl
San Diego Supercomputer Center 

University of California, San Diego

La Jolla, CA, USA 

lcrawl@ucsd.edu
Shweta Purawat
San Diego Supercomputer Center 

University of California, San Diego

La Jolla, CA, USA 

shpurawat@ucsd.edu
İlkay Altintaş
San Diego Supercomputer Center and 

Halıcıoğlu Data Science Institute 

University of California, San Diego

La Jolla, CA, USA 

ialtintas@ucsd.edu

    
=====================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================


Equal contribution


§ ABSTRACT 
Reliable performance metrics are necessary prerequisites to building large-scale end-to-end integrated workflows for collaborative scientific research, particularly within context of use-inspired decision making platforms with many concurrent users and when computing real-time and urgent results using large data. This work is a building block for the National Data Platform, which leverages multiple use-cases including the WIFIRE Data and Model Commons for wildfire behavior modeling and the EarthScope Consortium for collaborative geophysical research. 
This paper presents an artificial intelligence and machine learning (AI/ML) approach to performance assessment and optimization of scientific workflows. 
An associated early AI/ML framework spanning performance data collection, prediction and optimization is applied to  wildfire science applications within the WIFIRE BurnPro3D (BP3D) platform for proactive fire management and mitigation. 





Cyberinfrastructure, Workflows, Performance Analysis, Artificial Intelligence, Machine Learning 




§ INTRODUCTION

Scientific application workflows have become a key tool in natural disaster mitigation and response. Real-time sensor and satellite data now provide invaluable resources for urgent science analytics to be conducted with remarkable speed and precision. Workflows and larger cyberinfrastructures (CIs) powered by such data can deliver critical knowledge about imminent natural hazards such as wildfires <cit.>, earthquakes <cit.>, and volcanic eruptions <cit.>. 

However, the vast influx of raw and pre-processed data from geo-distributed sources presents challenges to the design of scalable cyberinfrastructures for data to knowledge workflows, thus heightening the need for developing a computing continuum of integrated cloud-to-edge resources <cit.>. A computing continuum especially enables novel implementations of urgent application workflow with particular attention to efficient data processing, and reliable but timely data to knowledge transfer to support urgent decision-making <cit.>.

Earlier works presented the WIFIRE cyberinfrastructure of integrated end-to-end workflows for wildfire behavior modeling <cit.>, as well as a use case of the computing continuum to support data-driven workflows for air quality prediction to manage wildfire impacts <cit.>. The WIFIRE Commons itself is one such use-case for the National Data Platform project, which will leverage the computing continuum to democratize scientific data access and analysis through a national cyberinfrastructure <cit.>. Fig <ref> outlines an early performance pipeline for the National Data Platform (NDP).






Our work particularly addresses the issue of performance optimization for the WIFIRE-powered BurnPro3D (BP3D) platform <cit.>. BP3D is a decision support platform to inform and optimize prescribed burn planning for wildfire management. The platform works in tandem with other WIFIRE frameworks, namely QUIC-Fire fire and atmospheric models and FastFuels 3D fuel structure models, in order to identify environmental conditions and ignition patterns that are optimal for prescribed burns. 

BP3D is a user-facing tool for geographically distributed land managers and fire planners, and is applied in a variety of computing environments with different capabilities. In this paper, we present preliminary steps towards a performance prediction framework that will be used by BP3D users to assess the necessary resource provisions to run BP3D given environmental and fuel data inputs. We present a performance framework for integrating data processing and AI/ML techniques in order to predict resource consumption during BP3D runs, and thereby improve the scalability and reliability of underlying BP3D data workflows. We implement a data processing architecture to collect and prepare performance data for AI/ML analysis, and demonstrate examples of predictive modeling techniques for performance evaluation and provision of our AI-driven data workflows. This is part of a larger work to optimize sub-systems for integrated end-to-end data cyberinfrastructures, which is critical to enabling efficient data processing and modeling across the computing continuum.

The rest of the paper is structured as follows: Section <ref> discusses related works in the existing scientific literature, Section <ref> describes a use case application of our workflow architecture, Section <ref> describes our methodological approach and design, Section <ref> demonstrates early results of our framework, and Section <ref> summarizes conclusions and future endeavors of this work. 



§ RELATED WORKS



Large-scale data cyberinfrastructures are next generation platforms for collaborative research workflow and data sharing. Beckman et al. <cit.> highlights the need for performance optimization across the computing continuum. Current case studies for scalable CIs include the Virtual Data Collaboratory for interdisciplinary data and science sharing presented by Parashar et al. <cit.>, and the EarthScope <cit.> framework for open access, real-time geophysical data, modeling, and educational services. Our work also draws from contributions in distributed and multimodal data architectures such as the Quantum Data Hub presented by Purawat et al. <cit.> and the AWESOME polystore using open-knowledge networks (OKNs) presented by Dasgupta & Gupta <cit.>.

 Nguyen et al. <cit.> presented methods for integrating machine learning techniques in scientific workflow systems to evaluate accuracy and scalability. As described in Parashar et al. <cit.>, the GeoSciFramework (commonly known as EarthScope) is one such case study for scalable architectures of scientific workflows and integrative machine learning environments that operate with continuously streaming geodetic and seismic data. However, an existing key problem area in developing scalable architectures for integrated machine learning and scientific workflows is developing knowledge management techniques to assimilate and prepare data from sources for AI/ML analysis. This concept is also referred to as the “AI-readiness” of data. AI-readiness is especially important for urgent computing applications such as natural hazard modeling and prediction-making. Baru et al. <cit.> are currently addressing the challenge of finding and matching AI-ready data and models in an integrated platform, while also following guidelines for FAIR <cit.> for data provenance. Holding scientific work to FAIR data management principles  (where FAIR stands for Findability, Accessibility, Interoperability, and Reusability) is a key step to ensuring the responsible deployment of AI models and other data services. 


By building upon existing work in integrated AI/ML and scientific workflow architectures as well, this paper will further previous research towards global, integrated cyberinfrastructures that enable equitable data-driven technology sharing.


§ WORKFLOW USE CASE


The use case application described in this section was created to predict total resource consumption of BurnPro3D simulations for prescribed burns and wildfire mitigation <cit.>. An execution of BP3D takes a single set of environmental input data and runs an ensemble of simulations over multiple Kubernetes servers. We created an integrated ML/AI workflow that retrieves the input parameter values given to a BP3D run and resource consumption data that is generated throughout the run and stored on Nautilus servers. The workflow then takes a linear regression approach to predicting total CPU and memory usage of a BP3D run. 

For the purposes of this paper, the chosen machine learning method (linear regression) is rather arbitrary, as our intentions at this stage of research are to demonstrate a functional data-driven pipeline for AI/ML performance analysis, as opposed to choosing the most accurate or robust modeling approach for the task. So, we used a basic linear regression implementation with default model parameter values made accessible using the popular Python package Scikit-Learn. In future extensions of this work, we plan to conduct more extensive sensitivity analyses and parameter estimation techniques in order to implement modeling techniques that better fit the given data. The results of this particular use case are discussed in Section <ref>. 
 


§ APPROACH
  



In this section, we describe our methodology for retrieving resource consumption data and integrating AI/ML-informed decision-making to analyze overall performance of the BurnPro3D architecture for prescribed burn modeling. We describe two primary objectives in developing our AI/ML solution: 1) data preparation for AI-readiness (Section <ref>), and 2) integration of predictive ML/AI modeling methods into the scientific workflow architecture of BP3D (Section <ref>). 

    

 §.§ AI-Ready Data Preparation

    We first discuss the steps that were taken towards reaching our AI-readiness objective. Data assimilation and preparation requirements 
    will vary depending on the form of ML/AI analysis being applied. The use case we describe in this paper demonstrates results obtained through linear regression. So, in this section, we discuss the steps taken to achieve data that is AI-ready in the specific context of linear regression modeling. 
    
    By accessing the BurnPro3D API, we can retrieve data about individual ensembles of simulation runs generated using the QUIC-FIRE model. This paper deals with two different classes of data relevant to BurnPro3D simulations: first are the model input data features (weather and atmospheric data, ignition and fuel conditions, geospatial data) for BP3D models; second are performance data (runtime, CPU usage, memory usage, storage I/O, network usage) generated during a BurnPro3D ensemble run.

    Each BP3D ensemble is hosted on one or more Kubernetes nodes, where simulation runs are hosted on different pods in the node. The performance data is then stored as time series data on a Nautilus server, where it can be queried using PromQL (a querying language for Prometheus servers in Nautilus) and visualized on a Prometheus web user interface and/or Grafana dashboard.

    
    Prior to retrieving performance data, we store the following identifiers for each BP3D run: the set of all input parameters passed into the simulation, the unique and corresponding ensemble IDs (i.e., the pod/node pair in Kubernetes), and total simulation runtime. Once these identifiers are stored, we proceed to collect the start/end timestamps and input parameter values for each run per ensemble. This data set will later be used as training data for performance optimizing AI/ML methods. Occasionally, a simulation run will fail, as indicated in the data by  timestamps in the start, end, or total simulation time categories. We chose not to include failed runs in the final training data set. Table <ref> describes input and output features collected after a simulation run of BurnPro3D, and Table <ref> describes performance features retrieved during the run.



    



    
    
    

    We can then retrieve the performance data that was collected and stored in our Nautilus server during each simulation run, and pre-process the JSON-formatted data to achieve tabular data sets suitable for basic AI/ML models. For our experiments, we are focusing on resource consumption over the duration of a BP3D simulation run. So, we query for the minimum memory requested for each pod, the total CPU usage, and total memory usage during the time range of each run. Total CPU usage and memory usage of a BP3D simulation will be the target variables that we want to predict using AI/ML modeling methods. We also collect the partial CPU and memory usage of a run from the start time until certain time periods (either predetermined or pseudo-random) to represent refreshed prediction times. This data will then be used with an AI/ML approach to predict the performance consumption metrics of the entire duration of each run. 
    
    
    

 §.§ Predictive Model Design

    Having assembled an AI-ready data set, we now want implement a predictive model to evaluate the performance of BP3D simulations. This stage follows a standard experimental procedure of: 1) choosing a predictive modeling method, 2) data pre-processing and feature analysis, and 3) model fitting, training, and testing. 
    
    We have two target features in our data to indicate resource utilization during a given simulation run: total CPU usage (as a percentage of total CPU capacity) and total memory usage (as a percentage of total memory available). Before training the model, we calculated Pearson correlation coefficients to determine which of the variables (which consist of BP3D simulation input parameters and resource consumption data collected from a BP3D run) in our data set were most strongly indicative of changes in resource utilization. Figure <ref> shows the correlation matrix for our assembled data set.

    

   We select features that are strongly correlated (i.e., with a Pearson coefficient >.5) with CPU Usage and/or Memory Usage. We then proceed to the model fitting stage, which would resemble the linear regression use case described in Section <ref>. 






 §.§ User Experience Design

    We consider two potential implementations for determining how users can interact with our predictive modeling workflow. One implementation would automatically refresh model predictions at set time intervals. To do this, we would automate queries to retrieve performance data of the current simulation over set time ranges (e.g. from the start of the run until 5 minutes in), and then rerunning the predictive model, taking into account the most recent query results in order to obtain an updated prediction of resource utilization over the entire simulation run.

    A second potential implementation would enable users to refresh performance predictions at any time point after the model has started running, and then continue to update the prediction as desired. There is a brief time period where the run has started, but the user is not able to request a prediction, as it takes time for data to be put onto Kubernetes servers. Our approach is to wait 45 seconds after the run starts, then automatically refresh the prediction. From then onwards, we allow the user to refresh the prediction as desired. This design approach would be similar to the previously described implementation, except that updated performance data would only be queried when/if the user indicates, rather than at set time intervals. Using this approach, the data collected at each model refresh would consist of performance metrics from the start time of the simulation to the refresh time. The data collected from the start until the refresh time would be used to obtain an updated prediction of resource utilization over the entire simulation run. We simulate these refresh times in our training data by inserting duration columns (like duration_t2 from Figure 2) and querying from the start time until the duration time, generating data such as transmitted_packages_t2. We feed these into the model to predict the final resource consumption metrics.

    The decision ultimately centers on balancing user experience and the effectiveness of model training. In the first implementation, the model is trained at consistent time intervals, which could improve its ability to recognize patterns in BP3D resource consumption, and as a result, enhance prediction accuracy. However, this implementation might negatively impact the user experience due to limited user control and potential frustration arising from the passive nature of updates. Conversely, the second approach offers the user greater flexibility, allowing them to update predictions as needed and providing a more interactive experience. While this approach might introduce complexities in model performance, we could potentially mitigate these with issues with strategic data manipulation—such as using a ratio of refresh time to resource metrics instead of analyzing these metrics separately. This ratio is shown in  Figure <ref> with the _t1_ratio and _t2_ratio metrics. A hybrid approach, integrating the consistent training intervals of the first method and the user-initiated updates of the second could also be considered. This would provide a more balanced approach, though its increased complexity could result in a less intuitive user experience.


    
    
    

    
    

    



§ EARLY RESULTS



In the initial stages of the workflow, we retrieve resource consumption data from over 900 BP3D runs. The retrieved data will include performance statistics under four distinct categories: CPU usage, memory usage, network usage, and storage I/O. Post-processing, the tabular data will describe performance features for each BP3D run. In addition to the unique pod/node IDs for each simulation run, the final, cleaned performance database will contain the features listed in Tables <ref> - <ref>.


    












The next stage of the workflow (AI/ML modeling), outputs predictions of total CPU usage (in seconds) and memory usage (in bytes) during a BurnPro3D run. To demonstrate possible results at this stage, we return to the use case application of linear regression introduced in Section <ref>. For our linear regression model, we selected training features based on the feature analysis results described in Section <ref>. Figure <ref> shows the preliminary results of a linear regression model using sample performance data retrieved from BurnPro3D runs. 




The linear regression model predicted CPU and memory usage with R-squared error rates of 0.70626 and 0.9221, respectively. However, given the shortage of training and testing data at this stage of our research, there are too few data points to obtain accurate or generalizable predictions through linear regression, and are currently working towards generating and preparing sufficient amounts of BP3D data in order to develop more robust AI/ML models. For the purposes of this paper, our demonstrated AI-readiness and AI/ML modeling pipelines are fundamental building blocks of an early workflow architecture.


§ CONCLUSION & FUTURE WORKS


We have presented our approach and early results of an integrated AI/ML workflow for performance analysis of the BurnPro3D fire management platform. Our use-case can be applied for the integration of AI-ready data preparation and AI/ML predictive modeling techniques in an end-to-end scientific workflow. where the use-case presented in this paper limits the use of ML/AI to identify relationships between BP3D input parameters and total resource consumption, our ongoing work aims to also optimize resource consumption for the purpose of mitigating uncertainty and improving accuracy of BP3D outputs. This work is part of a broader effort towards integrating AI/ML-driven methods for performance optimization in large cyberinfrastructures, namely the in-progress National Data Platform project. Immediate extensions of this work include the incorporation of a runtime prediction modeling stage into the current workflow, and the introduction of end-to-end uncertainty quantification metrics in order to align our work with FAIR data management standards for scientific research.





§ ACKNOWLEDGMENTS
 
The authors would like to thank the WIFIRE and WorDS
teams for their collaboration and support of this study by NSF 2040676, 2134904 and 2333609, and  the Nautilus Kubernetes Cluster of the National Research Platform funded by in part by NSF awards 1730158, 1540112, 1541349, 1826967, 2112167 and 2120019.

IEEEtran




 Paper Title: The Energy Cost of Artificial Intelligence of Things Lifecycle.




IEEEexample:BSTcontrol




















    Shih-Kai Chou,
Jernej Hribar, Mihael Mohorčič and Carolina Fortuna

Department of Communication Systems, Jožef Stefan Institute, Slovenia

Email:{shih-kai.chou, jernej.hribar, miha.mohorcic, carolina.fortuna}@ijs.si

    August 5, 2024
============================================================================================================================================================================================================================

plain
plain





[MACHU]
  iot[IoT]Internet of Things
  cr[CR]Cognitive Radio
  ofdm[OFDM]orthogonal frequency-division multiplexing
  ofdma[OFDMA]orthogonal frequency-division multiple access
  scfdma[SC-FDMA]single carrier frequency division multiple access
  rbi[RBI] Research Brazil Ireland
  rfic[RFIC]radio frequency integrated circuit
  rf[RF]radio frequency
  sdr[SDR]Software Defined Radio
  sdn[SDN]Software Defined Networking
  su[SU]Secondary User
  ra[RA]Resource Allocation
  qos[QoS]quality of service
  usrp[USRP]Universal Software Radio Peripheral
  mno[MNO]Mobile Network Operator
  mnos[MNOs]Mobile Network Operators
  gsm[GSM]Global System for Mobile communications
  tdma[TDMA]Time-Division Multiple Access
  fdma[FDMA]Frequency-Division Multiple Access
  gprs[GPRS]General Packet Radio Service
  msc[MSC]Mobile Switching Centre
  bsc[BSC]Base Station Controller
  umts[UMTS]universal mobile telecommunications system
  Wcdma[WCDMA]Wide-band code division multiple access
  wcdma[WCDMA]wide-band code division multiple access
  cdma[CDMA]code division multiple access
  lte[LTE]Long Term Evolution
  papr[PAPR]peak-to-average power rating
  hn[HetNet]heterogeneous networks
  phy[PHY]physical layer
  mac[MAC]medium access control
  amc[AMC]adaptive modulation and coding
  mimo[MIMO]multiple input multiple output
  rats[RATs]radio access technologies
  vni[VNI]visual networking index
  rbs[RB]resource blocks
  rb[RB]resource block
  ue[UE]user equipment
  cqi[CQI]Channel Quality Indicator
  hd[HD]half-duplex
  fd[FD]full-duplex
  sic[SIC]self-interference cancellation
  si[SI]self-interference
  bs[BS]base station
  fbmc[FBMC]Filter Bank Multi-Carrier
  ufmc[UFMC]Universal Filtered Multi-Carrier
  scm[SCM]Single Carrier Modulation
  isi[ISI]inter-symbol interference
  ftn[FTN]Faster-Than-Nyquist
  m2m[M2M]machine-to-machine
  mtc[MTC]machine type communication
  mmw[mmWave]millimeter wave
  bf[BF]beamforming
  los[LOS]line-of-sight
  nlos[NLOS]non line-of-sight
  capex[CAPEX]capital expenditure
  opex[OPEX]operational expenditure
  ict[ICT]information and communications technology
  sp[SP]service providers
  inp[InP]infrastructure providers
  mvnp[MVNP]mobile virtual network provider
  mvno[MVNO]mobile virtual network operator
  nfv[NFV]network function virtualization
  vnfs[VNF]virtual network functions
  cran[C-RAN]Cloud Radio Access Network
  vran[V-RAN]Virtual Radio Access Network
  bbu[BBU]baseband unit
  bbus[BBU]baseband units
  rrh[RRH]remote radio head
  rrhs[RRH]Remote radio heads 
  sfv[SFV]sensor function virtualization
  wsn[WSN]wireless sensor networks 
  bio[BIO]Bristol is open
  vitro[VITRO]Virtualized dIstributed plaTfoRms of smart Objects
  os[OS]operating system
  www[WWW]world wide web
  iotvn[IoT-VN]IoT virtual network
  mems[MEMS]micro electro mechanical system
  mec[MEC]Mobile edge computing
  coap[CoAP]Constrained Application Protocol
  vsn[VSN]Virtual sensor network
  rest[REST]REpresentational State Transfer
  aoi[AoI]Age of Information
  lora[LoRa™]Long Range
  iot[IoT]Internet of Things
  snr[SNR]Signal-to-Noise Ratio
  cps[CPS]Cyber-Physical System
  uav[UAV]Unmanned Aerial Vehicle
  rfid[RFID]Radio-frequency identification
  lpwan[LPWAN]Low-Power Wide-Area Network
  lgfs[LGFS]Last Generated First Served
  wsn[WSN]wireless sensor network 
  lmmse[LMMSE]Linear Minimum Mean Square Error
  rl[RL]Reinforcement Learning
  nb-iot[NB-IoT]Narrowband IoT
  lorawan[LoRaWAN]Long Range Wide Area Network
  mdp[MDP]Markov Decision Process
  ann[ANN]Artificial Neural Network
  dqn[DQN]Deep Q-Network
  mse[MSE]Mean Square Error
  ml[ML]Machine Learning
  cpu[CPU]Central Processing Unit
  gpu[GPU]Graphics Processing Unit
  tpu[TPU]Tensor Processing Unit
  ddpg[DDPG]Deep Deterministic Policy Gradient
  ai[AI]Artificial Intelligence
  gp[GP]Gaussian Processes
  drl[DRL]Deep Reinforcement Learning
  mmse[MMSE]Minimum Mean Square Error
  fnn[FNN]Feedforward Neural Network
  eh[EH]Energy Harvesting
  wpt[WPT]Wireless Power Transfer
  dl[DL]Deep Learning
  yolo[YOLO]You Only Look Once
  mec[MEC]Mobile Edge Computing
  marl[MARL]Multi-Agent Reinforcement Learning
  aiot[AIoT]Artificial Intelligence of Things
  ru[RU]Radio Unit
  du[DU]Distributed Unit
  cu[CU]Central Unit
  api[API]Application Programming Interface
  cf[CF]Carbon Footprint
  ci[CI]Carbon Intensity
  oran[O-RAN]Open Radio Access Network
  flops[FLOPs]floating-point operations per cycle per core
  FLOPS[FLOPs]floating-point operations
  5g[5G]fifth-generation
  embb[eMBB]enhanced Mobile Broadband
  urllc[URLLC]Ultra Reliable Low Latency Communications
  mmtc[mMTC]massive Machine Type Communications
  hdd[HDD]hard disk drive
  ssd[SSD]solid state drive
  fspl[FSPL]free space path loss
  cav[CAV]Connected Autonomous Vehicle
  xr[XR]Extended Reality
  ap[AP]access point
  ble[BLE]Bluetooth low energy
  uwb[UWB]ultra wide band
  rss[RSS]received signal strength
  nn[NN]Neural Network
  v2x[V2X]vehicle-to-everything
  gai[GAI]Generative AI
  aiaas[AIaaS]AI as a service
  mlp[MLP]Multilayer Perceptron
  ble[BLE]Bluetooth Low Energy
  lorawan[LoRaWAN]Long Range Wide Area Network
  FPGAs[FPGAs]Field Programmable Gate Arrays
  llm[LLM]Large language model
  fl[FL]federated learning



 




§ ABSTRACT 
ai coupled with existing iot enables more streamlined and autonomous operations across various economic sectors. Consequently, the paradigm of aiot having ai techniques at its core implies additional energy and carbon costs that may become significant with more complex neural architectures. To better understand the energy and cf of some aiot components, very recent studies employ conventional metrics. However, these metrics are not designed to capture energy efficiency aspects of inference. In this paper, we propose a new metric, the Energy Cost of aiot Lifecycle (eCAL) to capture the overall energy cost of inference over the lifecycle of an aiot system. We devise a new methodology for determining eCAL of an aiot system by analyzing the complexity of data manipulation in individual components involved in the aiot lifecycle and derive the overall and per bit energy consumption. With eCAL we show that the better a model is and the more it is used, the more energy efficient an inference is. For an example aiot configuration, eCAL for making 100 inferences is 1.43 times higher than for 1000 inferences. We also evaluate the cf of the aiot system by calculating the equivalent CO_2 emissions based on the energy consumption and the ci across different countries. Using 2023 renewable data, our analysis reveals that deploying an aiot system in Germany results in emitting 4.62 times higher CO_2 than in Finland, due to latter using more low-ci energy sources.












AIoT lifecycle, Energy Consumption, Carbon Footprint, Metric, Methodology







§ INTRODUCTION



Next-generation networks aim to deliver unprecedented levels of connectivity and intelligence, facilitating seamless interactions between a massive number of devices, services, and applications. The trend is especially noticeable in iot where devices are becoming increasingly more intelligent <cit.>, from smart home appliances <cit.> to industrial machinery <cit.>, Smart City <cit.>, v2x <cit.>, and beyond <cit.>.
With the help of ai, we are moving away from conventional iot applications towards aiot systems <cit.> designed to achieve more streamlined and autonomous operations. For example, in future smart factories, the integration of ai and iot will enable real-time monitoring and optimization of production processes, predictive maintenance, and enhanced decision-making capabilities. 

While aiot systems enable unprecedented automation and agility, their reliance on ai implies additional energy and carbon costs <cit.>. Especially in the cases when  models with many parameters are employed, such as llms, such costs may be significant <cit.>. In light of climate challenges,  the general ai research community's effort to better estimate  the costs of training <cit.>, estimate the cost of ai <cit.>, the cfs of llms <cit.> and find ways to scale models efficiently <cit.> are intensifying. Furthermore, the networking and iot communities increasingly relying on ai techniques are also investigating pathways towards <cit.> net-zero carbon emissions, analyzing the cf of various learning techniques <cit.> and data modalities while also developing various approaches to optimizing energy efficiency through hardware-software optimization <cit.>, scheduling <cit.>, more efficient model design <cit.> and energy and carbon consumption testing <cit.>.


Depending on the scope of the study, various metrics are employed to understand energy and cf. For instance, <cit.> considers normalized energy as well as energy for buffering in [J], energy [Wh] and [CO_2eq] for the various phases involved in fl edge systems <cit.>, [J] and GFLOPS of the neural architectures <cit.> and energy [Wh] for the scheduled loads in  <cit.>. However, these metrics are not specifically designed to capture the energy efficiency of a system or part of. As well noted in <cit.>, the traditional ITU standardized Energy-per-Bit [J/b] metric 'will no longer be able to reflect the environmental impact of the modern mobile services, especially network AI-enabled smart services'. 

Recently, a number of environmentally friendly metrics, such as accuracy per consumption (APC), that capture the performance of a ai model together with its environmental impact <cit.>. Nevertheless we observe the following. First, while the metrics proposed in <cit.> are powerful in terms of assessing the energy/performance tradeoffs of models, these metrics are less suitable in assessing the energy efficiency of a system such as AIoT. Second, that a metric somewhat similar to the Energy-per-Bit [J/b] would be more suitable for measuring energy efficiency. Third, other efficiency metrics have been proposed in wireless in particular and engineering and economy in general <cit.>. For instance, spectral efficiency, measured in [b/s/Hz] <cit.> provides a means of calculating the amount of data bandwidth available in a given amount of spectrum, while lifecycle emissions for vehicles, measured in [g/km] <cit.>, enables computing the cf in grams per kilometer.


Following these observations, in this paper, we aim to find a suitable energy efficiency metric that would be simple, general and holistically quantify the environmental cost of inference carried out in AIoT systems. The contributions of this paper are as follows:



    
  * We propose a new metric, the Energy Cost of AIoT Lifecycle (eCAL), also measured in [J/b],  that captures the overall energy cost of generating an inference in an AIoT system. Unlike capturing the energy required for transmitting bits that is done by the Energy-per-Bit [J/b] metric, or [J] for ai model complexity, the proposed eCAL captures the energy consumed by all the data manipulation components in an AIoT system during the lifecycle of a trained model enabling inference. 
    
  * We devise a methodology for determining the  eCAL of an aiot system. The methodology breaks down the AIoT system into different data manipulation components, i.e. data collection, storage, and data preprocessing, training along with evaluation, and inference, and, for each component, it analyzes the complexity of data manipulation and derives the overall and per bit energy consumption. 
    
  * Using the proposed metric, we show that the better a model is and the more it is used, the more energy efficient an inference is. For an example aiot configuration, the energy consumption per bit for making 100 inferences is 1.33 times higher than for 1000 inferences. This can be explained by the fact that the energy cost of data collection, storage, training and evaluation, all necessary to develop an ai model are spread across more inferences.
    
  * We evaluate the cf of the aiot system by calculating the equivalent CO_2 emissions based on the energy consumption and the ci across different countries. Using 2023 renewable  data, our analysis reveals that deploying an aiot system in Germany results in emitting 4.62 times higher CO_2 than in Finland, due to the latter using more low-ci energy sources. 
    
    
    


The paper is structured as follows. We first review the relevant literature in Section <ref>, we then describe the lifecycle of an aiot system, define eCAL and the methodology for deriving it in Section <ref>. Subsequently, the derivation of the energy consumption formulas  for different data manipulation components involved in the aiot lifecycle are provided in Sections <ref> to <ref>. In Section <ref>, the proposed metric, eCAL, is derived and analyzed, demonstrating its utility in capturing the overall energy cost of AIoT systems over its lifecycle. whereas in Section <ref> we discuss the cf aspect of such a system, comparing emissions across different European countries. 
Finally, Section <ref> concludes the paper and outlines future research directions.







§ RELATED WORKS



While aiot systems enable unprecedented automation and agility, their reliance on ai implies additional energy and carbon costs <cit.> that in some cases that employ models with many parameters, such as llms, may be significant <cit.>. In light of climate challenges and the dazzling footprints of the latest generation of LLMs <cit.>,  the general ai research community's effort to better understand and then formalize methods to asses the environmental impact of the ai revolution is rising. Due to the complexity of state of the art neural architectures, in many cases the energy cost of training is computed after the training by measuring the performed computation through interfaces such as the performance application programming interface <cit.>. Furthermore, <cit.> shows that the energy footprint of inference is more studied than the one for training and highlights below 70% accuracy between predicted and measured energy consumption. 

However, a  number of approaches and tools capable of proactively estimating the energy and environmental cost of training have also emerged. LLMCarbon <cit.> is a very recent end-to-end cf projection model designed for both dense and mixture of experts LLMs.  It incorporates critical LLM, hardware, and data center parameters, such as LLM parameter count, hardware type, system power, chip area, and data center efficiency, to model both operational and embodied cfs of a LLM. Furthermore, <cit.> aims at providing guidelines for scaling ai in a sustainable way. They analyze the energy efficiency of new processing units, the cf of the most prominent LLM models since GPT-3 and analyze their lifecycle carbon impact showing that inference and training are comparable. They conclude that, to enable sustainability as a computer system design principle, better tools for carbon telemetry are required, large scale carbon datasets, carbon impact disclosure and more suitable carbon metrics.

The networking and iot communities, increasingly relying on ai techniques, are also investigating pathways towards <cit.> net-zero carbon emissions. The authors of <cit.> notice that in spite of improvements in hardware and software energy efficiency, the overall energy consumption of mobile networks continues to rise, exacerbated by the growing use of resource-intensive AI algorithms. They introduce a novel evaluation framework to analyze the lifecycle of network AI implementations, identifying major emission sources. They propose the Dynamic Energy Trading and Task Allocation framework, designed to optimize carbon emissions by reallocating renewable energy sources and distributing tasks more efficiently across the network. Similar as in <cit.>, they highlight the development of new metrics to quantify the environmental impact of new network services enabled by AI.  

The authors of <cit.> introduce a novel framework to quantify energy consumption and carbon emissions for vanilla FL methods and consensus-based decentralized approaches, identifying optimal operational points for sustainable FL designs. Two case studies are analyzed within 5G industry verticals: continual learning and reinforcement learning scenarios. Together with the authors of <cit.>, they consider 
energy [Wh] and [CO_2eq] in their evaluations. The solution proposed in <cit.> is a hardware/software co-design that introduces modality gating (throttling) to adaptively manage sensing and computing tasks. AMG features a novel decoupled modality sensor architecture that supports partial throttling of sensors, significantly reducing energy consumption while maintaining data flow across multimodal data: text, speech, images and video. More energy efficient task scheduling has been considered in  <cit.>, while more efficient neural architecture design and subsequent model development in <cit.>. The energy efficiency of various programming languages is studied in <cit.> while energy and carbon consumption testing for AI-driven IoT services is developed in <cit.>.

Novel metrics to evaluate dl models, considering not only their accuracy and speed but also their energy consumption and cost have been proposed in <cit.>. The  four metrics are: Accuracy Per Consumption (APC) and Accuracy Per Energy Cost (APEC) for inference, and Time To Closest APC (TTCAPC) and Time To Closest APEC (TTCAPEC) for training. These metrics aim to promote the use of energy-efficient and green energy-powered dl systems by integrating energy considerations into the performance evaluation. Nevertheless, while powerful in terms of assessing the energy/performance tradeoffs of models, these metrics are less suitable in assessing the energy efficiency of a system such as AIoT. The Energy-per-Bit [J/b] has been defined for assessing the efficiency of transmissions and therefore does not capture the ai aspects as also noted in <cit.>. Looking at the electric vehicle literature, we notice that, when comparing to traditional gas vehicles they employ lifecycle emissions as a metric, measured in [g/km] <cit.>. 

Inspired by the simplicity and effectiveness of metrics such as transmission efficiency through energy-per-bit and the lifecycle emissions, and responding to calls for new metrics <cit.> that capture the energy cost in the era of ai enabled systems, this work proposes the novel Energy Cost of AIoT Lifecycle (eCAL) metric and methodology to derive it.



§ ECAL DEFINITION AND METHODOLOGY



In our work, we consider an aiot communication and computing system whose lifecycle is  depicted in Fig. <ref>, consisting of the following data manipulating components: 


  
Data Collection 
This data-manipulating component, depicted in the lower part of Fig. <ref>, the collected data N_S, includes receiving telemetry data from sensors to the terminal computing infrastructure via wired or wireless technologies. The collected information can be turned into indicators such as signal strength, and for example, be interpreted by the ai models to predict the location of a user. To move this data from the sensor to the ap, E_T [J] is needed. 




  
Storage and Data Preprocessing 
Once the data from iot devices arrives at the computing infrastructure, it is first stored on the hard drive, and then fetched for preprocessing. Therefore, the energy consumption, E_storage, of this component arises from reading and writing data N_S into the storage units, which can vary depending on data volume and modality. 
To ensure accuracy and reliability during the training process, the data, N_S, must go through several preprocessing steps such as cleaning, feature engineering and transformation. The energy consumption for preprocessing E_pre, depends on the integrity of the ingested dataset. For example, datasets with a lot of invalid or missing data, require more extensive cleaning and error correction. 




  
Training and Evaluation 
The training component comprises the model development of the ai/ml using selected ai/ml techniques, such as neural architectures, and data. In this step of the model development process, the processed data, N_S,T is fetched and utilized to learn weights and biases in order to approximate the underlying distribution. For most neural architectures, the learning processes rely on gpu and tpu performing complex tensor processing operations and consuming E_train. Once the neural architecture weights are learned using the data in view of minimizing a loss function, the model is considered ready for evaluation and deployment. Subsequently, the quality of the learned model is evaluated on the evaluation datasets, N_S,E, a process that consumes E_eval energy. The data storage, preprocessing, training, and evaluation form the end-to-end training of the ai/ml model and cumulatively require E_D energy to complete.




  
Inference
Once the model is trained, it can be used by applications in inference mode. Various applications can send samples of data N_I,P and receive model outputs in the form of forecasts for regression problems or discrete (categorical) for classification problems. The energy consumption E_inf of a single inference is relatively small, however for high volumes of requests it can become significant.


Based on this lifecycle, we proposed eCAL as the ratio between all the energy consumed by all data manipulation components and all the manipulated application-level bits: 


    eCAL   =
       Total energy of data manipulation components  [J]/Total manipulated application level data  [b].


In the upcoming sections, we employ the following methodology in order to finally derive the proposed metric. We follow the telemetry data flow, highlighted with blue in Figure <ref>, across the above-mentioned components and derive the computational complexity and subsequently the absolute and per bit energy consumption, highlighted with green and orange respectively, in each component. For instance, in Section <ref>, we derive E_T and E_T,b for the data collection component while in Section <ref>, we do the same for preprocessing and storage E_pre and E_pre,b respectively. With this formalism in place, Section <ref> is concerned with deriving eCAL.

In addition to providing the first  theoretical framework for assessing and quantifying the energy consumption of each component of an aiot system in the data manipulation workflow, we also discuss and show differences across countries in view of aiot inference cf.



§ ENERGY COST OF DATA COLLECTION




In this section, we derive the energy cost of data collection in an aiot system. To calculate the cost, we first determine the number of bits that need to be transmitted from iot devices to the server via wireless technology. Then we calculate the energy consumption based on the given transmitting power and corresponding transmission rate. In the last step, we derive the total energy consumption for collecting data from iot devices. The energy consumption of wired communication compared to the wireless part is negligible <cit.>. Therefore, in this work, we only consider the communication from the iot device to the ap via wireless technologies.


Each wireless technology relies on different 
multi-layer protocol stack adding overhead bits, e.g., header, error correction, etc., to ensure collected data integrity and efficient transmission thus increasing energy consumption. Therefore, each packet is composed of payload, e.g., measurements collected by an iot device, and overhead, with overhead size depending on the selected technology. The size of the transmitted data in bits (B_T) can be expressed as defined in <cit.>:

    B_T [b]=α N_S_payload+⌈α N_S/F_U⌉·Ω_U_Overhead,



where α represents bit precision, i.e., the number of bits required to store value of a sample in floating number. Typically α equals 32 bits for single precision (float) or 64 bits for double precision. Moreover, we denote samples as 
N_S which represents the telemetry data that the application requests from the device in floating-point form, i.e., when an application requests 256 samples, the payload size can be calculated as 8192 and 16384 bits for single precision and double precision, respectively. The second part of the equation represents the overhead.  First, the number of packets is determined, by rounding up the payload divided with the maximal number of bits in 
 one packet (F_U) for the selected access technology. To obtain the total overhead the number of packets is then multiplied with the number of overhead bits per packet, that also depends on the selected access technology.



To illustrate the behavior of Eq. <ref>, we consider a scenario of a single device transmitting N_S=256 of double-precision samples to the application and alter the overhead percentages in one packet. More specifically, we consider a fixed packet size of 2000 bits. We consider overhead of 
1%, 30%, 50%, and 70%. 
Additionally, we consider overhead percentages of three mainstream wireless technologies <cit.> for iot systems, 
namely ble 5.0 <cit.>, ZigBee <cit.>, and lorawan <cit.>. In Table <ref>, we summarize the parameters of these three technologies based on iot protocol stack, in which the uplink data goes through the physical layer to the application layer. 
Fig. <ref> illustrates the relationship between the number of samples and the size of the transmitted data for different percentages of overhead. As seen in the figure, higher overhead percentages result in significantly more data being sent than when overhead is low. For example, when transferring 256 samples, the data transfer requirements for the scenarios Ω_U=1% F_U  and Ω_U=70% F_U are 16564 and 28984 bits, respectively, which means the latter has 1.7 times higher data transfer requirements than the former.















Thus, we can conclude that when the size of samples (N_S) is small, leveraging wireless technologies with lower overhead, such as ble, is advantageous. These technologies efficiently transmit fewer total bits. However, as the size of samples increases, the impact of overhead becomes less significant. In these scenarios, technologies with higher overhead but larger maximum packet sizes can become more beneficial. This shift emphasizes the importance of the trade-off between overhead, sample size, and maximum packet size, and highlights the importance of selecting the appropriate technology to achieve adequate energy efficiency in aiot deployments.










 
  
  
   



 





Next, we determine the energy consumption for a given technology, which we define as follows:

    E_T[J]=P_T[W]/R_T[b/s]· B_T[b].

where P_T represents the transmitting power. As a result, we can obtain energy consumption per bit for transmitting part of aiot system as follows: 

    E_T,b[J/b]=P_T[W]· T_T[s]/B_T[b]=P_T[W]/R_T[b/s] ,

where T_T is the corresponding transmitting time. T_T can be derived from the corresponding transmitting rate (R_T) and B_T, and is expressed as T_T=B_T/R_T.

Considering the previous example, if we transmitted the 256 samples with 1% overhead with P_T=10  [mW] and R_T=10^3 [b/s], the energy consumption [J] of 1% overhead would be 0.165, and 70% is 0.29. Moreover, the energy consumption per bit [J/b] for both 1% overhead and 70% overhead would be 10^-5. 

We can observe from the above equations that we are able to determine that when considering energy consumption per bit (E_T,b), this metric is exclusively related to P_T and R_T, which indicates that the energy required to transmit one bit of data depends on the power used for transmission and the speed of the transmission. Meaning, that energy consumption per bit is the inverse of energy efficiency <cit.>.








 We consider the wireless technologies listed in Table <ref> to illustrate per-bit energy consumption with the corresponding P_T and R_T and calculated E_T,b in Table <ref>. The comparison between E_T,b of different technologies is shown in Fig. <ref>. 
 It can be seen that ble exhibits the best performance due to its low power consumption and high transmission rate, while lorawan exhibits the highest energy consumption per bit despite its low overhead. This emphasizes the crucial role of transmission power and rate in determining the overall energy efficiency in aiot systems. Our results underscore the need to evaluate both power consumption and transmission rate, not just overhead, when selecting the communication technology for aiot deployments. Prioritizing lower power consumption and higher transmission rates can significantly improve energy consumption per bit.
 
 
 
 
Finally, the total energy consumption of the data transmission in an aiot system is illustrated in Fig. <ref>. We assume that the signals are transmitted at one-minute intervals and uniformly. These results showcase the total energy consumption over a period of 24 hours of continuous operation. ble 5.0 benefits from low power consumption, a high data rate, and low overhead, resulting in the lowest energy consumption per bit and very low overall energy consumption. Despite lower energy consumption per bit of ZigBee wireless technology, overall energy consumption rises with larger data sizes due to higher overhead percentage, whereas, lorawan has the highest energy consumption owing to its E_T,b is more dominate factor of the energy consumption. Moreover, these technologies excel in different aspects. For example, lorawan is optimal for long-range communication, whereas ZigBee is suitable for mesh networking. 
Consequently, both the total number of bits that devices transmit (B_T) and energy consumption per bit (E_T,b) play a role in determining energy cost of data transmission. Once the data reaches the ap is passed to the server for storage and preprocessing. 




§ ENERGY COST OF STORAGE AND DATA PREPROCESSING


Typically, data is first stored on a hard drive, subsequently preprocessed, and then used for the ml model development. In the following, we consider the dataset with N_S samples collected via wireless transmissions as depicted in Figure <ref> and discussed in Section <ref>. 




 §.§ Storage

The energy consumption of storage primarily depends on the type of hard drive. According to <cit.>, a typical hard drive consumes 0.65 Watt-Hour per Terabyte stored with a hdd-based technology. On the other hand, ssd-based hard drive consumes 1.2 Watt-Hour per Terabyte. For example, the energy consumption (E_storage [J]) of a dataset with N_S= 256 samples in the form of double-precision floating number consumes 4.79× 10^-6 and 8.85× 10^-6 Joules for hdd and ssd-based technology, respectively, whereas the corresponding energy consumption per bit (E_storage,b [J/b]) are 2.92× 10^-10 and 5.4× 10^-10. 


 §.§ Data preprocessing
















 We consider a data preprocessing approach consisting of data cleaning and data standardization steps, while we leave other more complex approaches such as feature engineering and transformations to future work. We assume that the data-cleaning process is performed in two steps. The first step is identifying and removing invalid samples from a list or array which  incurs 0 FLOP since no floating points additions and multiplications are performed, only comparisons and memory operations.
 The second step is to determine the complexity of data standardization (M_DS). We provide two examples, i.e., min-max scaling and normalization process, both representing a standard approach for data standardization.
 



  
Min-max scaling
In this method, which we detail in Algorithm <ref>, we first initialize two values for the minimum and maximum. Then, we iteratively search for minimum and maximum values by going through all the valid samples to update these values (lines <ref> to <ref>), which involve comparisons but no FLOP. After finding the minimum and maximum values, we calculate the range (line <ref>), which requires one FLOP. Finally, we scale the dataset (lines <ref> to <ref>) by performing a subtraction and division for each valid sample, costing 2 (N_S - N_NaN - 1) FLOPS. Therefore, the corresponding total number of FLOPS (M^minmax_DS) of min-max scaling can be expressed as: 

    M^minmax_DS = 2· (N_S-N_NaN)-1,

where N_NaN is the number of invalid samples.





  
Normalization
Normalization is performed in three steps, as described in Algorithm <ref>. The algorithm starts by calculating the mean value (lines <ref> to <ref>), followed by the calculation of the standard deviation (lines <ref> to <ref>). Finally, the algorithm iteratively normalizes each sample (lines <ref> to <ref>). Thus, the total operation (N^norm_DS) can be expressed as:

    M^normalization_DS = 6·(N_S - N_NaN - 1) + 3.









As a result, the total number of FLOPS for preprocessing can be expressed as:

    M_pre = 
    
        2· (N_DS - N_NaN) - 1,    if min-max scaling, 
    
        6·(N_DS - N_NaN) - 3,    if normalization.








We can observe that the computing complexity of both standardization methods decreases linearly as the number of invalid samples increases. This indicates that a higher number of invalid samples lowers the computing complexity, leading to reduced energy consumption. However, this also means that the application has to request additional transmissions from the device and perform extra operations to preprocess the data on the server.







 
 
 
  



















With the computing complexity in FLOPS formalized, we can obtain total energy consumption (E_pre) of the preprocessing (E_pre) by utilizing the power consumption of the processing unit and the executing time (T_pre). Thus, it is expressed as follows: 

    E_pre[J]= P_pre[W]· T_pre[s],

where T_pre can be calculated as the ratio between FLOPs needed for preprocessing and the computational power of a processor (N_PU). 

    T_pre[s] = M_pre[FLOPs]/M_PU[FLOPs/s].




 

Consequently, the energy consumption per bit for preprocessing can then be determined as: 

    E_pre,b[J/b]=E_pre[J]/α N_S[b].


 
 
  
   
    
     
    
    
    
    
    
    
    
    
    
    


For example, the E_pre and the corresponding E_pre,b for performing data preprocessing on a CPU with a performance of M_PU=10 GFLOPs/s and consume power of P_pre=140 W can be seen in the Fig. <ref> and Fig. <ref>, respectively. More specifically, Fig. <ref> shows that the energy consumption for both standardization methods decreases as the number of invalid samples increases, confirming that higher invalid sample rates lead to lower computational requirements. In addition, Fig. <ref> illustrates the energy consumption per bit for normalization and min-max scaling with N_S=256, where normalization consumes three times more energy per bit than min-max scaling. These results highlight the importance of selecting an appropriate preprocessing method based on both the energy efficiency requirements and AI/ML technique requirements. After processing the samples, the training process described in the next section can be carried out. 



 
  
   
   
  


 


















 
 






              



§ ENERGY COST OF TRAINING


While there are many ai/ml models available, in this work we focus on a fully connected mlp, as they form the foundation of all modern AI methods and point the reader to works such as <cit.> for a more comprehensive view on estimating energy consumption in machine learning. A filly connected MLP architecture is depicted in Fig. <ref> with the blue and red arrows illustrating the forward and backwards propagation taking place during the training process while the training is summarized in Algorithm <ref>. The architecture comprises of one input layer with N_I neurons, K hidden layers, in each layer it has M neurons and N_O neurons in the output layer. As shown in the Fig. <ref>, the training process includes two steps, namely, training and evaluation. Typically, input samples are split into certain ratios for training and evaluation. For example, with a 70/30 split, a dataset with 256 samples will have 179 samples for training, and 77 samples for evaluation. Therefore, we introduce β for such ratio. More specifically, the input samples can be expressed as:

    N_S=β N_S_N_S,T+(1-β)N_S_N_S,E,

where N_S,T and N_S,E represent the number of samples for training and evaluation, respectively. 

Model training of mlp involves two main phases as depicted in Fig. <ref> and Algorithm <ref>. First, the training samples undergo forward propagation where neurons compute linear combinations of weights and biases, apply activation functions, and calculate the loss between predicted and actual values. Subsequently, the model performs backward propagation where it calculates gradients of the loss with respect to each parameter, updates these gradients in reverse through the layers, and adjusts the weights and biases accordingly, which is described in Algorithm <ref> from line <ref> to line <ref>. This process completes one epoch. Consequently, we can calculate the energy costs associated with forward and backward propagation, leading to an understanding of the overall energy consumption for training, subsequently model evaluation, and inference.




 §.§ Computational Complexity of Forward and Backward Propagation

Assuming L layers where L=K+2, the total FLOPS for the forward propagation of a single input sample for an MLP can be expressed as: 

    M_FP=∑_l=1^L-1 (2 · (M_l-1· M_l)+2· M_l)















 
 
 
 


 
 
  
  



where L represents the overall number of layers in the MLP, the first term in the sum represents one  multiplication and one addition (=2 FLOPs) related to the weight and bias corresponding to an the edges times the number of edges on that layer (=M_l-1· M_l) while the second term represents the summation of all the values of the incoming edges and the application of the activation function (=2 FLOPs) times the number of nodes in that layer (M_l). As in our guiding example we assume all hidden layers having the same number of neurons, Eq. <ref> becomes M_FP=(2 · N_I· M + 2 · M) + (2 · (K-1)· M^2 + 2 · M) + (2 · MN_O + 2 · N_O) using the notations from Fig. <ref>. Considering an example mlp
with 3 hidden layers with 5 neurons on each hidden layer, 6 neurons on the input layer. Moreover, we aim to train the mlp model and utilize it to make predictions of three-dimensional coordinates based on the input data, thus, there will be 3 neurons on the output layer. Therefore, the total number of FLOPs can be calculated as M_FP= 2(6· 5+ 5) + 2(2· 25+5) + 2(5· 3+3) = 226. 



Training consists of multiple epochs and batches. Therefore, the total complexity of forward propagation is: 

    M_MLP,FP   = N_epochs· N_batch·batchsize· M_FP
       =N_epochs· N_S,T· M_FP.

Continuing with the example with 256 samples that goes through the aforementioned mlp model across 10 epochs, the computational complexity for forward propagation can be calculated M_MLP,FP=10· 256· 226=578560 FLOPS as also depicted in Fig. <ref>. When all hidden layers have the same number of neurons and their number M is increasing, it can be seen from Fig. <ref> 
that the FLOPs also increase quadratically. Adding more hidden layers K results in increases more than linearly due to the multiplicative nature of the connections between layers. As per Eq. <ref> and Fig. <ref>, the relationship between the increase in complexity and the augmentation of the number of samples and epochs is linear. These observations highlight the importance of considering the impact of architectural parameters on the computational complexity of MLP models. 



As we can see from Algorithm <ref> (line <ref> to line <ref>), the backward propagation consists of approximately twice the computational complexity of forward propagation, which is also confirmed in <cit.>. Thus, we can approximate the computing complexity of training a model as: 

    M_MLP≈  3M_MLP,FP.

As a result, we can compute the energy consumption for training, model evaluation, and the corresponding energy consumption per bit. 





 
  
  
  





    
    






 §.§ Energy Cost of Training

We can obtain the energy consumption of the entire training process as:

    E_train=3M_MLP,FP[FLOPs]/PU_performance[FLOPs/s/W],

where PU_performance represents the theoretical peak performance of a processing unit. Moreover, the corresponding energy consumption per bit can be obtained from Eq. <ref> and the number of input sample, thus, we can show it as: 

    E_train,b=3M_FP/α PU_performance.







 §.§ Energy Cost of Model Evaluation

The model evaluation starts once training is complete. This functionality tests the performance of the model on a separate dataset. During the evaluation, the model processes the data using forward propagation without any adjustments to its parameters, which can be seen from lines <ref> to <ref> in Algorithm <ref>. Therefore, the energy consumption and energy consumption of model evaluation is expressed as:

    E_eval=M_FP· N_S,E/PU_performance.

By utilizing the same technique as in Eq. <ref>, the energy consumption per bit of evaluating the model can be expressed as: 

    E_eval,b = M_FP/α PU_performance.
 


§ ENERGY COST OF INFERENCE


The complexity of making an inference depends on the number of FLOPS required for forward propagating with input sample size N_I,P, i.e., N_inf=M_FP· N_I,P. For example, when we adopt the aforementioned MLP model with N_I,P=77, the complexity is calculated as N_inf=226· 77 = 17402 FLOPS. Therefore, the energy consumption of the forward propagation and the corresponding inference can be calculated as: 

    E_inf = M_FP· N_I,P/PU_performance.
 
We can then observe from Eq. <ref> and  <ref>, the cost per bit of inference is the same as in evaluation. Thus, we can conclude that the computational complexity between finishing one training and inference is on the magnitude of 3N_epochsN_S,T/N_I,P. This indicates that the training process involves significantly more computational operations, particularly when the number of epochs or 
N_S,T>>N_I,P. On the other hand, the energy consumption per bit depends on the model complexity and hardware capability. Moreover, the factor of three increases in the energy consumption per bit during training versus inference highlights the additional computational load inherent to training. Finally, the calculation of end-to-end energy consumption of an aiot system is presented in the next section.  







§ ECAL: THE ENERGY COST OF AIOT LIFECYCLE


In this section, we describe the end-to-end energy consumption of an aiot system for making inferences with a mlp model. First, we consider the energy consumption of developing the model illustrated on the left hand side of  Fig. <ref> as follows:

    E_D=E_T+E_storage+E_pre+E_train+E_eval,

and the energy consumption per bit that can be  expressed as: 

    E_D,b=E_D/B_T+α (2N_S+N_S,T+N_S,E).


To compare the energy consumption of all the data manipulation components required to develop and deploy an aiot system, we continue with the guiding example from the paper where the ai model relies on an an MLP with 3 hidden layers (5 neurons each), whereas on the input layer and output layer there are 6, 3 neurons, respectively, and it requests 256 samples from a iot device via ble to train and evaluate the model with a 70/30 split as discussed in Section <ref>. In addition, we utilize hdd for storage and normalization as data standardization as discussed in Section <ref>. The energy consumption for this example is shown in Fig. <ref> and reveals that the total energy consumed in this scenario is 0.0264J. Moreover, the energy consumption of a training session is equal to 141 times collecting the same amount of samples from the iot device, or 71 times of inference. However, these numbers can be significantly higher if the complexity of the model increases or the size of input samples for training is significantly larger than that used for inference as the respective analyses in Sections <ref> and <ref> showed.

Once the model is developed, it is packaged and deployed in the aiot system and it can produce inference in the form of continuous or discrete outputs. During its operational lifecycle, the model will be presented some input data and requested to produce the corresponding inference, thus we can express the associated energy cost as:

    E_inf,p=E_T+E_storage+E_pre+E_inf,

it means that we consider the energy cost of inference before the current model needs to be updated. The corresponding energy per bit of inference once deployed is calculated as: 

    E_inf,p,b=E_inf,p/B_T+3α N_I,P.


Next, the entire energy cost over the lifetime of a model in aiot the energy consumed for the model development, E_D is summed with the energy required for performing an inference E_inf,p times the number of times γ the inference is performed with the currently deployed model, and γ∈[1, ∞), also referred to as eCAL absolute and measured in [J] as: 

    eCAL_abs=E_D+γ E_inf,p.


Moreover, the averaged end-to-end energy consumption for one inference can be expressed as:

    eCAL_abs=eCAL_abs/γ.

As we observe from Eq. <ref>, as the number of inferences increases, the average energy consumption decreases and eventually approaches E_inf,p, which is depicted in the Fig. <ref>. This finding can also be utilized to measure how well the model is resilient toward performance loss from an energy consumption point of view. More specifically, with the same hardware setup, a model that requires less frequent retraining due to performance degradation will consume less energy than one that needs frequent retraining.



Finally, the proposed metric computing the corresponding energy consumption per bit over the lifecycle of the aiot with the currently deployed model as:

    eCAL   =
       eCAL_abs/B_T+α (2N_S+N_S,T+N_S,E)+ γ (B_T+3α N_I,P).

To further quantify this relationship, we first compare between Eq. <ref>, Eq. <ref>, and Eq. <ref>, which is depicted in Fig. <ref>. It shows that the energy consumption per bit of developing the model is 2.59× 10^-7[J/b], which has approximately 26 times higher than the operational lifecycle (9.89× 10^-9[J/b]). Furthermore, we can observe from the figure that the energy consumption per bit decreases with more inferences, dropping from 1.49× 10^-8 [J/b] for 100 inferences to 1.04× 10^-8 [J/b] for 1000 inferences which is 1.43 times improvement. It is also confirmed in Fig.<ref>, where it follows a similar trend as in Fig.<ref>. More specifically, as the number of the inferences in the current model increases, the energy efficiency of the system also increases. 












 
 
 
  








§ CARBON FOOTPRINT OF AIOT INFERENCE


The cf is defined as the product of energy consumption and ci, where ci indicates the amount of carbon dioxide equivalents (CO_2eq) emissions associated with generating electrical energy, accounting for all greenhouse gases by weighting each gas by its Global Warming Potential (GWP). This relationship is expressed as:

    CF[gCO_2eq] = E[kWh] · CI [gCO_2eq/kWh].



Each country has its own ci, which varies depending on its energy production methods. For example, in 2023, Finland derived 40% of its electricity from hydro and wind sources, with only 10% originating from high ci sources such as coal or gas. This indicates a predominantly low-carbon energy grid. In contrast, Germany, despite utilizing 44% renewable energy, still recorded a relatively high ci <cit.>. This is attributed to the fact that 38% of its electricity production relied on burning gas, oil, and coal, which are substantial contributors to greenhouse gas emissions. This is shown in Table <ref>, which provides a summary of the average ci values for different countries in 2023, along with the corresponding cf for end-to-end training and inference in an aiot system. The energy consumption is calculated by utilizing the aforementioned setup, along with Eq. <ref> and Eq. <ref>. We can see that the table reflects the the significant impact of ci on the cf. For example, deploying the same model in Germany results in greenhouse gas emissions that are 4.62 times higher than those in Finland. This highlights the direct correlation between the ci of a country and the environmental cost of technological operations, especially with the integration of ai into the wireless communication systems. 

Although the cf for both end-to-end training and inference phases are relatively low, it is important to note that these figures pertain to less complex models and energy-efficient transmission methods. In more realistic scenarios, where models may be more complex and energy demands higher, the cf can substantially increase.





 

Additionally, by considering the total energy consumption in Eq. <ref>. We can determine how much CO_2 is emitted into the atmosphere in different locations as the number of inferences increases, this is shown in Fig. <ref>.

The results highlight the significant variation in the cf due to differences in average ci among countries. For example, Finland, benefiting from its lowest average ci, exhibits the lowest CO_2 emissions among the five countries analyzed. Therefore, by leveraging lower ci methods to power the system, such as wind or hydro, we can move towards a more sustainable aiot system.













§ CONCLUSION AND FUTURE WORK


In this work, we proposed a novel metric, namely, eCAL, unlike traditional metrics that focus only on the energy required for transmitting or ai models. It can capture the overall energy cost of generating an inference in an aiot system during the whole lifecylce of a trained model. We propose a detailed methodology to determine the eCAL of an aiot system by breaking it down into various data manipulation components, such as data collection, storage, preprocessing, training, evaluation, and inference, and analyzing the complexity and energy consumption of each component. Our proposed metric demonstrates that the more a model is utilized, the more energy-efficient each inference becomes. For example, considering a simple MLP architecture, the energy consumption per bit for 100 inferences is 1.43 times higher than for 1000 inferences. Moreover, we assess the cf of the aiot system by calculating the equivalent CO_2 emissions based on energy consumption and ci across different countries. Our analysis, using 2023 renewable data, shows that cf varies deploying an aiot system in different geographic locations, indicating the importance of considering regional energy profiles and their associated cis in the deployment of AIoT systems. 

Through the proposed eCAL metric, this study provides foundations for understanding the energy consumption of aiot systems. The assumptions are that the model is based on MLPs which are a type of feed-forward neural architectures. Future work may focus on removing that assumption and extend the study to more complex architectures such as recurrent, convolutional, recursive, attention based, etc. and provide a relative complexity comparison of these categories. Furthermore, as virtualization permeates 6G networks, the impact of network slices on the data collection part could be derived. Finally,  the development of  dynamic energy management algorithms to optimize the cf in view of a more sustainable operation is relevant. 








§ ACKNOWLEDGEMENTS


This work was supported in part by the HORIZON-MSCA-IF project TimeSmart (No. 101063721), the European Commission NANCY project (No.101096456), and by the 
Slovenian Research Agency under grants P2-0016 and J2-50071.











./templates/IEEEtran




 Paper Title: Alpha-VI DeepONet: A prior-robust variational Bayesian approach for enhancing DeepONets with uncertainty quantification.


Can Developers Prompt? A Controlled Experiment for Code Documentation Generation
    

Hans-Alexander Kruse 0009-0006-5568-7841,
Tim Puhlfürß 0000-0001-8421-8071,
Walid Maalej 0000-0002-6899-4393


Universität Hamburg
  
Hamburg, Germany

hans-alexander.kruse@studium.uni-hamburg.de,
tim.puhlfuerss@uni-hamburg.de,
walid.maalej@uni-hamburg.de

    August 5, 2024
======================================================================================================================================================================================================================================================================



        
§ ABSTRACT 
We introduce a novel deep operator network (DeepONet) framework that incorporates generalised variational inference (GVI) using Rényi's α-divergence to learn complex operators while quantifying uncertainty. By incorporating Bayesian neural networks as the building blocks for the branch and trunk networks, our framework endows DeepONet with uncertainty quantification. The use of Rényi's α-divergence, instead of the Kullback-Leibler divergence (KLD), commonly used in standard variational inference, mitigates issues related to prior misspecification that are prevalent in Variational Bayesian DeepONets. This approach offers enhanced flexibility and robustness.

We demonstrate that modifying the variational objective function yields superior results in terms of minimising the mean squared error and improving the negative log-likelihood on the test set. Our framework's efficacy is validated across various mechanical systems, where it outperforms both deterministic and standard KLD-based VI DeepONets in predictive accuracy and uncertainty quantification. The hyperparameter α, which controls the degree of robustness, can be tuned to optimise performance for specific problems.

We apply this approach to a range of mechanics problems, including gravity pendulum, advection-diffusion, and diffusion-reaction systems. Our findings underscore the potential of α-VI DeepONet to advance the field of data-driven operator learning and its applications in engineering and scientific domains.









§ INTRODUCTION
 

As scientific machine learning methodologies take centre stage across diverse disciplines in science and engineering, there is an increased interest in adopting data-driven methods to analyse, emulate, and optimise complex physical systems. The behaviour of such systems is often described by laws expressed as systems of ordinary differential equations (ODEs) and partial differential equations (PDEs) <cit.>. A classical task then involves the use of analytical or computational tools to solve such equations across a range of scenarios, e.g., different domain geometries, input parameters, and initial and boundary conditions (IBCs). Solving these so-called parametric PDE problems requires learning the solution operator that maps variable input entities to the corresponding solution of the underlying PDE system. Tackling this task using traditional tools (e.g., finite element methods <cit.>) bears a formidable cost, as independent simulations need to be performed for every different domain geometry, input parameters, or IBCs. Driven by this challenge, a growing field of neural operators for solving parametric PDEs has come forth recently. 

Learning to solve PDEs is closely related to operator learning. Instead of learning to solve a specific PDE, it can be advantageous to learn the operator that maps a functional parameter of the PDE (such as initial values, boundary conditions, force fields, or material parameters) to the solution associated with the given parameter. Neural operators exemplify this approach by learning to solve entire classes of PDEs simultaneously. These specially designed deep neural networks can represent the solution map of parametric PDEs in a discretisation-invariant manner, meaning the model can be queried at any arbitrary output location. Unlike neural networks that map between two finite-dimensional vector spaces, neural operators can represent mappings between spaces with infinite dimensions <cit.>. Popular operator learning frameworks include deep operator networks (DeepONets) <cit.> and Fourier neural operator (FNO) <cit.>. This article focuses on DeepONet, which has been applied to a wide range of problems, including solid and fluid mechanics <cit.>, reliability analysis <cit.>, heat transfer <cit.>, and fracture mechanics <cit.>. De et al. <cit.> utilised DeepONets for modelling uncertain and partially unknown systems. Cai et al. <cit.> applied DeepONets to model field variables across multiple scales in multi-physics problems while He et al. <cit.> employed DeepONets to predict full-field, nonlinear elastic–plastic stress responses in complex geometries. Other notable applications of DeepONets include weather forecasting <cit.>, reduced-order modelling <cit.>, finance <cit.>, and others <cit.>.

While DeepONets excel at solving differential equations (DEs), a crucial aspect often overlooked is the uncertainty quantification of the predictions.  Traditional DeepONet architectures have been found to produce over-confident predictions, implying they are poorly calibrated <cit.>. This means they might underestimate the true range of possible outcomes, potentially leading to unreliable results. Additionally, they do not quantify the inherent uncertainties associated with their predictions.
In engineering applications like aerodynamics or structural analysis, uncertainty quantification is paramount. By quantifying uncertainties in predictions related to phenomena such as stress distribution, engineers can make informed decisions about the risk and reliability of their designs. 

Several attempts have been made to incorporate uncertainty quantification within the DeepONet architecture. However, existing approaches face some limitations. Lin et al. <cit.> proposed a Bayesian DeepONet based on replica-exchange stochastic gradient Langevin diffusion <cit.> that considers the standard deviation of the output to be known, which may not always be realistic. Additionally, training the replicas in this approach is computationally expensive. Yang et al. <cit.> used randomised priors and trained an ensemble of deterministic models, which can also be computationally demanding depending upon the size of the ensemble. While the method of ensembles offers some degree of uncertainty estimation through multiple model predictions, it does not adhere to the more principled framework that is provided by Bayes' rule and hence cannot be considered truly Bayesian. 

Variational inference (VI)-based Bayesian methods offer a promising framework for incorporating uncertainty quantification into deep learning models. This approach has given rise to Bayesian neural networks (BNNs) <cit.>. In BNNs, the parameters of the neural network are treated as random variables, which allow the uncertainty in the parameters to propagate through the network, ultimately reflecting the uncertainty in the model predictions. The variational inference approach is particularly advantageous for neural networks with a large number of parameters, as traditional Markov chain Monte Carlo (MCMC)-based methods become computationally expensive to use. Variational inference, on the other hand, provides a scalable alternative for approximating the posterior distribution of the parameters in BNN. 

It seems natural to extend the idea of uncertainty quantification in BNNs to that of a Bayesian DeepONet. Building on this idea, Garg et al. <cit.> proposed VB-DeepONet that uses BNNs as building blocks for the DeepONet architecture. However, VB-DeepONet inherits certain challenges associated with BNNs. Their approach employs a fully factorised standard Gaussian distribution as the prior over the DeepONet model parameters. Such a prior might not accurately reflect the true distribution of these parameters, leading to a sub-optimal approximation of the posterior distribution. As indicated in Li et al. <cit.>, Knoblauch et al. <cit.>, and Wenzel et al. <cit.>, such a misspecified prior can hinder the overall effectiveness of variational inference, where the approximated posterior concentrates around a single point leading to overconfident predictions and underestimated uncertainty <cit.>. This hinders the model's ability to effectively quantify uncertainties. 

We propose to address these limitations in VB-DeepONet by introducing prior-robust variational inference for DeepONets. Standard VI relies on the Kullback-Leibler divergence (KLD), which is sensitive to prior selection, especially when the number of parameters is large. Alternative divergence measures exist that offer robustness to misspecified priors. Knoblauch et al. <cit.> introduced the Generalised Variational Inference (GVI) framework that allows for defining divergence metrics beyond KLD in the context of BNNs. GVI incorporates a hyperparameter that controls the degree of robustness to prior misspecification. This leads to posteriors that are less influenced by priors which deviate significantly from the observed data <cit.>. 

In this paper, we extend the GVI framework to DeepONets to achieve greater robustness to prior misspecification. In particular, we propose the use of Rényi's α-divergence <cit.> as a robust alternative to KLD within the VI framework. This novel operator learning approach using GVI leads to an improved approximated posterior distribution, demonstrably enhancing both prediction accuracy and the quality of uncertainty estimates as evidenced by log-likelihood values on unseen test data.

The rest of the article will provide background on deterministic DeepONets, followed by details of the proposed variational Bayesian framework and four numerical examples highlighting the efficacy of our approach. We conclude by discussing the results of different hyperparameter settings and recovering standard KLD-VI results as a special case.



§ BACKGROUND ON DEEPONETS

Operator networks were first introduced by Chen and Chen <cit.>, where they considered shallow networks with a single hidden layer. Subsequently, this concept was significantly developed with deep neural network architectures, leading to the creation of DeepONets <cit.>. To define operators, it is useful to consider two different classes of functions residing in two separate Banach spaces, 𝒜 and 𝒮, respectively. One class of functions, a(y) ∈𝒜, with domain in Ω_Y ∈ℝ^D, while the other class of functions, s(y) ∈𝒮, also having domain in Ω_Y. An operator 𝒢 can be defined as a mapping between these two function spaces, 𝒢: 𝒜→𝒮 such that s(y) = 𝒢(a(y)).

DeepONet is a specialised deep neural network designed to approximate the operator 𝒢. In the following sections, we review the DeepONet and its variational Bayesian extension (VB-DeepONet), and discuss their limitations in the context of uncertainty quantification.



 §.§ Deterministic DeepONets

Consider a governing equation for a physical phenomenon within a spatial domain Ω over a time period (0, T]:
 
    
    ℱ(s)(x, t)    = u(x,t),  x, t∈Ω× (0,T], 
    
            ℬ(s)(x,t)    = s_b(x,t),  x, t∈∂Ω× (0,T],
    
            ℐ(s)(x, 0)    = s_0(x),   x ∈Ω.


Here, ℱ is a nonlinear differential operator, x is the location in space, t is the time; s denotes the solution of the differential equation, and u denotes the external source function. Furthermore, ℬ(s)(x,t) = s_b(x) and ℐ(s)(x, 0) = s_0(x) define the boundary and initial conditions, respectively. The true solution operator is denoted by 𝒢(u)(x,t). To keep the notations compact, we denote the tuple (x,t) by y. 

DeepONets <cit.> are designed to learn an approximate operator 𝒢_θ parameterised by a deep neural network with parameter vector θ. 

The architecture for the DeepONet consists of two main sub-networks: a branch neural network and a trunk neural network (see <ref>). 

    
  * Branch Network: This is a neural network, with trainable parameters θ_B, that maps from space ℝ^M →ℝ^P. It takes as input M number of sensor measurements of the function a(y) ∈𝒜, represented by the M-dimensional vector a = ay_1, …, ay_M (where y_k x_r, t_v denotes some location x_r and some time instant t_v), and it outputs a vector of weights b_1(a), …, b_P(a).

    
  * Trunk Network: This is a neural network, with trainable parameters, θ_T, that takes as input y∈Ω_Y and outputs a P-dimensional basis vector ψ_1(y), …, ψ_P(y).
  
The final output of the DeepONet 𝒢_θ: ℝ^M ×ℝ^D →ℝ is obtained by taking a dot product of the outputs from the branch and trunk networks, which approximates the value of s(y):

    s(y) ≈ s(y; θ) =𝒢_θ(a)(y) ≈ b_0 + ∑_p=1^P b_pa; θ_B ψ_p y; θ_T.

The trainable parameters of the DeepONet θ consist of the bias b_0 and the combined parameters of the branch and trunk networks, i.e., θ = b_0, θ_B, θ_T.

Training a DeepONet involves supervised learning using pairs of the representative function a(y) and the corresponding solution s(y). A representative training set looks like:

    𝒟 = a^(i), y_k, s^(i)y_k: 1 ≤ i ≤ N_1, 1 ≤ k ≤ N_2

where a^(i) is the i^th training example of the  M-dimensional vector a obtained at M locations of y, and s^(i) is the corresponding solution function obtained at N_2 locations of y from solving the PDE (<ref>). Note that the N_2 locations may differ from M sensor locations where the representative input functions a are measured. The M sensor locations can be either random or uniformly spaced and remain fixed during training. 

The parameter vector θ is estimated by solving the optimisation problem over the training dataset 𝒟:

    θ̂   = _θ1/N_1 N_2∑_i=1^N_1∑_k=1^N_2𝒢_θa^(i)(y_k)  -  s^(i)y_k^2 
        = 1/N_1 N_2∑_i=1^N_1∑_k=1^N_2b_0 + ∑_p=1^P b_p a^(i); θ_B ψ_p y_k; θ_T  -  s^(i)y_k^2.

The optimisation problem can be solved using stochastic gradient descent <cit.> to obtain the optimal parameter values for the DeepONet. This approach is referred to as D-DeepONet in the rest of the article, where “D” stands for deterministic. 

A key limitation of D-DeepONet is its inability to quantify uncertainty in the predictions. As it produces a point estimate, it fails to account for potential measurement errors or uncertainties in the model parameters. 



 §.§ Variational Bayesian DeepONets (VB-DeepONet)

VB-DeepONet <cit.> addresses the limitations of D-DeepONet by incorporating layers of Bayesian neural networks within both the branch and trunk networks of the DeepONet architecture. This integration allows the model to capture prediction uncertainties, enhancing its robustness against overfitting.

However, VB-DeepONet faces challenges related to posterior inference using standard variational inference. A common practice in VB-DeepONet is to employ a fully factorised standard Gaussian distribution as the prior for the DeepONet model parameter vector θ. This prior assumes pairwise independence among model parameters, implying that each parameter distribution is unimodal. Although intended to be non-informative, this choice of prior can still exert significant influence on the posterior distribution given the number of parameters of deep neural networks can be quite large. Hence, this prior might not accurately reflect the true distribution of the parameters, potentially leading to sub-optimal uncertainty quantification. Moreover, standard KLD-VI with mean-field Gaussian variational families often exhibit “mode-seeking” behaviour, where the approximated posterior tends to concentrate around a single point. Additionally, if the model parameters exhibit high correlations, these VI approximations can produce overly confident predictions, failing to capture the true range of uncertainty. These issues can significantly hinder the VB-DeepONet's ability to effectively quantify uncertainties, a crucial aspect for reliable engineering applications.

The next section discusses the proposed α-VI DeepONet approach that uses an alternative divergence measure, the Rényi's-α divergence, to address the limitations of VI in VB-DeepONet and achieve robust uncertainty quantification.



§ PROPOSED Α-VI DEEPONET
 
We propose using Bayesian neural networks in both the trunk and branch networks (shown in <ref>) instead of their deterministic counterparts, similar to VB-DeepONet. However, a fully factorised normal distribution is hardly ideal as a prior for BNNs. Constructing alternative prior beliefs that accurately reflect our judgments could be computationally prohibitive too. Nonetheless, by employing an alternative divergence metric (D), more robust posterior beliefs can be produced with an imperfect prior. Therefore, in the proposed approach, we seek to use a divergence metric, different from KLD, that results in posteriors more robust to poorly specified priors and provides reliable uncertainty quantification. 

In the deterministic DeepONet, the parameter vector θ of the deep neural network (from <ref>) is a point estimate. However, as noted in <cit.>, multiple different parameter settings can perform equally well, implying that the set of values that each parameter can take may be captured by a distribution over those plausible values. A Bayesian paradigm allows us to place distributions on the model parameters, representing uncertainty about their exact values. Using Bayes' rule, we update these parameter distributions with the data (𝒟 from <ref>) as follows:

    p(θ | 𝒟) = p(𝒟 | θ)  p(θ)/p(𝒟),

where p(𝒟 | θ) is the likelihood and p(θ) is the prior distribution over the model parameter vector. The denominator, p(𝒟) = ∫ p(𝒟| θ)  p(θ)  dθ, is often an intractable constant known as marginal likelihood. The posterior distribution,
p(θ | 𝒟 ), allows us to perform inference on unseen data (𝒟^*). For prediction, we use this posterior to compute the predictive distribution:

    p(𝒟^* | 𝒟, θ) = ∫ p(𝒟^* | θ) p(θ | 𝒟) dθ.

We now examine these individual components representing our modelling choices that influence the learned posterior distribution. 



 §.§ Fully Factorised Prior

In BNNs, we often default to a fully factorised standard normal distribution as a prior over the model parameters. This implies that each parameter component is assumed to be independent and drawn from a Gaussian distribution (denoted by 𝒩) with zero mean and unit variance. Mathematically, this can be expressed as:

    p(θ) = ∏_l=1^L p(θ_l) = ∏_l=1^L𝒩(θ_l; 0,1) = 𝒩 (θ; , _L),

where L denotes the dimensionality of the model parameter vector θ. It is important to acknowledge that this fully factorised prior is a “misspecified prior” as it does not capture the interdependencies that exist between the individual components θ_l of the parameter vector θ. 
To alleviate the negative influence of this “misspecified prior,” we use a different divergence metric in <ref>.






 §.§ Likelihood Function

The likelihood function describes the probability of observing the solutions given the model parameters (and deterministic input functions). We write the likelihood function considering that each example i is independent and identically distributed, given the input functions and parameters, as follows:

    p(𝒟 | θ) = ∏_i=1^N_1 ps^(i)(y_1), …, s^(i)(y_N_2) |a^(i), θ,

where N_1 represents the total number of examples of pairs a and s.
We further assume that the outputs s^(i)(y_k) at different locations are also statistically independent of each other k = 1,…, N_2. Consequently, the joint distribution of the solution for the i^th example across all N_2 locations factorises into a product of independent marginal distributions for each location:

    p(𝒟 | θ) = ∏_i=1^N_1∏_k=1^N_2 ps^(i)(y_k) |a^(i), θ.

Furthermore, we assume that the individual output distributions ps^(i)(y_k) |a^(i), θ at each location follow a Gaussian distribution. The mean and standard deviation of these distributions depend on the input function a, location y, and model parameters θ. This dependence reflects the influence of the input function and location on the solution value, mediated by the model parameters. Mathematically, this is expressed as:

    p(𝒟 | θ)    = ∏_i=1^N_1∏_k=1^N_2𝒩s^(i)(y_k);  μ_ka^(i), y_k, θ, σ_ka^(i), y_k, θ
    
           = ∏_i=1^N_1∏_k=1^N_2𝒩s_k^(i);  μ_k(θ) , σ_k (θ),

where μ_k represents the mean and σ_k represents the standard deviation. 
For notational convenience, we use the shorthand notation s_k^(i)≡ s^(i)(y_k) to represent the solution value at location y_k for the i^th solution data. Additionally, we only highlight the dependence of the mean and standard deviation on the model parameters θ for brevity. The other variables, a^(i) and y_k, are treated as deterministic (and known) quantities in this context.

A sample prediction from this network represents a stochastic pass through the random BNN parameters, followed by a sample from the Gaussian likelihood. A number of these samples are taken to quantify the total uncertainty and construct the confidence intervals.



 §.§ Posterior Approximation via Variational Inference

Within a fully Bayesian framework, our goal is to determine the posterior distribution of the model parameters given the observed data. This posterior distribution, denoted as p(θ | 𝒟), represents the probability of various parameter configurations (θ) after observing the data 𝒟.
Unfortunately, due to the intractable normalising constant as explained in <ref>, directly computing the posterior distribution analytically is not feasible. 

Generally, there are two approaches to solve this class of problems - sampling-based approaches, e.g., Markov chain Monte Carlo and approximation-based optimisation approaches, e.g., variational inference, expectation propagation (EP). 
Sampling-based approaches, exemplified by MCMC-techniques, are often regarded as the gold standard for Bayesian inference <cit.>. They involve constructing a Markov chain that gradually explores the parameter space, eventually converging to a distribution proportional to the posterior. However, for problems with high-dimensional parameter spaces (often exceeding 20,000 parameters in our case), these approaches can become computationally impractical <cit.>. We note here that the proposed approach puts no limit on the complexity of the model, that is the number of parameters. Hence, we will focus on the approximation-based approaches, where a simpler variational distribution is introduced to approximate the posterior, and the key lies in optimising the parameters of the variational distribution to minimise the Kullback-Liebler divergence between itself and the intractable posterior. This optimisation process allows us to efficiently train complex models despite the high dimensionality of the parameter space.



  §.§.§ Variational Inference with KLD

VI <cit.> introduces a surrogate posterior family of distributions, denoted by Q, that is tractable to sample from. The goal is to find a member of Q distribution that closely resembles the true posterior in terms of the shape. VI accomplishes this by minimising the KLD between the approximate posterior q(θ;η) ∈ Q and the true posterior p(θ|𝒟). The KLD, denoted as D_KL[ q || p], quantifies the difference between the q distribution and the reference p distribution weighted by the q distribution, and is mathematically defined as:

    D_KLq(θ) ||  p(θ) = ∫ q(θ)logq(θ)/p(θ)dθ = 𝔼_q(θ)[ log(q(θ)) - log(p(θ)) ].
 
Using the KLD, the minimisation problem in VI is formulated as finding the best parameters η^* of the variational distribution that brings it closer to the true posterior:

    q(θ; η^*) = _q(θ;η) ∈ Q D_KL q(θ; η) || p(θ|𝒟) ,

where η are the parameters of the variational distribution, q(θ; η). Here, the KLD measures the information lost by approximating the true posterior with the variational distribution q. However, directly computing this integral is impractical because the true posterior is unknown. VI addresses this by leveraging the connection between the KLD and the evidence lower bound (). 
VI rewrites the KLD using the following identity:

    D_KL q(θ; η) || p(θ | 𝒟)    = 𝔼_q(θ;η)log(q(θ; η)) - logp(𝒟 | θ) p(θ)/p(𝒟)
    
        =log p(𝒟) - 𝔼_q(θ;η)log(p(𝒟 | θ)) - D_KLq(θ; η) || p(θ)_
    
       = log p(𝒟) + ℒ(q) ,

where the expectation in the  (in <ref>) is taken with respect to the approximate posterior q(θ; η). Note that p(𝒟) is the model evidence, which is a constant term for a given dataset and also constant with respect to q(θ;η) and can therefore be ignored during the minimisation process <ref>. 
From <ref>, we see that maximising the  is equivalent to minimising the KLD (as KLD is non-negative). So, VI aims to maximise the  which turns out to be the minimisation of the variational free energy ℒ(q) (see <ref>.
This translates to finding an approximate posterior q(θ;η) that balances two key factors (common in Bayesian inference): 

    
  * Data fit: The first term, 𝔼_q(θ;η) [log(p(𝒟 | θ))], represents the expected log-likelihood of the data under the approximate posterior. A high value for this term indicates a good fit between the model and the data.

    
  * Prior regularisation: The second term, D_KL[q(θ; η) || p(θ)], is the KL divergence between the approximate posterior and the prior distribution, p(θ). It acts as a regulariser, penalising overly complex posterior distributions that deviate significantly from the prior.


The expressions of these terms highlight the impact of our prior modelling choices on the optimisation problem. A poorly specified prior – independent standard Gaussian distribution – over parameters, as is the case with BNNs, can degrade the quality of the posterior approximation. However, it remains the most expedient choice for BNNs from the perspective of computational expense. As suggested by Knoblauch et al. <cit.>, we can opt for a difference divergence metric instead of KLD to alleviate the negative influence of misspecified prior. For example, KLD is known to exhibit a mode-seeking behaviour <cit.>. By selecting a different divergence metric one can improve the quality of the posterior distribution, both in terms of fitting the data and providing better uncertainty estimates <cit.>.

In this context, we examine the effect of using the flexible Rényi's-α divergence on the posterior predictive performance. We modify the variational free energy objective by replacing the KLD with Rényi's-α divergence:

    ℒ(q) = -𝔼_q(θ;η) [log p(𝒟| θ)] + D_AR^(α)[q(θ;η)||p(θ)].

To compute the expected value of the log-likelihood term, we use Monte-Carlo integration by drawing samples from the variational distribution q given the parameters η. The likelihood function is computed using <ref> and we approximate the first term in <ref>, as follows:

    𝔼_q(θ;η)[log(p(𝒟|θ))]    ≈1/N_q∑_c=1^N_qlog p(𝒟|θ^(c)), θ^(c)∼ q(θ;η) 
    
            = 1/N_q∑_c=1^N_qlog∏_i=1^N_1∏_k=1^N_2𝒩 s_k^(i); μ_kθ^(c) , σ_kθ^(c), θ^(c)∼ q(θ;η) 
    
            = 1/N_q∑_c=1^N_q∑_i=1^N_1∑_k=1^N_2log𝒩s^(i)_k ; μ_kθ^(c)σ_kθ^(c), θ^(c)∼ q(θ;η).

where N_q is the number of Monte Carlo samples, and θ^(c) is the c^th sample drawn from q(θ;η). In practice, we found that using 25 Monte Carlo samples (N_q = 25) is sufficient for accurate approximation, aligning with findings in 
<cit.> which suggest that even a single sample can yield satisfactory results.

In the next section, we detail Rényi's α-divergence as an alternative to KLD, which represents the second term of our objective function.




 §.§ Rényi's-α divergence
 
Rényi's α-divergence, introduced by Rényi <cit.>, offers a robust alternative to KLD. It is characterised by a hyperparameter α, which controls the degree of robustness, and it converges to KLD as a limiting case when α approaches 1. We denote this divergence by D_AR^(α) and follow the parameterisation provided in <cit.>:

    D_AR^(α)[q(θ)||p(θ)] = 1/α(α-1)log( 𝔼_q(θ)[ ( p(θ)/q(θ))^1-α] ).

This divergence can be computed in closed form when both q and p are normal distributions. However, in general, we approximate this divergence across various distributions by Monte-Carlo sampling. In the limit as α approaches 1, D_AR^(α) simplifies to D_KL:

    lim_α→ 1 D_AR^(α)[·] = D_KL[·].


In the context of our problem, D_AR^(α) represents the second term of the objective function in <ref>, which is approximated as follows:

    D_AR^(α)[q(θ;η)||p(θ)] ≈1/α(α-1)log(1/N_q∑_c=1^N_q[ ( p(θ^(c)))/q(θ^(c);η))^1-α] ),

where θ^(c) denotes the c^th Monte Carlo sample from q(θ;η). 



In <ref>, we plot divergence values D_AR^(α) between q(θ) and p(θ), assuming both are normal distributions with p(θ) as a standard normal distribution. The divergence value decreases as α increases, with KLD as the case when α = 1. 

Note that the objective function in <ref> contains a data-loss term and a complexity term. The behaviour of the optimisation problem differs significantly depending on whether α is greater than or less than 1. For α > 1, D_AR^(α) exhibits a mode-seeking behaviour, causing the posteriors to become more concentrated while being penalised less for deviating from the prior distribution. In this regime, the objective function places more emphasis on the data-loss term, and the complexity term – which measures the divergence between the prior and the variational posterior – receives less weight as α increases. In contrast, for α < 1, D_AR^(α) encourages a mass-covering behaviour, akin to expectation propagation. As α decreases below 1, the values of D_AR^(α) increase, resulting in more weight being assigned to the complexity term and less to the data-loss term. It is important to note, however, that a larger regularisation term does not necessarily imply that a misspecified prior will dominate the inference, as demonstrated in the experiments by <cit.>.

Given that the true posterior is unknown, it is uncertain whether the mode-seeking or mass-covering behaviour is preferable, making the choice problem-dependent. For scenarios where the true posterior is multimodal, a divergence that produces wider parameter variances (i.e., mass-covering behavior) may be beneficial for achieving more robust predictive estimates. In particular, we aim to set D_AR^(α) such that the posterior is robust to priors that strongly contradict observed data, and provides reliable uncertainty quantification. We rely on validation sets and adjust α as a hyperparameter to minimise metrics such as the normalised mean squared error or negative log-likelihood.



 §.§ GVI Posterior Approximation

We choose the approximate posterior to be the mean-field normal (MFN) variational family given by:

    Q_MFN = 
        ∏_l=1^L𝒩(θ_l; μ_l^q, σ_l^q).


We find the q(θ;η) within the Q_MFN family that minimises the objective function. This class of approximate variational family is commonly used for BNNs. Once the variational family is decided, the optimisation reduces to finding the parameters, η =[μ^q,σ^q], of the variational family that minimises the loss, ℒ(q) (<ref>). Thus the optimisation problem tries to find:

    η^* = _η{ℒ(q) }.




 §.§ Training

To optimise the variational free energy objective function (acting as the loss function), as defined in <ref>, we employ the Bayes-by-Backprop algorithm <cit.>. This algorithm approximates the expectation within the objective function through Monte Carlo sampling. Specifically, we use N_q = 25 samples for this approximation. 

A key technique for efficient training is the reparameterisation trick <cit.>. This allows us to automatic differentiation to compute the unbiased gradients of the loss function with respect to the parameters and hence enables the use of gradient-based optimisers commonly employed in deep learning.  In our implementation, we utilise the Adam optimiser <cit.> with its default settings. The entire training procedure is carried out in TensorFlow Probability <cit.>.

For Gaussian distributions, the reparameterisation trick involves sampling a standard Gaussian random variable ϵ, and then scaling it by the standard deviation parameter, σ^q, and shifting it by the mean, μ^q. To ensure the positivity of the standard deviation, we employ a soft-plus activation function. The resulting parameterisation can be expressed mathematically as follows:

    θ =μ^q + log (1 + exp(σ^q)) ⊙ϵ

where ⊙ denotes element-wise multiplication. 

Combining the expected negative log-likelihood (<ref>) and the Rényi's α divergence (<ref>), we obtain the final variational free energy objective:

    ℒ(q) =
        - 1/N_q∑_c=1^N_q∑_i=1^N_1∑_k=1^N_2log𝒩s^(i)_k ; μ_kθ^(c) , σ_kθ^(c)  + D_AR^(α)qθ^(c);η||pθ^(c),

where θ^(c)∼ qθ^(c);η.

To optimise the variational parameters, η =[μ^q,σ^q]. we compute the gradients using the Bayes-by-Backprop equations:

    Δ_μ^q = ∂ℒ(q)/∂θ + ∂ℒ(q)/∂μ^q


    Δ_σ^q = ∂ℒ(q)/∂θϵ/1 + exp(-σ^q) + ∂ℒ(q)/∂σ^q,

where the ∂ℒ(q)/∂θ term is obtained through standard backpropagation. The overall training procedure is summarised in <ref>.






§ NUMERICAL STUDIES

To assess the performance of our proposed approach, we conduct a numerical study involving four prototypical problems: two ODEs – the antiderivative operator and the gravity pendulum – and two PDEs – diffusion-reaction and advection-diffusion equations. These problems represent a diverse spectrum of complexities, allowing for a rigorous evaluation of our model's capabilities. 

We investigate the influence of the hyperparameter α on posterior predictive performance while maintaining consistent DeepONet architecture and optimisation parameter settings across all experiments. Detailed architectural specifications for each problem are tabulated in <ref>. To optimise variational parameters, we employ a full-batch Adam optimiser with standard hyperparameter settings: maximum 10000 epochs, learning rate = 0.001, β_1 = 0.9, and β_2 = 0.999. We execute ten independent optimisation runs with varying initialisations for each problem, discarding non-convergent cases identified by oscillatory loss behaviour.

The average values and standard deviations of the converged runs are reported for eleven different values of α  in <ref> and <ref>. We evaluate the model test performance using two complementary metrics: the normalised mean squared error (NMSE) and the negative log-likelihood (NLL). NMSE provides a straightforward measure of prediction error by averaging the squared differences between observed and predicted mean values of responses, normalised by the norm of the observed values. NLL, on the other hand, assesses how well the predicted probability distribution aligns with actual outcomes, accounting for both the prediction mean and its uncertainty.





 §.§ Problem 1: Antiderivative Operator

We commence our numerical investigation with the fundamental problem of the antiderivative operator over the domain x ∈ (0,1]:

    ds/dx = u(x),

subject to the initial condition, s(0) = 0. In this case, the independent variable is exclusively x, rendering y = x. Our objective is to approximate the solution s(x), driven by the input function u(x): 

    s(x) = s(0) + ∫_0^x u(τ)dτ,  x ∈[0,1],

Here, the input function a(y) from the general formulation is equivalent to the source term u(x).

To construct the training dataset, we sample input functions u(x) from a zero-mean Gaussian random field (GRF) characterised by:


    
    u(x)    ∼𝒢𝒫(0, κ(x_1,x_2)) 
    κ(x_1,x_2)    = exp( -x_1 - x_2^2/2 ℓ^2),


where κ is the radial basis function (RBF) covariance kernel with length-scale ℓ. As outlined in <ref>, we generate a training set comprising N_1 = 3000 input functions u(x) discretised at M=100 equidistant points within the domain (implying a^(i)∈ℝ^100, i=1,…, N_1). For each input vector a^(i), we compute the corresponding solution at N_2 = 20 randomly selected points, i.e., s^(i)(x_k),  k=1,…, N_2. Notably,  once the operator is learned, solutions can be evaluated at any arbitrary location within the domain. To assess generalisation performance, we employ a test set of 10000 input functions, evaluating predicted solutions s(x) for each of these test input functions at 100 equidistant locations in the domain. 

<ref> showcases the model's predictive capabilities by visualising solutions for two representative test cases chosen to highlight its performance. This ensemble of points, representing the solution queried at the 100 equidistant locations, provides a visual approximation of the mean solution and the associated uncertainty. A comparative analysis with predictions from D-DeepONet and ground truth solutions for these representative examples further underscores the efficacy of using a different value of α than 1. For a comprehensive assessment, we present a comparison of performance using NMSE and NLL metrics averaged over all 10000 test cases for eleven different α values ranging from 0.25 to 3.00. Tables <ref> and <ref> present these results. In both metrics, α = 1.25 achieved the lowest NMSE and NLL. Notably, the NMSE is reduced by more than 50% compared to the standard KLD-VI (with α=1.00), indicating a substantial improvement in mean predictions.





 §.§ Problem 2: Gravity Pendulum under External Forcing

We next consider the nonlinear dynamics of a gravity pendulum subjected to an external force.  The system is governed by the following nonlinear ODE:

    d^2ϕ/dt^2 = -k sinϕ + u(t),

where ϕ represents the angular displacement, k is a constant determined by gravitational acceleration and pendulum length, and u(t) is the time-dependent external forcing function. The time domain for simulation is set to t ∈ (0,1]. Transforming this second-order ODE into a state-space form yields:

    ds/dt =
    [ ds_1/dt; ds_2/dt;         ] =
    [              s_2; -k sins_1 + u(t);                  ],

with initial conditions s(0) = 0    0^T, where s_1 = ϕ and s_2 = dϕ/dt. Given the purely time-dependent nature of the problem. the domain reduces to a scalar temporal variable, i.e., y = t. For this problem, we consider the generalised input function a(y) as equal to the time-dependent forcing function u(t).

To construct the training dataset, we set k=1 and generate N_1 = 3500 training input functions, u(t), sampled from the GRF defined in <ref> at M=100 time points. For each training input function, we compute the corresponding reference solution, s_1, using a fourth-order Runge-Kutta integrator at N_2 = 20 randomly selected time points. This data is used to train the operator. A test set of 10000 input functions is employed to evaluate the predicted solution for each test input function at 100 equidistant points within the time domain t ∈ (0,1].  <ref> presents the model's predictions for two representative test inputs.

A visual comparison of the test prediction cases reveals that the deterministic D-DeepONet struggles to accurately capture the reference solution. The standard KLD-VI solution, obtained at α=1.00, yields an improvement in the mean predictions, while the α-VI DeepONet with α = 2.00 demonstrates superior performance over both in terms of mean prediction accuracy, as evident in <ref>. In terms of distributional fit, as measured by negative log-likelihood, the standard KLD-VI has the lowest NLL values and exhibits a better performance (<ref>). However, the NLL for α = 2.00 is only marginally higher. Considering both the metrics, α = 2.00 offers a favourable balance between accurate mean predictions and reasonable distributional fit.






 §.§ Problem 3: Diffusion-Reaction System

We extend our analysis to the diffusion-reaction PDE, which involves derivatives in both spatial and temporal coordinates. The two coordinates are denoted by the tuple y = {x,t}, where x represents the spatial location and t represents time. Diffusion-reaction equations model the combined effects of diffusion and chemical reactions within a system, finding applications in diverse fields where heat transfer, mass transfer, and chemical kinetics occur simultaneously. 

Given an external source term u(x), we consider the following diffusion-reaction PDE:

    ∂ s/∂ t = D_c ∂^2 s/∂ x^2 + ks^2 + u(x),  x ∈ (0,1),   t ∈ (0,1].

This equation represents a diffusion-reaction system influenced by a source term and characterised by a quadratic dependence on the solution. We set the diffusion coefficient D_c = 0.01 and the reaction rate k=0.01. In this case, the source term u(x) serves as the input function, equivalent to a(x) without the dependence on the temporal variable t. We consider the PDE with zero initial and boundary conditions i.e., s(x,0) = s(0,t) = s(1,t) = 0.

To generate the training data, we numerically solve the PDE using a finite difference method on a (x × t)≡(100×100) grid, following a similar approach as in <cit.>. The training dataset comprises N_1 = 500 distinct source terms, u(x), generated from a Gaussian random field with ℓ = 0.5 (<ref>), evaluated at M=100 equidistant points in space. For each training input function, the solution s(y) is obtained at N_2 = 100 random points y sampled from the 100×100 grid; these sampled solution points are denoted by filled white circles in <ref>. Subsequently, we employ this training dataset to learn the operator mapping from u(x) to the PDE solution s(x,t).


To assess generalisation performance, we employ a separate test set comprising 10000 diverse source terms. <ref> presents a visual comparison of the model's predicted solution against the reference solution for a representative test source term across the entire 100 × 100 spatiotemporal grid. The figure also includes the associated uncertainty (one standard deviation) and absolute error. It is seen that regions of low response values (visualised as bluish areas) exhibit reduced smoothness in the predicted mean solution compared to the reference. As such, these regions demonstrate higher uncertainty, as evidenced by increased standard deviation.

A comprehensive performance assessment of predictive performance across different values of α is conducted using NMSE and NLL metrics, the averages of which are summarised in Tables <ref> and <ref>, respectively. Consistent with previous findings, α values greater than 1 generally outperform the standard KLD-VI case (α=1.00). For this specific problem, α = 3.00 achieved the lowest NMSE, while α = 2.50 yielded the lowest NLL. Notably, the performance metrics at  α = 2.00 are only marginally inferior to the optimal values, suggesting a degree of robustness in the model's performance within this parameter range. 





 §.§ Problem 4: Advection-Diffusion Equation

We now extend our analysis to the advection-diffusion PDE, which combines both advection (transport due to fluid flow) and diffusion (random movement of particles). This equation is fundamental to various physical processes, including solute transport in fluids. We consider the following advection-diffusion equation: 

    ∂ s/∂ x + ∂ s/∂ t -D_c ∂ s^2/∂ x^2 = 0,  x ∈ (0,1),    t ∈ (0,1],

subject to a parametric initial condition s(x,0) = u sin^2(2π x) and periodic boundary conditions i.e., s(0,t) = s(1,t). The diffusion coefficient is set to D_c = 0.1. 
In this case, the operator maps the initial condition s(x,0) to the solution s(x,t) at the final time. As such, the input function is the initial condition itself, defined as a(x) = s(x,0) = u(sin^2(2π x)). 

To construct the training dataset, we generate N_1 = 1000 unique initial conditions by sampling u from a GRF (with input domain defined by sin^2(2 π x)) with length-scale ℓ = 0.5 (<ref>), discretised at M=100 spatial points. The PDE is then solved numerically on a spatiotemporal grid (x × t) ≡ (100×100) using a finite difference method. For each initial condition, the solution s(x,t) is evaluated at N_2= 100 randomly sampled points within the grid, following similarly as the data generation process outlined in <ref>. This dataset serves to train the operator.

For evaluation, a separate test set of 10000 initial conditions is employed. <ref> presents a comparison of the model's predicted solution against the reference solution for a sample test initial condition across the entire spatiotemporal domain. A quantitative evaluation of the model's predictive performance across different values of α was conducted using the NMSE and NLL metrics, summarised in Tables <ref> and <ref>. In contrast to the previous examples where larger α values (i.e., values greater than 1) typically yielded superior performance, this problem exhibits optimal results at  α = 0.5 in terms of both NMSE and NLL. While values of α greater than 1 typically led to subpar performance in this case, the specific value of α= 1.75 can be considered as a reasonable choice for α values greater than 1.







To assess the model's robustness and generalisation capabilities beyond training data distribution, we conducted out-of-distribution (OOD) testing. Two distinct OOD datasets were generated, each comprising 100 examples.

The first OOD dataset was constructed by altering the kernel length-scale of the GRF (<ref>) used to generate initial conditions. While the training data utilised a length-scale of ℓ = 0.5, the OOD dataset employed a reduced length-scale of ℓ = 0.2 within the RBF kernel. This modification introduces increased fluctuations in the generated initial conditions compared to the training distribution.

The second OOD dataset was generated using a fundamentally different kernel, the rational quadratic kernel, which is defined as:

    κ(x_1,x_2) = exp( 1 + ||x_1-x_2||^2/2 ρℓ^2)^-ρ,

This kernel is parameterised by an additional scale mixture parameter, ρ. For our experiments, we set ρ = 1.0 and ℓ = 0.5 to generate 100 OOD test cases. This represents 100 different initial conditions for the advection-diffusion example.

The predictive performance of models trained with three different  α values: 0.5, 1.0 (standard KLD-VI), and 1.75, was evaluated on these OOD test datasets. Similar to the in-distribution analysis, the NMSE and NLL metrics were computed. The average results, summarised in  <ref>, indicate that the models exhibit reasonable generalisation capabilities on OOD data. Consistent with the in-distribution findings, α =0.5 outperformed both the standard KLD-VI (α=1.00) and the α=1.75 model in terms of both NMSE and NLL for both OOD datasets. This improvement over the standard KLD-VI model translated to upto a 7% reduction in NMSE and an 8.7% reduction in NLL for the tested OOD scenarios. Interestingly, the performance on the rational quadratic kernel samples was superior to that on the RBF kernel samples.






§ DISCUSSION

The results from the numerical investigation demonstrate the superior performance of the proposed α-VI DeepONet compared to both the deterministic D-DeepONet and the standard KLD-VI DeepONet, as evidenced by the consistently lower NMSE and NLL values across all four problems (Figures <ref> and <ref>). This underscores the effectiveness of the α-VI DeepONet framework in capturing complex input-output relationships and quantifying associated uncertainties.





A key finding is the influence of the hyperparameter α on model performance, which varies across different problems. While a universal optimal α value is desirable, our results indicate that cross-validation is necessary to determine the most suitable setting for each specific problem. Notably, the optimal α values for all four problems were mostly different from the standard KLD-VI case (α = 1), highlighting the limitations of the KL divergence under prior misspecification.

The choice of α is greatly influenced by the underlying structure of the true posterior distribution. Mode-seeking methods may struggle in scenarios with multi-modal posteriors, as they are susceptible to local optima. Conversely, mass-covering methods may be less effective when the true posterior exhibits widely separated modes. Therefore, the optimal α value is inherently problem-dependent.

For problems like the antiderivative, gravity pendulum, and diffusion-reaction problems, larger α values (greater than 1) yielded superior performance, suggesting a preference for mode-seeking behaviour. Conversely, the advection-diffusion problem favoured smaller α values (less than 1), indicating a need for mass-covering behaviour. The selection of α should consider both NMSE and NLL, as these metrics provide complementary insights – NMSE looks only at the mean prediction, whereas NLL considers distributional fit – into model performance. 

While the α-VI DeepONet offers significant flexibility, it is important to acknowledge the associated computational overhead. Compared to the deterministic DeepONet, the α-VI DeepONet incurs notably higher training times. This increase arises from two primary factors: the approximation of the Rényi α-divergence, which necessitates Monte Carlo sampling, and the doubling of trainable parameters due to the estimation of both mean and variance for each weight. However, compared to a standard KLD-VI DeepONet, the training times are approximately 1.1x longer, which is reasonable for the gain in flexibility.



§ CONCLUSIONS

In this work, we have introduced a novel, prior-robust, uncertainty-aware DeepONet framework grounded in generalised variational inference (GVI). This framework enables the learning of complex nonlinear operators while providing robust uncertainty quantification. Unlike previous approaches, our framework adopts an optimisation-centric perspective on Bayesian modeling by minimising the GVI objective. By replacing the Kullback-Liebler divergence with Rényi's α divergence, we enhance the model's robustness to prior misspecification and achieve superior predictive performance compared to the standard KLD-VI approach. The hyperparameter α offers flexibility in controlling the degree of robustness, allowing adaptation to different problem characteristics.

Our numerical investigation across four benchmark problems consistently demonstrates the advantages of the α-VI DeepONet over both deterministic and standard variational methods in terms of NMSE and NLL. The optimal α value varies across problems, highlighting the importance of tuning the hyperparameter.

While the α-VI DeepONet offers significant improvements, it is not without limitations. The introduction of the α hyperparameter increases model complexity and requires additional computational resources due to the approximation of Rényi's α divergence. Furthermore, the mean-field assumption underlying the variational posterior may not be adequate in all cases.

Future research directions include exploring alternative variational families (e.g., normalising flows) to address the limitations of the mean-field approximation and developing more efficient approximation techniques for Rényi's α-divergence. Additionally, investigating the application of the proposed framework to a wider range of complex systems is a promising avenue for future work.



§ ACKNOWLEDGEMENTS

R. Nayek acknowledges the financial support received from the Science and Engineering Research Board (SERB) via Grant number SRG/2022/001410, and the Ministry of Ports, Shipping and Waterways. 

unsrtnat





 Paper Title: An Explainable Vision Transformer with Transfer Learning Combined with Support Vector Machine Based Efficient Drought Stress Identification.





[1,2]Aswini Kumar Patraaswinipatra@gmail.com / akp@nerist.ac.in

3]Ankit Varshney 

[2]Lingaraj Sahools@iitg.ac.in

[1]Dept. of CSE, NERIST, India

[2]Dept. of BSBE, IIT Guwahati, India

[3]Dept. of Physics, IIT Guwahati, India


Early detection of drought stress is critical for taking timely measures for reducing crop loss before the drought impact becomes irreversible. The subtle phenotypical and physiological changes in response to drought stress are captured by non-invasive imaging techniques and these imaging data serve as valuable resource for machine learning methods to identify drought stress. While convolutional neural networks (CNNs) are in wide use, vision transformers (ViTs) present a promising alternative in capturing long-range dependencies and intricate spatial relationships, thereby enhancing the detection of subtle indicators of drought stress. We propose an explainable deep learning pipeline that leverages the power of ViTs for drought stress detection in potato crops using aerial imagery. We applied two distinct approaches: a synergistic combination of ViT and support vector machine (SVM), where ViT extracts intricate spatial features from aerial images, and SVM classifies the crops as stressed or healthy and an end-to-end approach using a dedicated classification layer within ViT to directly detect drought stress. Our key findings explain the ViT model's decision-making process by visualizing attention maps. These maps highlight the specific spatial features within the aerial images that the ViT model focuses as the drought stress signature.  Our findings demonstrate that the proposed methods not only achieve high accuracy in drought stress identification but also shedding light on the diverse subtle plant features associated  with drought stress. This offers a robust and interpretable solution for drought stress monitoring for farmers to undertake informed decisions for improved crop management.







An Explainable Vision Transformer with Transfer Learning
Combined with Support Vector Machine Based Efficient
Drought Stress Identification
    *
    
===========================================================================================================================================





§ INTRODUCTION

Drought stress adversely affects the growth, development and yield of plants. The rise in global temperature coupled with the growing demand for water severely depletes soil water reserves. Water deficit and temperature stress can cause damage to the reproductive growth phase of crops, resulting in a substantial reduction in yield <cit.>. Furthermore, prolonged exposure to water deficit stress particularly in the reproductive stage can significantly affect grain quality. Ensuring sustainable crop yields in the face of escalating drought conditions has become a great challenge <cit.>. The plant's response to drought stress is influenced by a number of factors such as stress duration, severity, genotype and its developmental stage <cit.>. In plants, drought stress in early phase induces reduced stomatal conductance, transpiration and net carbon assimilation, whereas mid- and late-phases result in visible leaf senescence, stagnant growth and wilting. The early detection of drought induced phenotypic changes in plants, facilitates in preventing the adverse impacts of drought from irreversible damage through timely irrigation and other management practices. The point based, contact type and destructive methods based estimation of drought stress impact on crops are limited by good approximation of representative area as well as sample size and hence, difficult to implement over larger agricultural production systems. On the contrary, image-based, non-invasive and high-throughput analyses of subtle phenotypical changes in plants due to drought and other stresses, provides an efficient means of monitoring  stress levels in crops and good prediction of drought stress impact <cit.>. Various imaging techniques and sensors are utilized to precisely capture stress responses, including simple red-blue-green (RGB) , thermal, multispectral, hyperspectral, or fluorescence imaging methods <cit.>. However, traditional image processing generates significant amount of variation in light intensity, shading, occlusion, etc. leading to variation in quality thereby increases complexity of image processing <cit.>. Machine learning (ML) techniques, on the other hand, solve such complex problems which are nonlinear in nature by identifying key feature to be extracted from the acquired image in real time, based on knowledge and/or domain expertise of the user. Hence, ML aids in better decision-making and informed actions in real-world scenarios with minimal human intervention <cit.>. Deep learning (DL) <cit.> techniques however, help in extracting different information from the raw image data during input training, by means of various convolutions which allows larger learning capabilities, that result in the best classification/regression and higher performance and precision. Convolutional Neural Networks (CNNs) have been developed for providing a much-detailed representation of the image features, for a more discriminative and detailed analysis of plant images, leading to highly accurate classifications <cit.>. Many studies have used deep CNN models to detect drought stress in affected plants, mostly solved either by finely tuned CNNs or by training from scratch. 

 Early work by Zhuang et al. <cit.> utilized a combination of segmentation, color, and texture feature extraction, employing a gradient boosting decision tree (GBDT) method to detect water stress in maize. However, a subsequent study using the same RGB dataset demonstrated that a deep convolutional neural network (DCNN) significantly outperformed GBDT in terms of performance <cit.>.
Sankararao et al. <cit.> developed a comprehensive pipeline for detecting water stress in groundnut canopies using hyperspectral imaging. The pipeline included steps for image quality assessment, denoising, and band selection, followed by classification using Support Vector Machine (SVM), Random Forest (RF), and Extreme Gradient Boosting (XGBoost/XGB). Transfer learning with a pre-trained DenseNet-121 model has been utilized to classify soybean color images into four levels of drought severity <cit.>. Similarly, a controlled dataset is created for chickpea crops to identify water stress at different stages—control, young seedling, and pre-flowering—using a CNN-LSTM hybrid model where CNNs that served as feature extractors and LSTM models for predicting water stress levels <cit.>.
The evaluation of various CNN models, including AlexNet, GoogLeNet, and Inception V3, for distinguishing between stress and non-stress conditions in water-sensitive crops such as maize, okra, and soybean, revealed that GoogLeNet achieved the highest identification accuracy <cit.>.
Chlorophyll fluorescence images of wheat canopies have been studied employing a multi-step process involving segmentation and feature extraction with a correlation-based gray-level co-occurrence matrix (CGLCM) and color features, followed by classification using different ML classifiers. The tree-based methods, particularly random forest (RF) and extra trees classifier, is found giving superior performance <cit.>.
A dataset of aerial images of potato canopies has been studied using a deep learning-based model for drought stress identification, leveraging various imaging modalities and their combinations <cit.>. An explainable deep learning framework was later proposed using the same dataset, based on CNN architectures and transfer learning, achieving improved accuracy <cit.>.
A regression approach is applied to predict the drought tolerance coefficient using hyperspectral images of tea canopies; the comparison of SVM, RF, and Partial Least Squares Regression (PLSR) revealed that SVM was the most effective <cit.>. Machine learning and deep learning methods were compared, including DNN, SVM, and RF, for drought stress detection using full spectra and first-order derivative spectra <cit.>. Goyal et al. <cit.> created a dataset of RGB images of maize crops and proposed a custom-designed CNN model that outperformed existing state-of-the-art CNN architectures in early drought stress detection.  However, Vision Transformers (ViTs) offer notable advantages over CNNs in capturing long-range dependencies and global context due to their self-attention mechanism, whereas CNNs rely on local receptive fields and often struggle to capture relationships across various parts of an image <cit.>. Additionally, ViTs exhibit greater flexibility with input image sizes and handle complex patterns and data variations more effectively <cit.>.

The self-attention mechanism in ViTs enables more accurate classification by considering the entire image context, which enhances accuracy and robustness compared to traditional CNNs <cit.>. For example, Dosovitskiy et al. demonstrated that ViTs could outperform CNNs in image classification tasks, highlighting their potential for diversified applications [<cit.>]. Recent studies have also applied ViTs to plant disease identification <cit.>.  In this study, we applied ViTs to drought stress identification to (a) achieve higher accuracy in detecting drought stress, b) leverage the ViT’s superior ability to handle complex patterns and variations inherent in field images captured in natural settings, ensuring the model’s practicality and relevance to real-world conditions, c) interprete model by utilizing ViTs to generate attention maps, which provide insights into the model’s decision-making process and contribute to the transparency and explainability of the predictions.




§ MATERIALS AND METHODS

In this section, we begin by introducing the experimental dataset. Next, we present the drought stress classification model using two different approaches. In the first approach, we employ the vision transformer with transfer learning. In the second approach, we propose a framework that utilizes a vision transformer as a feature extractor, followed by the integration of an SVM as the classifier. Additionally, we investigate the interpretability of the model by generating and analyzing the attention maps. This comprehensive use of the ViT elucidates how the spatial features of drought stress can be precisely identified. Finally, we discuss the performance metrics for the proposed approaches.


 §.§ Preparing the Data

The potato crop aerial images utilized in this study have been sourced from a publicly accessible dataset that encompasses multiple modalities <cit.>. Collected from a field at the Aberdeen Research and Extension Center, University of Idaho, these images serve as valuable resources for training machine learning models dedicated to crop health assessment in precision agriculture applications. Acquired using a Parrot Sequoia multi-spectral camera mounted on a 3DR Solo drone, the dataset features an RGB sensor with a resolution of 4,608 × 3,456 pixels and four monochrome sensors capturing narrow bands of light wavelengths: green (550nm), red (660nm), red-edge (735nm), and near-infrared (790nm), each with a resolution of 1,280 × 960 pixels. The drone flew over the potato field at a low altitude of 3 meters, with the primary objective of capturing drought stress in Russet Burbank potato plants attributed to premature plant senescence.




The dataset consists of 360 RGB image patches in JPG format, each measuring 750×750 pixels. These patches were derived from high-resolution aerial images through cropping, rotating, and resizing operations. The dataset is divided into a training subset of 300 images and a testing subset of 60 images. Ground-truth annotations, provided in both XML and CSV formats, indicate regions of healthy and stressed plants marked by rectangular bounding boxes. These annotations were manually created using the  LabelImg software. An example RGB image and its annotated regions are shown in Fig. <ref>. The testing subset is independent of the training subset and comes from different aerial images. Additionally, the dataset includes corresponding image patches from spectral sensors with red, green, red-edge, and near-infrared bands, each sized 416×416 pixels. However, we are only utilizing the RGB images due to the limitations of the low-resolution monochromatic images.

We utilized the augmented dataset prepared by Butte et al., expanding the original set from 300 to 1,500 images for training. The test set comprised 60 images. From each augmented and test image, annotated windows (i.e., rectangular bounding boxes) were extracted, creating separate folders for "healthy" and "stressed" images. These folders served as input for our proposed model. The final count of training images for the "stressed" and "healthy" classes is 11,915 and 8,200, respectively. The test set, consisting of 60 images, yielded 401 healthy and 734 stressed images. We evaluated our method on this test set and further assessed the model performance through 5-fold cross-validation.



 §.§ Vision Transformer (ViT)
 



A vision transformer (ViT) is a type of neural network architecture that has revolutionized the field of computer vision <cit.>. Unlike traditional convolutional neural networks (CNNs), which process images pixel by pixel, ViT processes the input images in a sequential manner by dividing them into fixed-size patches, linearly embedding these patches, and then applying self-attention mechanisms for capturing global dependencies <cit.>. The VIT architecture used for our work is inspired by Dosovitskiy et al.<cit.> and depicted in Fig <ref>. It processes images by dividing them into fixed-size embedded patches, linearly transforming these patches, and treating them as sequences. The transformer architecture is then applied, incorporating multi-head self-attention mechanisms <cit.> that enable the model to capture long-range dependencies within the image. Layer normalization is applied before and after the multi-head attention, ensuring stable training, and a Multilayer Perceptron (MLP) head is added to the transformer's global representation for task-specific processing. 



  §.§.§ ViT Architecture
 The proposed ViT architecture is based on the layers in <cit.> and Dosovitskiy et al.<cit.> and comprises five main components: patch embedding, positional encoding, transformer encoder , normalization layer and a classification head. This is illustrated in Fig <ref>.

  
  * Patch Embedding: Input images are divided into fixed-size patches, which are then linearly embedded to create a sequence of embeddings.
  
  
  * Positional Encoding: To capture spatial information, positional encodings are added to the patch embeddings, allowing the model to understand the relative positions of different patches.
  
  
  * Transformer Encoder: The embedded patches are fed into a Transformer Encoder, which is the core component of the ViT architecture. This encoder consists of twelve identical encoder layers stacked together. Each attention layer analyzes the relationships between pairs of patches, allowing the model to understand how different image regions interact and influence each other. Each attention layer consists of the following:
        
            
  * Multi-Head Self-Attention: This mechanism allows the model to weigh the importance of different parts of the image. It captures global dependencies between the patches.
            
  * MLP (Multi-Layer Perceptron) Block: This block introduces non-linearity to the network and further processes the information from the attention layer.
            
  * Normalization Layers: Layer normalization is applied after the multi-head attention and MLP blocks to stabilize training.
            
  * Dropout: The model uses dropout at two levels: within the attention mechanism and after the MLP block. Dropout is used to prevent over-fitting by randomly dropping out neurons during training.           
        
   
  * Normalization Layer: Layer normalization applied to the output of the encoder serves several crucial purposes: Reduces internal co-variate shift, improves gradient flow, acts as regularization, handles varying input distributions and accelerates convergence.     
    
  
  * Classification Head: This head typically consists of a simple MLP layer that maps the feature representation to the desired output, such as class probabilities.






  §.§.§ Information Processing in ViT

At the core of the Vision Transformer (ViT) is the concept of a token, which plays a crucial role in the model's ability to process and understand images. A token is a fixed-size vector that represents a small patch of the input image. These tokens are at the core of the Vision Transformer model, forming the input sequence to the Transformer layers. This token-based approach enables the model to process and understand the image by focusing on the relationships between these patches through self-attention. The detailed explanation is given below.


    
  * Dividing the Image into Patches:
    
        
  * The input image is divided into smaller, fixed-size patches. For example, an image of size 224x224 pixels might be divided into patches of size 16x16 pixels, resulting in (224/16)^2 = 196 patches.
    
    
    
  * Flattening and Embedding:
    
        
  * Each image patch is then flattened into a one-dimensional vector. For instance, a 16x16 patch with 3 color channels (RGB) would be flattened into a vector of length 16 × 16 × 3 = 768.
        
  * These flattened vectors (patch representations) are then linearly embedded into a higher-dimensional space. This is typically done using a learnable linear projection, transforming each vector into a fixed-size embedding, say of dimension 768.
    
    
    
  * Tokens:
    
        
  * After linear embedding, each flattened and embedded patch becomes a "token." In this example, the image is transformed into a sequence of 196 tokens, each representing a 16x16 patch of the original image.
    
    
    
  * Adding Positional Encoding:
    
        
  * Since the transformer model does not inherently understand the order or position of tokens, positional encodings are added to each token to incorporate information about its original position in the image. This helps the model understand spatial relationships between patches.
    
    
    
  * Processing by Transformer Layers:
    
        
  * These tokens are then processed by the Transformer layers, which include self-attention mechanisms. The self-attention mechanism computes relationships between these tokens to understand how different parts of the image relate to one another.
    




 §.§ ViT with Transfer Learning

The proposed framework, as shown in Fig <ref> effectively combines transfer learning, the power of the Vision Transformer and attention-based interpretability to address the challenging task of drought stress identification in potato crop images captured in natural settings. A core component of this approach is the utilization of pre-trained weights. This technique, known as transfer learning, involves leveraging knowledge gained from solving one problem (often a large-scale image classification task) and applying it to a different but related problem. By employing pre-trained weights, the model can benefit from the rich feature representations learned from a massive dataset, accelerating training and potentially improving performance. Using this the ViT is trained over multiple epochs using a combination of binary cross-entropy loss, Adam optimizer and learning rate tuning. Experimenting with different learning rates is essential to find the optimal value for convergence and generalization. Eventually, the classification layer is responsible for making the final prediction. It takes the output of the ViT model and maps it to two classes: "healthy" and "stressed." This layer typically consists of a fully connected neural network with a sigmoid activation function for binary classification.

To enhance model interpretability, the architecture incorporates a mechanism to capture attention weights. Attention weights reveal which parts of the input image the model focuses on when making a decision. By visualizing these weights, researchers can gain insights into the model's decision-making process and identify key image features that contribute to the classification. 



  §.§.§ Attention Maps
 





Attention maps provide insights into how the model focuses on different parts of the image during the self-attention mechanism. This mechanism allows the model to selectively attend to specific areas while processing visual information. The image is first divided into patches. Self-attention then helps the model prioritize relevant patches and their relationships for effective feature extraction.

Attention maps act as a visual representation of the weights assigned by the self-attention mechanism to each patch. These maps can be visualized for each self-attention layer within the ViT model, as each layer progressively learns more intricate relationships between these image features. Higher weights indicate a greater focus on a specific patch and its connection to others. By analyzing these maps, we can essentially see the model's "thought process" during image understanding. We can identify which regions it prioritizes for information extraction. The following section highlights the computation behind the attention mechanism <cit.>. 

At its core, self-attention computes a weighted sum of the values (features) based on the similarities (attention scores) between different positions in the sequence. This is achieved through three learnable matrices: Query (Q), Key (K), and Value (V). 

Query Matrix (Q): The query matrix is responsible for capturing the information about the current token being processed. It learns to encode the features of the token in a format suitable for comparison with other tokens. Each token in the sequence is associated with a query vector, which represents its characteristics in the context of the entire sequence.

Key Matrix (K): The key matrix holds information about the relationship between the current token and other tokens in the sequence. It learns to encode the features that determine how relevant each token is to the current token.

Value Matrix (V):  

The key matrix holds information about the relationship between the current token and other tokens in the sequence. It learns to encode the features that determine how relevant each token is to the current token.

Given a sequence of tokens X = [x_1, x_2, ..., x_n], the attention scores A are computed as:


    A = softmax( QK^T/√(d_k))


where Q = XW_Q, K = XW_K, V = XW_V, and W_Q, W_K, W_V are weight matrices. d_k represents the dimensionality of the key vectors.

Once the attention scores are computed, they are used to compute a weighted sum of the values:


    Attention(Q, K, V) = A · V


where A is the attention matrix.


























To capture different relationships between tokens, ViT (and Transformers in general) often employ multiple attention heads. Each head learns different sets of Q, K, V weight matrices and computes separate attention scores and weighted sums. The results from all heads are concatenated and linearly transformed to maintain a consistent output dimension.
Since self-attention does not inherently consider the order of tokens, positional encoding is typically added to the token embeddings to provide positional information.
 
Algorithm for Capturing Attention Weights:
To enhance model interpretability, the architecture incorporates a mechanism to capture attention weights. The algorithm <ref> outlines the steps during the training of ViT to capture and utilize attention weights for the drought identification task. The key components of this class include the initialization, forward pass, attention weight capture, output size determination, and retrieval of attention weights. By visualizing these weights, researchers can gain insights into the model's decision-making process and identify key image features that contribute to the classification. 

Initialization
The VisionTransformerBinary class is initialized with a pre-trained Vision Transformer model passed as . During initialization, the model assigns the provided  to its own  attribute. Additionally, it sets up a fully connected (linear) layer () with a size appropriate to the output of the Vision Transformer. This layer is responsible for converting the output of the Vision Transformer into a format suitable for binary classification. An empty list  is also initialized to store the attention weights captured during the forward pass.

Forward Pass
The  function is central to the operation of the VisionTransformerBinary class. When an input image  is passed through the model, the function first clears any previously stored attention weights. This ensures that the attention weights list is fresh and only contains data relevant to the current input.

Next, the function registers hooks on the self-attention layers of each block in the Vision Transformer. These hooks are responsible for capturing the attention weights during the forward pass. For each block in the Vision Transformer's encoder layers, the self-attention layer is accessed, and a hook is registered to capture its attention weights using the  helper function. The hooks are stored in a list to facilitate their removal later.

With the hooks in place, the input image is passed through the Vision Transformer and then through the fully connected layer. This produces the model's output for the given input. After the forward pass is complete, the hooks are removed to free up memory, ensuring that they do not persist and interfere with future operations.

Attention Weights Capture
The  helper function is designed to capture the attention weights during the forward pass. It is triggered by the hooks registered on the self-attention layers. This function receives the module, input, and output as arguments. From the input, it extracts the query, key, and value components, which are essential for computing the attention weights. These components, along with the output, are appended to the  list. By capturing these components, the model can later compute and analyze the attention scores, which provide insights into the regions of the input image that the model focuses on during classification.

Output Size Determination
The  helper function determines the output size of the Vision Transformer model. This function performs a forward pass with a zero tensor of appropriate dimensions through the Vision Transformer. By doing so, it captures the shape of the output tensor produced by the Vision Transformer. The size of the last dimension of this output tensor is then returned, which is used to initialize the fully connected layer with the correct input size.

Retrieval of Attention Weights
The  function provides a simple interface to retrieve the captured attention weights. It returns the  list, allowing external components or functions to access the attention weights for further analysis or visualization.

The modular design of the class, with separate functions for initialization, forward pass, attention weight capture, output size determination, and retrieval, provides a clear and maintainable structure. This design facilitates the integration of attention-based insights into the classification process, enhancing the interpretability and performance of the model. 






 §.§ Integrating Vision Transformer (ViT) and Support Vector Machine (SVM)


This framework focuses on combining a vision transformer (ViT)  and a support vector machine (SVM) within a three-step approach for effective stress identification.

  
  * Feature Extraction: For each image in the dataset, extract the final hidden state or pooled representation from the ViT model to obtain a feature vector. Let 𝐗 represent the set of embedded features extracted from a dataset of images, and 𝐲 represent the corresponding class labels.
  
  
  * Training SVM: Train an SVM classifier using the extracted features 𝐗 and their corresponding class labels 𝐲.
  
  
  * Classification: For a new, unseen image, extract features using the pre-trained ViT model and use the trained SVM classifier to predict its class.



  §.§.§ Support Vector Machine (SVM)

Support Vector Machines <cit.> aim to find a hyperplane that best separates a given set of data points into different classes. Given a set of labeled data points (𝐱_1, y_1), (𝐱_2, y_2), …, (𝐱_n, y_n) where 𝐱_i is the feature vector of the i-th data point, and y_i is its corresponding class label (y_i ∈{1, 0} for binary classification), SVM seeks to find a hyperplane defined by the equation:


    𝐰·𝐱 + b = 0


where 𝐰 is the weight vector and b is the bias.

The goal is to maximize the margin, which is the distance between the hyperplane and the nearest data point from each class. The margin is computed as the perpendicular distance from a data point 𝐱_i to the hyperplane:


    margin = 1/𝐰· | 𝐰·𝐱_i + b |


Subject to the constraint that for all data points:


    y_i (𝐰·𝐱_i + b) ≥ 1


This constraint ensures that data points are correctly classified and lie on the correct side of the hyperplane.

The SVM optimization problem can be formulated as:


    Minimize  1/2𝐰^2


Subject to the constraints:


    y_i (𝐰·𝐱_i + b) ≥ 1   for all  i


This is the primal form of the optimization problem. However, the SVM problem is often reformulated in its dual form, which introduces Lagrange multipliers α_i to handle the constraints. The dual formulation is:


    Maximize  ∑_i=1^nα_i - 1/2∑_i=1^n∑_j=1^nα_i α_j y_i y_j 𝐱_i ·𝐱_j


Subject to the constraints:


    0 ≤α_i ≤ C   for all  i



    ∑_i=1^nα_i y_i = 0


The solution to the dual problem provides the values of α_i, and the weight vector 𝐰 and bias b can be obtained from these values.

In cases where the data is not linearly separable, SVM can be extended to handle non-linear decision boundaries using the kernel trick. The feature space is implicitly mapped to a higher-dimensional space, making it possible to find a linear separating hyperplane in that space.

Classification is performed based on the decision function derived from the trained model. Once the SVM model is trained with a set of support vectors, it identifies a hyperplane that best separates different classes in the feature space. The decision function for a new data point 𝐱 is given by:


    f(𝐱) = 𝐰·𝐱 + b


Here, 𝐰 is the weight vector, and b is the bias term. The sign of f(𝐱) determines the predicted class:


    Class 1,    if  f(𝐱) > 0 
    Class 0,    if  f(𝐱) < 0


The magnitude of f(𝐱) provides a measure of how far the data point is from the decision boundary. Larger magnitudes indicate greater confidence in the classification.

The key role of support vectors in this process is that they are the data points lying closest to the decision boundary. Support vectors effectively determine the position and orientation of the hyperplane. The optimization process in SVM aims to maximize the margin between the classes, and support vectors are the data points defining the edges of this margin. In practice, many data points do not significantly contribute to the definition of the decision boundary. Only the support vectors, with non-zero Lagrange multipliers α_i in the dual formulation, are crucial for determining the hyperplane. This property makes SVM memory-efficient and computationally faster, especially in high-dimensional spaces.`



  §.§.§ ViT+SVM Framework

The process begins with input images that are fed into the ViT. The ViT, pre-trained on a vast dataset, is adept at extracting meaningful features from the images. Once the features are extracted by the ViT, they are compiled into a feature matrix, which also includes the corresponding labels indicating whether the plants in the images are healthy or stressed. This matrix forms the input to the SVM, a robust classifier known for its effectiveness in handling high-dimensional data. The SVM is trained to discern between the two classes—healthy and stressed—based on the features provided. Finally, the trained SVM predicts the class of new images, categorizing them as either healthy or stressed. The entire approach is depicted in Fig <ref>.
The efficacy of this framework is evaluated using a designated test set and k-fold cross-validation. 


 §.§ Performance Evaluation Metrics

The model's performance underwent assessment using various evaluation metrics, including accuracy, precision, recall (sensitivity) and the Receiver Operating Characteristic (ROC) curve. These metrics are computed based on the counts of true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN), which collectively form a 2x2 matrix known as the confusion matrix. In this matrix, TP and TN indicate the accurate predictions of water-stressed and healthy potato crops, respectively. FP, termed as type 1 error, denotes predictions where the healthy class is inaccurately identified as water-stressed. FN, referred to as type 2 error, represents instances where water-stressed potato plants are incorrectly predicted as healthy. The classification accuracy is a measure of the ratio between correct predictions for both stressed and healthy images and the total number of images in the test set.


    Accuracy = True Positive + True Negative/Total Population



    Precision = True Positive/True Positive + False Positive



    Recall = True Positive/True Positive + False Negative



    F1-score = 2 ·Precision·Recall/Precision + Recall



Cross-validation is crucial in model development for two key reasons: it helps prevent over-fitting by assessing a model's performance across different subsets of the data, and it ensures the model's generalization ability, providing a reliable estimation of its effectiveness under various conditions. The model was trained and evaluated using k-fold cross-validation, a robust technique for assessing the generalization performance of the model. In each iteration of the k-fold cross-validation process, the dataset was partitioned into k folds, and the model was trained on k-1 folds while being validated on the remaining fold. This process was repeated k times, ensuring that every fold had the opportunity to serve as the validation set. For each fold, the Receiver Operating Characteristic (ROC) curve was plotted, illustrating the trade-off between true positive rate and false positive rate at various thresholds. After completing the k-fold cross-validation, the individual ROC curves were aggregated, and the mean ROC curve was calculated and plotted. AUC (area under the curve) measures the entire two-dimensional area underneath the entire ROC curve. AUC provides an aggregate measure of performance across all possible classification thresholds. One way of interpreting AUC is as the probability that the model ranks a random positive example more highly than a random negative example. AUC ranges in value from 0 to 1. A model whose predictions are 100% wrong has an AUC of 0.0; one whose predictions are 100% correct has an AUC of 1.0.This comprehensive approach provides a more reliable estimation of the model's performance, capturing its consistency across different subsets of the data and enhancing the overall assessment of its predictive capabilities.



§ RESULTS AND DISCUSSION

In this section, we present the experimental results of our proposed model for identifying drought stress in potato crop field images. First, we distinguish between healthy and stressed images. Then, we identify the spatial features responsible for the stress. Our experiments with the proposed Vision Transformer (ViT) framework were conducted in three ways:

    
  * ViT with Transfer Learning: Leveraging pre-trained weights.
    
  * ViT+SVM with Pre-trained Weights.
    
  * ViT+SVM with Optimal Weights. The optimal weights are the ones at which the model performs best while executing the ViT with pre-trained weights in the first case.


The PyTorch library and it's various sub-packages are leveraged for deep learning functionalities. In addition to this, sklearn, pandas, numpy packages are used for developing, training, and evaluating deep learning models, as well as for performing data analysis and pre-processing. The data pre-processing involves resizing images to (224, 224) pixels and transforming them into tensors. 




 §.§ Performance of ViT with Transfer Learning

We relied on PyTorch, a core library for deep learning, which provides the essential tools for tensor computations and neural network construction. Specifically,  and  were used to build and train the neural network. To handle image data, we used Torchvision's  for accessing pre-trained architectures and  for preprocessing images. Additionally, TQDM was employed to create progress bars, enhancing visibility into the training process. For data analysis and manipulation, we leveraged scikit-learn for machine learning utilities and Pandas for managing structured data.

The dataset preparation involved mapping class names to numerical labels using , where 'healthy' images were assigned the label 0 and 'stressed' images were assigned the label 1. We applied image transformations to resize the images to 224x224 pixels and convert them into tensor format, ensuring compatibility with the ViT model. The datasets were loaded using  from Torchvision, which facilitated the organization of images into training and testing sets. Data loaders were then created to handle batching, shuffling, and efficient data retrieval during training and validation, with a batch size of 128.

To adapt the Vision Transformer (ViT) architecture (as depicted in Fig <ref>) for our specific task of binary classification, we began by configuring the model using the models.vit_b_16 variant, setting pretrained=False to start with a clean slate. This approach allowed us to build the model from scratch without any influence from pre-existing learned features. Subsequently, we loaded custom pre-trained weights into the ViT model to realize the approach as shown in Fig <ref>. This step was crucial as it transferred learned representations from a previously trained model to our current architecture, leveraging prior knowledge to enhance performance. A custom class was defined to extend the ViT model by adding a fully connected layer for binary classification. This class also included methods to capture attention weights, which are essential for understanding the model's focus on different image regions.

The training process involved setting up a binary cross-entropy loss function to evaluate the performance of the classification model. We used the Adam optimizer with a learning rate of 0.001 and 0.0009 to update the model's weights. The training loop consisted of multiple epochs, during which the model was trained on the training dataset and validated on the test dataset. Training and validation losses were computed to assess the model's performance and adjust parameters accordingly. Throughout the process, attention weights were captured to provide insights into the model's decision-making process.






The learning curve typically plots both the training loss and validation loss curves on the same graph.
This allows to analyze how the model is performing on both the data it's trained on (training loss) and unseen data (validation loss).
The plot shown in Fig <ref> compares the training losses over the first 10 epochs for VIT trained from scratch versus VIT trained with pre-trained weights. The Vision Transformer (ViT) with pre-trained weights demonstrates a notably faster and more consistent reduction in loss compared to the ViT trained from scratch. This indicates superior initial performance and quicker convergence. As a result, we chose to further experiment with pre-trained weights rather than training the model from scratch.

The training loss is plotted in Fig <ref> for 20 epochs when using a Vision Transformer (ViT) with pre-trained weights and a learning rate of 0.001. The goal of training a machine learning model is to minimize the loss on the validation set. In an ideal scenario, the training loss and validation loss will both decrease as the model trains. However, this is not always the case.
In this graph, the training loss decreases steadily over the course of training. This suggests that the model is learning the training data well. However, the validation loss starts to increase after a certain number of epochs, specifically after epoch 8. This is a sign of over-fitting. Over-fitting is a problem that occurs when a model learns the training data too well, and it is no longer able to generalize to unseen data.

The training loss is plotted in Fig <ref>  for 20 epochs using a Vision Transformer (ViT) with pre-trained weights. The learning rate is 0.001 for epochs 1 to 10 and 0.0009 for epochs 11 to 20. The training loss decreases steadily, indicating the model is learning the training data well. However, the validation loss fluctuates and increases after epoch 6, suggesting over-fitting. The training loss is plotted in Fig <ref> for 20 epochs using a Vision Transformer (ViT) with pre-trained weights. The learning rate is 0.001 for epochs 1 to 8, with no dropout. For epochs 9 to 20, the learning rate remains 0.001, with an attention dropout of 0.2 and an MLP dropout of 0.1. The training loss decreases steadily, but the validation loss fluctuates and increases after epoch 8, indicating over-fitting once dropout is applied. The training loss is plotted in Fig <ref> for 20 epochs using a Vision Transformer (ViT) with pre-trained weights. The learning rate is 0.001 for epochs 1 to 8 and 0.0009 for epochs 9 to 20. An attention dropout of 0.2 and an MLP dropout of 0.1 are applied for epochs 9 to 20. The training loss decreases steadily, but the validation loss fluctuates and increases after epoch 8, suggesting over-fitting as dropout is applied and the learning rate is adjusted. 

In summary, all figures show a decrease in training loss, indicating the model is learning from the training data. Validation loss generally starts to increase or plateau after a certain number of epochs, suggesting overfitting is a common issue in these experiments. The introduction of dropout in Figures <ref> and <ref> seems to have exacerbated the over-fitting problem. However, the optimal result for the specified test set, reaching an accuracy of 0.929, is attained with a learning rate of 0.001 and without applying attention dropout or MLP dropout. The Vision Transformer (ViT) with transfer learning demonstrated impressive performance, achieving a precision of 0.95 and recall of 0.94 for stressed images, as well as a precision of 0.89 and recall of 0.91 for healthy images. The F1-scores for stressed and healthy images were 0.95 and 0.90, respectively, highlighting the model’s strong capability in accurately classifying both categories.



  §.§.§ Analyzing Attention Maps

Visualizing attention weights provides insights into how the ViT focus on different parts of an input during processing. By examining these weights, researchers and practitioners can understand the model's decision-making process, diagnose potential biases, and improve interpretability.

The following pseudocode outlines a systematic approach to calculate and visualize attention weights from a Vision Transformer model. This process involves capturing attention weights during the forward pass, computing attention scores, and generating visual representations of these scores.

Initialization:
The process begins by initializing an empty list to store the attention weights that will be captured during the forward pass of the Vision Transformer model. This list will later be used to compute and visualize the attention maps.

Forward Pass and Capture Attention Weights:
The next step involves iterating through each layer of the Vision Transformer model. For each layer, a hook is registered to capture the attention weights. The input image is then passed through the Vision Transformer to compute the output features. This stage ensures that attention weights are collected during the forward pass for later analysis.

Calculate Attention Score:
After capturing the attention weights, the algorithm processes each weight to compute the attention scores. This involves extracting the query, key, and value tensors from the hook outputs. The attention score is computed as per the principles discussed in section <ref>. The attention scores are then normalized to ensure they are in a range suitable for visualization.

Visualize Attention Maps:
In the visualization phase, each normalized attention map is resized to match the dimensions of the input image. A colormap (e.g., 'hot') is applied to the attention map to highlight areas of high attention. The attention map is then overlaid on the original image to create a visual representation of where the model is focusing.

Display:
Finally, both the original image and the overlaid attention maps are displayed, allowing for an interpretation of how the Vision Transformer model is making its decisions based on different regions of the input image.







The stressed image along with the corresponding attention maps from the 12 encoder blocks of the Vision Transformer (ViT) is shown in Fig <ref>. Several key observations can be made from these attention maps, including spatial relevance, hierarchical processing, interpreting model decisions, visualization of learned features, and using them as a diagnostic tool for model improvement. Each attention map is resized and overlaid on the original image, with colors indicating the attention intensity:

    
  * Red/Hot Colors: Indicate areas of high attention.
    
  * Yellow/Warmer Colors: Show areas of moderate  attention.
    
  * Dark/Cold Colors: Represent areas of low attention.
    




Spatial Relevance: The spatial relevance can give insights into which parts of the image the model finds important for differentiating between classes. The observations from the Fig <ref> are as follow:

    
  * Layer 1: The model's attention is broadly spread with higher intensity in some central regions. This indicates the model initially focuses on larger areas, possibly capturing the general structure of the image.
    
  * Layer 2-3: Attention starts to become more localized, with specific regions of high attention emerging. The model begins to identify and focus on more distinct features.
    
  * Layer 4-7: Attention further refines, highlighting specific areas more sharply. These layers likely focus on intermediate features that are critical for further processing.
    
  * Layer 8-12: Attention becomes very concentrated on specific, smaller regions. The model is likely focusing on fine-grained details and specific objects or patterns.

In the initial layers, the model's attention is broadly spread, capturing the overall structure of the image. As we move to higher layers, the attention becomes more localized, focusing on specific features and objects.

Hierarchical Processing: As we move through the layers of the ViT, attention maps can show how the model progressively refines its understanding of the image. In the lower layers, attention is distributed across large regions, capturing global context. As we ascend through the layers, the attention narrows down to more specific features, highlighting finer details and important objects within the image. This hierarchical processing is crucial for the model to effectively balance global and local information.

Interpreting Model Decisions:
By visualizing attention maps, we can interpret why the model made certain predictions. For instance, if the attention maps highlight specific objects or patterns in an image, it suggests that those elements influenced the classification decision. This interpretability can help validate the model’s decisions and identify potential biases or weaknesses.

Visualization of Learned Features:
The attention maps provide a visual representation of the features learned by the ViT. Unlike abstract feature vectors, these maps directly relate model activations to spatial locations in the input image. This visualization helps in understanding how the model processes visual information and forms its internal representations.

Diagnostic Tool for Model Improvement:
Analyzing attention maps can serve as a diagnostic tool for improving model performance. By examining where the model attends and comparing it with ground truth or human perception, we can identify areas where the model might be lacking or where it might over-emphasize certain features. This feedback loop can guide model refinement and training strategies.

In summary, attention maps in image classification tasks with Vision Transformers provide a transparent view into the inner workings of the model, highlighting which parts of the input image contribute most to its decision-making process. 


 §.§ Performance of ViT+SVM

We used a pre-trained Vision Transformer (ViT) model to extract features from both the training and testing datasets. These extracted features were then utilized by an SVM to identify stress, aiming to distinguish between stressed and healthy images. The implementation was done using the PyTorch framework, leveraging both its core library and the torchvision module. Key libraries such as torch, torchvision, csv, pandas, and scikit-learn were imported to facilitate feature extraction, data transformations, file handling, and classification tasks.

The pre-trained ViT model weights were sourced from a specified path and loaded into an initialized ViT model. We used the ViT-B/16 model, whose architecture was previously discussed, and obtained it from torchvision.models. The model's weights were updated using torch.load and subsequently, the model was set to evaluation mode to facilitate feature extraction. Data transformations were defined to standardize the input images. Each image was resized to 224x224 pixels and converted into a tensor. Data loaders were created for both training and testing datasets , with a batch size of 32. The feature extraction process was encapsulated in a specific function. This function is called twice, once for the training data loader and once for the test data loader. This extracts features from each image in the respective datasets and saves them along with their corresponding labels to separate CSV (comma-separated values) files. Feature extraction was performed in a no-gradient context to optimize performance. The parameters used are mention in Table <ref>.




We extracted features from the ViT in two scenarios: firstly, using the pre-trained weights of ViT-B/16, which were trained on ImageNet; and secondly, by training the ViT from scratch as detailed in Section 3, starting with the ImageNet initial weights up to epoch 8. These extracted features were separately fed as inputs to an SVM to evaluate performance in both scenarios. We conducted 5-fold cross-validation to determine the ROC curve, mean AUC, and mean accuracy. Additionally, we examined both cases using the specified dataset provided in the original data repository.




Table <ref> shows the accuracy and AUC (Area Under the Curve) of approaches: ViT+SVM (Pre-trained Weights) and ViT+SVM (Optimal Weights). The models were evaluated on a specified test set and using 5-fold cross-validation. ViT+SVM (Optimal Weights) outperforms ViT+SVM (Pre-trained Weights) in terms of both mean accuracy (0.95 vs. 0.88) and mean AUC (0.96 vs. 0.95) with 5-fold cross-validation. Though, both the approaches achieved the same accuracy on the specified test set, but ViT+SVM (Optimal Weights) generalizes better to unseen data and suggests potentially less over-fitting as evidenced by its higher mean accuracy and AUC with 5-fold cross-validation. Upon examining the mean RoC curves given in Fig <ref>, it is evident that ViT+SVM (Optimal Weights) is performing consistently as the folds have a small variance in AUC (0.01) and each fold has a high AUC (> 0.95). On the other hand, ViT+SVM (Pre-trained Weights) have a mean AUC of 0.95 and mean variance of 0.02.





















 §.§ Comparison of the Models




Table <ref> presents the performance metrics of various models on a specific dataset. The comparison shows that each model has its strengths and is suitable for different scenarios.

Performance on Specified test set:
    ViT with Transfer Learning consistently performs best across most metrics, making it the most balanced and robust model for this task.
    ViT+SVM (Optimal Weights) excels in precision for the stressed class and recall for the healthy class, making it ideal where high precision in detecting stressed instances and high recall in identifying healthy instances are crucial.
    ViT (Pre-Trained Weights) provides a balanced performance but does not outperform the other models in any specific metric.



K-fold cross validation: 
The ViT with Transfer Learning model achieved an average accuracy of 86.8%, with F1-scores of 85.6% for the "Healthy" class and 88.8% for the "Stressed" class. The model demonstrated consistent performance across the five folds, although there was a slight dip in accuracy in the fifth fold (83%).

The ViT with SVM model achieved a significantly higher average accuracy of 95%, with F1-scores of 93.6% for Class 0.0 and 95.6% for Class 1.0. This model consistently performed well across all folds, with the lowest accuracy being 92.7% and the highest reaching 95.8%.




The comparative results, as shown in Fig <ref> indicate that the ViT with SVM model outperforms the ViT with Transfer Learning model in terms of both accuracy and F1-scores. The average accuracy of the ViT with SVM model is substantially higher (95% compared to 86.8%). Furthermore, the F1-scores for both classes are also higher in the ViT with SVM model, indicating better classification performance for both "Healthy" and "Stressed" classes.

The five-fold accuracies for the ViT with SVM model show less variability and higher consistency compared to the ViT with Transfer Learning model, suggesting that the integration of SVM helps in stabilizing the model's performance across different data splits.

In conclusion, integrating Support Vector Machine with Vision Transformer (ViT) significantly enhances the model's performance compared to using transfer learning alone. The ViT with SVM model demonstrates superior accuracy and F1-scores, making it a more reliable choice for classification tasks. The higher consistency across folds further reinforces the robustness of the ViT with SVM model. This comparative analysis highlights the benefits of combining advanced deep learning architectures with classical machine learning techniques for improved performance in identifying drought stress.


§ CONCLUSION

Drought stress represents a severe threat to crop yield and quality, disrupting normal plant growth and survival rates. Detecting early signs of drought stress is crucial for effective crop management and intervention. Traditional methods, primarily reliant on Convolutional Neural Networks (CNNs), have made significant strides in capturing spatial hierarchies in image data. However, Vision Transformers (ViTs) offer a compelling alternative by leveraging self-attention mechanisms to capture long-range dependencies and complex spatial relationships, thus enhancing the detection of subtle drought stress indicators.

Our study successfully addressed the challenge of stress identification using smaller datasets by harnessing the feature extraction capabilities of Vision Transformers combined with the classification prowess of SVM. Unlike conventional CNN architectures, Vision Transformers utilize self-attention mechanisms to effectively capture relationships across different parts of the image, enabling them to model long-range dependencies, which is particularly advantageous for complex datasets where traditional CNNs may struggle to capture global context. Moreover, Vision Transformers can handle images of varying resolutions without necessitating architectural modifications, enhancing their flexibility in handling diverse datasets. Our framework also emphasized explainability by generating attention maps, which provide insights into the model’s focus areas within the images, thereby offering transparency in its decision-making process.

However, our comparative analysis revealed that integrating SVM with ViT significantly enhances classification performance. The ViT+SVM model achieved a higher average accuracy of 95%, with F1-scores of 93.6% for stressed images and 95.6% for healthy images, outperforming the ViT with transfer learning approach. This integration not only improved accuracy but also demonstrated greater consistency across different data splits, highlighting the robustness of the ViT+SVM model.

In summary, our study illustrates the superior performance of combining Vision Transformers with Support Vector Machines over traditional transfer learning methods. This approach provides a more reliable and accurate solution for drought stress identification, while the attention maps offer valuable insights into the model’s decision-making process. The findings affirm the potential of advanced deep learning and machine learning techniques in enhancing agricultural practices and decision-making, paving the way for improved crop management strategies.
 

unsrt







 Paper Title: HGOE: Hybrid External and Internal Graph Outlier Exposure for Graph Out-of-Distribution Detection.


















  IIP, ICT, CAS
      

 SCST, UCAS
    Beijing
  China
hejunwei22s@ict.ac.cn



[1]

  IIP, ICT, CAS
  
  Beijing
  China
xuqianqian@ict.ac.cn



  SCST, UCAS
  
  Beijing
  China
jiangyangbangyan@ucas.ac.cn



  IIP, ICT, CAS
  
  Beijing
  China
wangzitai@ict.ac.cn



  IIP, ICT, CAS
      

 SCST, UCAS
    Beijing
  China
sunyuchen22s@ict.ac.cn


[1]

  SCST, UCAS
    

  IIP, ICT, CAS
    

  BDKM, CAS
  
  Beijing
  China
qmhuang@ucas.ac.cn












§ ABSTRACT 
With the progressive advancements in deep graph learning, out-of-distribution (OOD) detection for graph data has emerged as a critical challenge. While the efficacy of auxiliary datasets in enhancing OOD detection has been extensively studied for image and text data, such approaches have not yet been explored for graph data. Unlike Euclidean data, graph data exhibits greater diversity but lower robustness to perturbations, complicating the integration of outliers. To tackle these challenges, we propose the introduction of Hybrid External and Internal Graph Outlier Exposure (HGOE) to improve graph OOD detection performance. Our framework involves using realistic external graph data from various domains and synthesizing internal outliers within ID subgroups to address the poor robustness and presence of OOD samples within the ID class. Furthermore, we develop a boundary-aware OE loss that adaptively assigns weights to outliers, maximizing the use of high-quality OOD samples while minimizing the impact of low-quality ones. Our proposed HGOE framework is model-agnostic and designed to enhance the effectiveness of existing graph OOD detection models. Experimental results demonstrate that our HGOE framework can significantly improve the performance of existing OOD detection models across all 8 real datasets.







<ccs2012>
   <concept>
       <concept_id>10010147.10010178.10010187</concept_id>
       <concept_desc>Computing methodologies Knowledge representation and reasoning</concept_desc>
       <concept_significance>500</concept_significance>
       </concept>
   <concept>
       <concept_id>10010147.10010257.10010321</concept_id>
       <concept_desc>Computing methodologies Machine learning algorithms</concept_desc>
       <concept_significance>300</concept_significance>
       </concept>
   <concept>
       <concept_id>10002950.10003624.10003633</concept_id>
       <concept_desc>Mathematics of computing Graph theory</concept_desc>
       <concept_significance>500</concept_significance>
       </concept>
 </ccs2012>


[500]Computing methodologies Knowledge representation and reasoning
[300]Computing methodologies Machine learning algorithms
[500]Mathematics of computing Graph theory
























HGOE: Hybrid External and Internal Graph Outlier Exposure for Graph Out-of-Distribution Detection
    Qingming Huang
    August 5, 2024
=================================================================================================




[1]Corresponding authors.



§ INTRODUCTION

Nowadays, graph-structured data have shown significant success in handling non-Euclidean relationships, prevalent in multimedia systems such as social networks <cit.>, knowledge graphs <cit.>, citation networks <cit.>. These capabilities facilitate advanced applications like scene graph generation <cit.>, visual analysis <cit.>, anomaly detection<cit.> and recommender system <cit.>, 
by modeling complex relationships between heterogeneous data types<cit.>. 
However, a significant challenge arises from the i.i.d. assumption on which the mainstream graph learning methods depend, i.e., the training and testing graph data are from the same distribution. This assumption often fails in real-world scenarios, particularly in domains where data is complex and lacks sufficient labeling, such as in the case of drug molecules and proteins. For example, a new drug might not be any part of the already annotated data. Consequently, this gives rise to an interesting issue: how to determine whether such a new drug is present in the annotated drug library? This problem, known as Graph Out-Of-Distribution (GOOD) detection, is crucial in advancing the use of graph learning in real-world scenarios. 

Although several methods for graph OOD detection have been developed <cit.>, they only utilize ID graph data. When the full data distribution is complicated, merely modeling the ID data might be insufficient to capture the essential clues for OOD detection. In fact, auxiliary public OOD data is often accessible to help the detector discover such clues. Exposing such auxiliary OOD data to the model, also known as Outlier Exposure (OE), is widely studied for image data <cit.>. However, the application of OE for graph samples has not yet been explored. To bridge this gap, we attempt to investigate the incorporation of OE into graph OOD detection.



Unlike image data, introducing outliers to assist in the OOD detection of graph data presents two significant challenges. 

Firstly, since graphs are abstractions of patterns from the natural world, they exhibit inherent diversity. For instance, various non-Euclidean structures ranging from water molecules to complex social networks can be represented using graphs, yet these graphs can significantly differ in structure and properties. In contrast, images possess a Euclidean structure, which facilitates easier feature transfer. Therefore, utilizing existing graphs as outliers directly is often not effective.

Secondly, graph data exhibits less robustness to perturbations, resulting in the possibility of OOD samples existing within the boundary of ID classes. For example, as illustrated in Figure <ref>, phenol and benzene differ only by a hydrogen atom and an oxygen atom, yet exhibit vastly different properties; phenol is solid at room temperature, while benzene is liquid. This suggests that slight perturbations could transform ID graph sample into an OOD sample. In contrast, image data typically exhibit a more compact intra-class distribution and more well-separated classes (e.g., dogs and cats), where minor perturbations often do not change their categories. Therefore, merely incorporating some external outliers for exposure could be effective for image data. But considering that outliers may exist even within the same class of graphs, the external outlier exposure remedy is insufficient for graph OOD detection.

To address the first challenge, existing studies suggest that introducing diverse outliers closer to the in-distribution data is beneficial. Therefore, given the inherent diversity of graph data, we propose incorporating cross-domain external outliers into training. Moreover, in addition to these external outliers, we aim to synthesize outliers that are nearer to in-distribution data. Regarding the second challenge, since there exist subgroups within a class, and outliers also exist between these subgroups, we consider synthesizing some internal outliers between subgroups to assist in OOD detection. Because these samples are close to In-Distribution samples, if the model can accurately identify these samples, it would further enhance the OOD detection performance. 

Motivated by this, we propose a general hybrid graph outlier exposure (HGOE) framework for graph OOD detection which integrates both external and internal outliers. The external outliers are easily collected from public database. As for internal graph outliers, we design a graphon-based ID-mixup strategy to simulate the OOD region among subgroups and synthesize OOD samples. Given these outliers, we further propose a boundary-aware OE loss to adaptively learn from true outliers and prevent the unintended bias.

In summary, the contributions of this paper are as follows:

    
    
  * We propose a novel hybrid graph outlier exposure framework for graph OOD detection. It simultaneously utilizes external outliers and internal outliers to enhance the diversity. To the best of our knowledge, this is the first trial of outlier exposure in graph-level OOD detection tasks.
    
  * To synthesize internal outliers, we design an ID-mixup method that can effectively generate outlier graph samples between ID subgroups based on graphons. 
    
  * We further introduce a novel boundary-aware loss. By instantiating the HGOE framework with a SOTA detector, we have surpassed the competitors on 8 real-world graph datasets.




§ RELATED WORK

Graph Neural Networks. Graph neural networks (GNNs) have been widely adopted in various deep learning tasks due to their ability to process graph-structured data, which can effectively extract both the structural and attribute information of graphs <cit.>. GNNs have achieved remarkable results in various deep learning tasks in recent years, such as recommendation systems, natural language processing and computer vision <cit.>. GNNs can be broadly categorized into two distinct classes, which encompass spectral-based GNNs and spatial-based GNNs <cit.>. 

Existing Spectral-based Graph Neural Networks leverage spectral graph theory for graph analysis, offering the advantage of incorporating global graph topology information. However, they also exhibit certain limitations, including high computational complexity, challenges in handling dynamic or heterogeneous graphs, and a deficiency in local perception ability<cit.>. For instance, ChebNet approximates spectral graph convolutions using Chebyshev polynomials of arbitrary order, while GCN simplifies this by employing only the first two Chebyshev polynomials as the graph convolution, effectively creating a fixed low-pass filter<cit.>. 

Spatial-based Graph Neural Networks leverage the spatial information of nodes within the graph to perform graph convolutions<cit.>. These networks are rooted in message-passing mechanisms<cit.>, wherein each node updates its features based on the features of its neighboring nodes. Notably, Graph Attention Networks (GAT)<cit.> introduce a self-attention mechanism to compute node neighbor weights, allowing for dynamic and adaptive aggregation within neighborhoods. GAT further enhances model capacity by employing multi-head attention. Meanwhile, GraphSAGE<cit.> learns how to aggregate node features from various sources using diverse functions like mean, max-pooling, or LSTM, making it suitable for inductive learning tasks that involve new nodes or graphs during testing. Graph Isomorphism Networks (GIN)<cit.>, another class of graph convolutional network, employ sum and MLP operations to aggregate node features. 
Additionally, Graph Structural Neural Networks<cit.> provide a versatile solution for incorporating structural graph properties into the message-passing aggregation scheme of GNNs.

Out-of-distribution (OOD) Detection. 
Existing deep learning-based classification methods often exhibit overconfidence on unseen classes <cit.>. To address this issue, out-of-distribution (OOD) detection <cit.> involves the task of distinguishing test samples from distributions different from the seen training data. It comprises post-hoc and fine-tuning approaches <cit.>. Post-hoc methods <cit.> leverage the logit space and output scores of models that trained on in-distribution data to classify ID and OOD data. Fine-tuning approaches <cit.> introduce extra regularization terms during training or incorporate auxiliary training data, referred to as outlier exposure, which can be either real, synthetic, or sampled from the feature space. Outlier exposure has proven effective in enhancing OOD detection performance. 

However, these methods are typically applied to image or text data. OOD detection in graph data remains an under-explored area. Graph OOD detection <cit.> aims to determine whether test graphs originate from within the in-distribution relative to the training set or are out-of-distribution. Some studies <cit.> focus on graph anomaly detection, where the training data comprises both in-distribution and anomalous data, positioning it as a subset of graph OOD detection. OCGIN <cit.> utilizes a GIN as its encoder and capitalizes on an SVDD <cit.> objective for one-class graph anomaly detection. GLocalKD <cit.> employs joint random distillation to pinpoint anomalous graphs on both local and global scales. GOOD-D <cit.> differentiates between ID and OOD graph data through hierarchical contrastive learning and perturbation-free graph data augmentation, having been exposed only to ID data during training. AAGOD <cit.> introduces a novel learnable amplifier generator, designed to generate graph-specific amplifiers, thereby enhancing the detection of OOD graphs on trained GNNs. GraphDE <cit.> introduces a generative framework for debiased learning and OOD detection in graph data.

Graph Data Augmentation. 
Graph data augmentation, involving transformations to enrich or enhance information in the given graph, has been extensively explored. Non-learnable augmentation methods <cit.> achieve this through perturbation or random sampling of edges, nodes, or subgraphs. In contrast, learnable augmentation methods <cit.> train an Augmenter with learnable parameters using techniques like Decoupled Training, Joint Training, or Bi-level Optimization <cit.>. Among these methods, some utilize graph mixup <cit.> to generate new graphs. G-mixup <cit.> enhances classification performance by generating new graphs through mixup based on estimating graphons of the same class. S-Mixup <cit.> employs soft alignments for node matching to achieve instance-level graph mixup.

As an orthogonal direction to existing graph-level OOD detection approaches, our approach focuses on introducing outlier samples that assist in OOD detection training. Our framework relies on utilizing in-distribution graphons to generate internal outliers within them, rather than directly performing augmentations on the original graph structure.



§ PRELIMINARIES

In this section, we provide definitions for key terms and concepts utilized throughout this paper.



 §.§ Problem Setting

Denote a graph by G =(𝒱, ℰ, X), where 𝒱 denotes the set of nodes, ℰ denotes the set of edges, and X∈ℝ^n × d is the node feature matrix. The adjacency matrix is denoted as A∈{0,1}^n × n, where A_ij = 1 indicates a connection between nodes v_i and v_j and A_ij = 0 otherwise.

In this paper, we focus on the unsupervised graph-level OOD detection problem. Specifically, given a set of unlabeled in-distribution graphs 𝒟^in = {G_i}^N_i=1 drawn from the distribution 𝒫^in, the aim of unsupervised graph-level OOD detection is to learn a graph OOD scoring function f(·) based on the ID data 𝒟^in. A higher score s=f(G) indicates a higher probability to be an OOD graph. The scoring function is evaluated on a test set 𝒟_test= 𝒟_test^in∪𝒟_test^ood (𝒟_test^in∩𝒟_test^ood = ∅) where 𝒟^in_test∼𝒫^in and 𝒟^ood_test∼𝒫^ood. It should be emphasized that graph data sourced from 𝒫^in and 𝒫^out might fall into multiple categories. However, in the unsupervised graph-level OOD task, the model is not provided with any category-specific labels.



 §.§ Graphon

A graphon, denoted by the function 𝒲:[0,1]^2 → [0,1], is a continuous, bounded, and symmetric function in graph theory, extensively used to describe graph generation <cit.>. The value 𝒲(i,j) approximates the probability of an edge between nodes i and j in a specific graph. A graphon can be considered as a function that embodies the characteristics of a class of graphs, allowing for the sampling of graphs from the graphon. These sampled graphs share similar topological features. Distinct graphons represent different graph classes, enabling comparative analysis of their structural features. These graphons are estimated using step function approximations <cit.>. Graphons are particularly useful in synthesizing new graphs that reflect patterns found in real-world data, offering insights into the underlying structure of complex networks. However, there is no closed-form expression for graphons. Previous studies <cit.> have employed a two-dimensional step function to estimate graphons, which can be considered a matrix representing the probability of edge existence. In this paper, we denote it as W ∈ [0,1]^D × D, where D is the dimensionality of the graphon. This matrix W can then be used to generate graphs with a number of nodes less than D.



§ METHODOLOGY





 §.§ Overall Framework



In general, with a graph OOD score function f(G), the basic learning objective for unsupervised graph OOD detection can be described as:

    min_f 𝔼_G ∼𝒟_in [ℒ_ood(f(G))],

where ℒ_ood is the loss function.

By integrating the hybrid graph outlier exposure (HGOE) procedure, we hope the exposed OOD data have higher OOD scores than ID data. Subsequently, we simultaneously minimize the OOD score of ID data and maximize that of OOD data:

    min_f 𝔼_G ∼𝒟_in   [ℒ_ood(f(G)) + . 
       .     β·𝔼_G^'∼𝒟_OE [ℒ_GOE(f(G),f(G^'))]].


Apparently, such an introduction of HGOE is independent of the basic graph OOD model and thus can be applied to most existing models.

In deploying the GOE framework, we identify two pivotal challenges: (C1) How to acquire high-quality graph outlier data? (C2) How to design a ℒ_GOE to effectively utilize these OE data? To tackle (C1), we introduce both internal and external outliers. To address (C2), we develop a boundary-aware loss function. 
The overall pipeline of HGOE is illustrated in Figure <ref>. To facilitate the generation of internal outliers, we start with the division of graphs into distinct subgroups, wherein each subgroup is defined by the similarity in properties among its constituent graphs.  This is followed by the estimation of the graphon for each subgroup. Thereafter, we deploy an ID-mixup technique that combines the graphons from these subgroups to obtain internal outliers, incorporating node feature information from external outliers into this process. The resulted internal and external outliers are then fed into the OOD detector, which is further optimized through the implementation of an innovatively designed boundary-aware loss. In subsequent sections, we will introduce the HGOE framework in detail.



 §.§ Mixture Outlier Training Strategy

Our mixture outlier training strategy incorporates outliers from two sources: real-world external outliers and synthesized internal outliers. External outliers are derived from public graph datasets similar to the training graphs, while internal outliers are generated from ID graphs.

External Outliers.
In contrast to image data, graph data in the real world display significantly more complex structures, with notable gaps in characteristics between fields like social networks and protein networks. To investigate the role of real-world outliers, we sample from several graph datasets to create a real-world OE dataset. During the training of different ID data, we select graphs from the OE dataset that are consistent with them in feature dimensions, ensuring no overlap between the OE graphs and the training and test sets. This process results in an external outlier dataset, denoted as 𝒟^ext_oe.

Internal Outliers.
The quality and diversity of external outliers data are crucial for outlier exposure, but are limited by the scope and quality of the existing auxiliary dataset. Synthesizing additional outliers then emerges as a solution, offering tailored, abundant, and diverse graph data for training. However, the challenge lies in ensuring that the synthesized data resonates with real-world outlier scenarios and does not introduce unintended biases. As discussed in the introduction, there exist internal outliers among the in-distribution graph data, which are distributed near the boundaries of subgroups. To obtain these internal outlier samples, we design an internal outlier generation strategy, which will be described in the next subsection.



 §.§ Internal Outlier Synthesis





A straightforward way to generate graphs is to interpolate each pair of samples in the dataset directly. However, unlike Euclidean data such as images, different graphs usually have different sizes (e.g., different numbers of nodes) and have unique topology in Non-Euclidean spaces. Therefore, we propose to mix up the graph generators instead of graph samples themselves to synthesize more realistic graph data.

To find internal outliers for input graphs, it is imperative to first divide the graphs into subgroups. For the input ID graphs, we adopt graph contrastive learning techniques (e.g., GraphCL <cit.>) for feature extraction, and then perform k-means clustering to obtain k subgroups 𝒞_i, where 𝒟^in = ⋃_i=1^k𝒞_i. 
Since the members in each subgroup often share similar properties, we assume that each subgroup within the ID data is generated by a specific graphon. And we use the widely-used universal singular value thresholding (USVT) method <cit.> to estimate the graphon W_i for each 𝒞_i in 𝒟^in.
    
For each pair of non-overlapping subgroups 𝒞_i and 𝒞_j in 𝒟^in, we perform a mixup operation over the corresponding graphons W_i and W_j as follows:

    ℳ = λ W_i + (1 - λ) W_j,

where λ∈ [0,1] is the balancing hyperparameter. The mixup result ℳ combines the structural features of subgroups 𝒞_i and 𝒞_j, and can be considered a new graph generator positioned between the two subgroups. By sampling based on this graphon, we could obtain graphs lying in the interpolated regions between these subgroups, preserving the subgroups's topologies.


Random Size Based Sampling. 
The interpolated graphon ℳ∈ [0,1]^N × N has the capability to generate infinitely many graphs. However, the naive generated graphs are very likely to have a size around N. This limits the diversity of the synthesized outliers, which does not meet our goal in the HGOE framework to synthesize outliers broadly distributed near the original subgroups, with various sizes of graphs. In order to increase the diversity of outliers, we employ a random size based sampling strategy. We first randomly sample the size of the graph r ∈ [2, N] and then generate the graph from the sampled graphon ℳ'  ∈ [0,1]^r × r. The existence of an edge between nodes i and j is determined by sampling from a Bernoulli distribution with the parameter ℳ'(i, j).


External Feature Alignment.
Merely sampling from the interpolated graphons could only produce pure structure graph outliers with the structure information (denoted as A'). Therefore, we further propose to generate node features based on external outlier features. Specifically, we calculate the structural features of both the generated internal outlier structure A' and the external outliers. And then we construct the node features for the generated internal outliers based on that of the external outliers with the most similar structure features. Here, the structural features s^diff_i of the i-th node in a d_s-dimensional space is obtained through a d_s-step random walk-based diffusion process on the graph:

    s^diff_i = [T_ii, T^2_ii, …, T^d_s_ii] ∈ℝ^d_s,

where T = A'D^-1 denotes the transition matrix for the random walk on the graph, and D is the degree matrix of A'.  After computing the structural features of A', we compare them with the structural features of external outliers. For each node in the synthesized internal outlier G', we assign node features that are most closely aligned with the structural features of nodes in the external outliers. Note that our external feature alignment approach is training-free. By performing structural searches on nodes with existing features, we can effectively assign outlier features to nodes that initially lack features.

Through the aforementioned ID-mixup based procedure, we can generate an arbitrary number of internal outliers, denoted as 𝒟^int_oe. Compared to real-world outliers, these synthetic outliers are not limited in quantity or source. Furthermore, being located near in-distribution samples, they effectively compensate for the uncontrolled quality issue of external outliers.



 §.§ Boundary-Aware OE Loss
 


The introduced outliers further present two critical issues: How can we ensure that the introduced outliers do not fall within the in-distribution area, and how to find those more important outliers? Take the social graph ID data as an example. Intermixing social graphs could still result in a social graph, so the generated samples are not OOD data and should be excluded from the outlier dataset. At the same time, different outliers can have varying levels of importance, with points on the boundary potentially being critical points where a change in properties is about to occur.

To solve these problems, we design the following boundary-aware OE Loss ℓ_ba which is adaptively aware of whether the sample is in ID or OOD space. For an input outlier graph G', the loss is calculated as:

    ℓ_ba(s_G',τ)= -(l - s_G')^γmax(log(s_G'), τ),

where s_G'=sigmoid(f(G')) is the OOD score scaled by a sigmoid function, l and γ are hyperparameters. Note that τ is an ID-boundary threshold, which we adaptively set as the smallest s among the ID samples:

    τ=min_G ∈𝒟_insigmoid(f(G)).


To better understand Eq. (<ref>), we rewrite it in the following form:

    ℓ_ba(s_G',τ) = 
     
    -(l - s_G')^γlog(s_G'),    if log(s_G') > τ
    
    - τ (l - s_G')^γ ,    if log(s_G') ≤τ.

From this formulation, we have the following observations:

    
  * When log(s_G') > τ, it indicates that G' is a valid outlier. In this case, the smaller s_G' is, the closer it is to the in-distribution boundary. Apparently, such near-boundary outliers are important in guiding the detector to distinguish OOD samples from ID ones. Therefore, we weigh it by the term (l - s_G')^γ, giving higher weights to these boundary samples. As γ increases, the model pays more attention to samples nearer to the ID boundary.
    
  * When log(s_G') < τ, it means that the outlier G' is very likely invalid and fall into in-distribution space. Blindly treating it as an outlier might be harmful to the learning. Therefore, by minimizing - τ (l - s_G')^γ, the OOD score is reduced, which enhances the recognition ability for in-distribution samples.


Then, considering both external and internal outliers, the total graph outlier exposure loss is formulated as:

    ℒ_GOE=∑_G' ∈𝒟_OE^int∪𝒟_OE^extℓ_ba(s_G',τ).


Finally, we can instantiate the overall training objective ℒ as follows:

    ℒ   =  ∑_G ∈𝒟_in[ℒ_ood(f(G)) + β∑_G' ∈𝒟_OEℓ_ba(s_G',τ) ]  
       = ∑_G ∈𝒟_inℒ_ood(f(G)) + β∑_G' ∈𝒟_OE^int∪𝒟_OE^extℓ_ba(s_G',τ).

In practice, our ℒ_GOE can be added as a regularization term to most existing OOD detection models, without modifying their network architectures.



§ EXPERIMENTS
 

 
 






 §.§ Experimental Setup
 



  §.§.§ Datasets

We select three pairs of datasets from the TU dataset <cit.> and five pairs from the OGB dataset <cit.>. Each pair of datasets belongs to the same field and shares similar features, but exhibits distribution shifts between the two datasets in the pair. 90% of the In-Distribution (ID) samples are used for training, while 10% of the ID samples and an equivalent number of OOD samples are used for testing. For more detailed information about these datasets, refer to Table <ref>. 





For external outlier data, we grouped datasets with identical feature counts into a unified external dataset. Based on the feature counts of our selected datasets, as detailed in Table <ref>, this organization resulted in two distinct external dataset collections, with node feature numbers being 1 and 9, respectively. In this configuration, for each In-Distribution dataset, we utilized other datasets from the corresponding external dataset collection for training purposes, deliberately excluding both the OOD dataset for testing and the ID dataset itself.



  §.§.§ Competitors

We adopt the following three categories of graph OOD detection methods as our competitors:

    
  * Non-deep Two-step Methods.
    We use WL <cit.> graph kernels as feature extractors and employ local outlier factor (LOF) <cit.>, one-class SVM (OCSVM) <cit.>, and isolation forest (iF) <cit.> as detectors to perform OOD detection.
    
  * Deep Two-step Methods.
    The process is similar to the above methods, but the feature extractor is replaced with graph deep self-supervised methods InfoGraph <cit.> and GraphCL <cit.>, and the detectors are replaced with isolation forest (iF) and Mahalanobis distance (MD) <cit.>. Compared to graph kernels, self-supervised methods can extract features of graphs better.
    
  * End-to-end Methods.
    We utilize three popular end-to-end learning methods, including OCGIN <cit.>, which uses graph neural networks for feature extraction and is optimized with SVDD. GLocalKD <cit.> uses distillation learning for graph anomaly detection, and GOOD-D <cit.> employs multi-level contrastive learning for end-to-end OOD detection.

For the proposed framework, we instantiate it on the SOTA graph OOD detector baseline GOOD-D <cit.>. Besides, we also implement two ablated variants to show the impact of internal and external outliers. Specifically, HGOE w/o IO and HGOE w/o EO denote the variants without internal and external outliers, respectively.




  §.§.§ Implementation Details

For HGOE, the ratio of external to internal outliers was set to 1:1, with the total number equal to the number of in-distribution samples seen during training. The hyperparameter λ was set to 2, and γ was set to 2. The dimension of the structural features s_i^diff was set to 16, and the number of clusters for all datasets was determined to be 3. GraphCL was run for 50 iterations with an embedding dimension of 32, and the probabilities for node dropping, feature masking, and edge removing were all set to 0.1. During ID-mixup, λ was randomly chosen from the range [0.01,1]. AUC (Area under the ROC Curve) <cit.> is used as the performance metric. A higher AUC value indicates better detection performance.



 §.§ Main Results
 





We evaluate the performance of HGOE and other competitors, with the results presented in Table <ref>. Our findings include: 

(1) Two-step detection methods do not perform as well as end-to-end detection methods, especially where non-deep learning methods are inferior to deep learning-based feature extraction methods. Within self-supervised methods, GraphCL-MD demonstrates the best performance, showing GraphCL's capability in extracting features for graph OOD detection. This also explains why we chose to utilize GraphCL features for clustering. 

(2) For our HGOE framework, compared to GOOD-D without the use of graph outliers, there was an enhancement in performance across all 7 datasets. On the ENZYMES+PROTEIN dataset, the average performance improve from 60.15 to 64.44. On Tox21+SIDER, it increased from 64.98 to 68.24, and on FreeSolv+ToxCast, the performance increase from 78.79 to 83.36.

(3) The performance improvement on IMDB-M+IMDB-B was not substantial because both are social datasets, and the outliers we introduce were from molecular and protein datasets, which are biologically oriented. However, using these as outliers still contributed to enhanced detection on IMDB-M+IMDB-B. This indicates that selecting outliers similar to the ID distribution, as opposed to introducing outliers completely unrelated to the ID data, is more effective.



 §.§ Visualization




  §.§.§ ID-mixup Visualization.
 After obtaining the subgroups within the known distribution, we performed a mixup of the estimated graphons and visualized the resulting mixed graphons in the form of heatmaps, as shown in Figure <ref>. The graphon in the center represents the result of performing ID-mixup on the two adjacent graphons. It is evident that the graphon after ID-mixup retains the structures of the original graphons, forming a new graph generator. This demonstrates that our ID-mixup can effectively blend the distributions of subgroups to a certain extent, fulfilling our hypothesis.






  §.§.§ Score Distribution Visualization.
 Based on the OOD scores for samples in the test set, we visualize the frequency distribution of OOD scores for both ID and OOD samples using different colors. As shown in Figure <ref>, the left column depicts the score distributions without using HGOE, while the right column shows the results after applying the HGOE framework. The less the overlap and the greater the distance between the ID and OOD areas are, the better the corresponding model's performance is. It is observable that our HGOE method has decreased the overlap area between ID and OOD distributions, which also explains the reason for the performance improvement.










 §.§ Ablation Study
 



  §.§.§ Using Only One Type of Outliers.
 In Table <ref>, HGOE w/o IO and HGOE w/o EO represent the results of using only internal outliers and external outliers, respectively. We observe that using just one of these types yields better results than not using any OE (Outlier Exposure) samples at all. Moreover, in most cases, the performance of using only external outliers is superior to that of using only internal outliers, but inferior to using both types. This indicates that the combination of both types of outliers can more effectively enhance the performance of hybrid graph outlier exposure.



  §.§.§ The Strategy for Adaptive Allocation of τ.

In Section <ref>, we introduce an adaptive parameter τ in the  distribution-aware OE loss, which is set as the smallest normalized score among the ID samples. To explore the specific role of τ, we implement methods setting τ according to the maximum, average, and minimum normalized scores. The results, as shown in Figure <ref>, indicate that the min strategy for allocating τ performs the best. However, this strategy negatively affects performance on the IMDB-M+IMDB-B dataset, which we believe is due to its significant difference from the other datasets.






 §.§ Sensitivity Analysis






  §.§.§ ID-mixup Weight λ.
 We explore the sensitivity of HGOE with respect to λ by evaluating its performance during ID-mixup across different λ ranges. As seen in Table <ref>, we progressively narrow the range of λ from [0.01,1.0] to [0.4,0.6]. In this process, a performance decline was observed in most datasets. This suggests that for ID-mixup, blending two in-distribution samples at varying ratios can increase the diversity of internal outliers, thereby enhancing detection effectiveness. However, an opposite trend of performance increase, rather than decrease, was observed in the CLintox+LIPO dataset pair. This anomaly may be related to the characteristics of these datasets, both being molecular graphs from the OGB and possessing extremely similar average node and edge counts, resulting in closer distributions. If ID-mixup outliers falling within the in-distribution range, it could lead to a greater performance decrease compare to other datasets. So when λ is set to a middle value, the generated graphon lies between subgroups. This explains why choosing a λ value range [0.3,0.7] yields better results.



  §.§.§ Hyperparameter γ of Boundary-aware OE Loss.
 As described in our framework, γ plays a significant role in the boundary-aware loss. We vary the value of γ to {0, 0.5, 1.0, 1.5, 2.0, 2.5, 3.0}, and as observed in Figure <ref>, as γ increases, there is a performance improvement when γ∈ [0,2]. However, performance starts to decline for some datasets when it exceeds 2 and 2.5. This indicates that within an appropriate range, introducing γ can effectively weight the samples, and this effect improves with the increase of γ, up to a point where it begins to decrease. This phenomenon validates the rationality and effectiveness of our designed boundary-aware OE loss.









§ CONCLUSION

In this work, we investigate the enhancement of OOD detection performance on graph-level data through hybrid graph outlier exposure. We demonstrate that not only external outliers are crucial for graph OOD detection, but internal outliers also play a significant role. Based on this, we propose a carefully designed ID-mixup based method for synthesizing internal outliers by generating OOD samples between in-distribution subgroups and aligning these with external outlier features. Upon obtaining these synthesized internal outliers, we effectively utilize the characteristics of outlier samples by adaptively allocating learning weights to OE samples using a well-designed boundary-aware OE loss. Our HGOE framework provide substantial performance improvements for other graph OOD detection methods. Extensive experiments on 8 real-world datasets show its superior performance.



This work was supported in part by the National Key R&D Program of China under Grant 2018AAA0102000, in part by National Natural Science Foundation of China: 62236008, U21B2038, U23B2051, 61931008, 62122075 and 61976202, in part by Youth Innovation Promotion Association CAS, in part by the Strategic Priority Research Program of the Chinese Academy of Sciences, Grant No. XDB0680000, in part by the Innovation Funding of ICT, CAS under Grant No.E000000, in part by the China Postdoctoral Science Foundation (CPSF) under Grant No.2023M743441, in part by the Postdoctoral Fellowship Program of CPSF under Grant No.GZB20230732, and in part by the China National Postdoctoral Program for Innovative Talents under Grant BX20240384. 



ACM-Reference-Format




