Paper Title: Explainable and Controllable Motion Curve Guided Cardiac Ultrasound Video Generation.







Cardiac Ultrasound Video Generation




 


1National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, School of Biomedical Engineering, Medical School,
Shenzhen University, China

xinyang@szu.edu.cn; nidong@szu.edu.cn 

2Medical Ultrasound Image Computing (MUSIC) Lab, Shenzhen University, China

3Marshall Laboratory of Biomedical Engineering, Shenzhen University, China

4Faculty of Applied Sciences Macao Polytechnic University, Macao, China 

5Shenzhen RayShape Medical Technology Co., Ltd, China




J.Yu and R.Chen et al.


Explainable and Controllable Motion Curve Guided Cardiac Ultrasound Video Generation
    Junxuan Yu1,2,3Junxuan Yu and Rusi Chen contribute equally to this work. Rusi Chen1,2,3⋆ Yongsong Zhou1,2,3 Yanlin Chen1,2,3 Yaofei Duan4 Yuhao Huang1,2,3 Han Zhou1,2,3,5 Tan Tao4 Xin Yang1,2,3() Dong Ni1,2,3()
    August 5, 2024
======================================================================================================================================================================================================================



§ ABSTRACT 
Echocardiography video is a primary modality for diagnosing heart diseases, but the limited data poses challenges for both clinical teaching and machine learning training.
Recently, video generative models have emerged as a promising strategy to alleviate this issue.
However, previous methods often relied on holistic conditions during generation, hindering the flexible movement control over specific cardiac structures.
In this context, we propose an explainable and controllable method for echocardiography video generation, taking an initial frame and a motion curve as guidance. 
Our contributions are three-fold.
First, we extract motion information from each heart substructure to construct motion curves, enabling the diffusion model to synthesize customized echocardiography videos by modifying these curves.
Second, we propose the structure-to-motion alignment module, which can map semantic features onto motion curves across cardiac structures.
Third, The position-aware attention mechanism is designed to enhance video consistency utilizing Gaussian masks with structural position information.
Extensive experiments on three echocardiography datasets show that our method outperforms others regarding fidelity and consistency. The full code will be released at https://github.com/mlmi-2024-72/ECMhttps://github.com/mlmi-2024-72/ECM.







§ INTRODUCTION

Echocardiography is a primary method that relies on dynamic video to obtain structural information for clinical diagnoses <cit.>. 
However, training radiologists with diagnostic skills and establishing machine learning models both suffer from limitations on video resources.
Recently, video generation models have demonstrated a promising ability to solve this problem, owing to their powerful capability in modeling data distribution <cit.>.
Several studies about video generation based on specific conditions have been investigated.
The typical ones relied on canny edges or depth maps <cit.> extracted from additional videos as conditions. Nevertheless, these conditions are non-editable and lack motion-driving information.
In contrast, Shi et al. <cit.> proposed the Motion-I2V framework to predict dense optical flow and guide video generation, maintaining both spatial and motion consistency.
Wang et al. <cit.> then employed a self-tracking training method.
Specifically, they specified the box positions of the first and last frames along with motion paths, to control the movement of objects.
However, these paths are basic and coarse, inadequate for ultrasound video synthesis that demands an accurate representation of intricate structure motions.





In the field of echocardiography video generation, 
Zhou et al. <cit.> proposed the OnUVS framework to synthesize ultrasound videos by animating source images and leveraging motion information from driving video. 
Nevertheless, OnUVS faced challenges due to the anatomical structure gap between the source image and the driving video, making accurate motion control difficult.
In addition, some studies have mined the structural information of the heart to guide the movement of videos.
For instance, Reynaud et al. <cit.> developed a Generative Adversarial Network (GAN) capable of generating echocardiography videos corresponding to the left ventricular ejection fractions (LVEFs).
They further employed a cascade video diffusion model conditioned on randomly sampled frames, enhancing the synthesis quality of echocardiography video <cit.>.
Van et al. <cit.> utilized segmentation masks of end-diastolic (ED) frames as a condition to generate four-chamber heart videos.
However, both LVEFs and ED masks are relatively sparse conditions, leading to an imbalance of information between intricate motions and limited conditions. 
This sparsity poses challenges for effectively controlling fine-grained cardiac structures, thus limiting the ability to capture the full complexity of heart movements and dynamics.

To address the above issues, we propose an explainable and controllable motion curve guided video diffusion model (ECM) that can synthesize video guided by the initial frame and motion curves (Fig. <ref>). 
Our contributions are threefold: 
(1) We innovatively mine the motion of the echocardiography video to obtain motion curves, which fully reflect the movement of each cardiac structure. This easily controlled approach enables the customization of videos through the modification (scaling and replacing) of the initial motion curve.
(2) As the curve lacks category information of the structure, we propose a Structure-to-Motion alignment mechanism. This mechanism extracts the semantic features of each cardiac structure and maps them with motion curve features, aiming to align visual and motion information effectively.
(3) We design position-aware attention masks based on the position of the cardiac structure movement, effectively enhancing the motion consistency of each structure.
To the best of our knowledge, ECM is the first study to apply cardiac motion curve guidance in echocardiography video generation. Extensive experimental results show that the proposed ECM is a flexible, controllable, and reliable method.




§ METHODOLOGY




Fig. <ref> shows the framework of ECM. During the training process, ECM takes an original video as input. 
A pretrained variational auto-encoder (VAE) from Stable Diffusion (SD) <cit.> is then utilized to downsample the video into latent features. 
By gradually adding noise to the latent and then learning to denoise it, the model can obtain the generated video with a pretrained VAE decoder. 
Remarkably, we develop the structure-to-motion alignment module to match cardiac structures with motion, yielding aligned features that condition the ECM model. 
Additionally, we employ Gaussian masks for each structure using a Position-aware Attention Mechanism, incorporating these into the spatial layers of the diffusion model. 
During the testing process, ECM generates echocardiography videos using an initial frame and motion curves as input. Users can customize the videos by replacing the initial frame or adjusting (scaling or replacing) the motion curves, highlighting the controllability and interpretability of our method.




 §.§ Extraction of Motion Curves

Previous studies <cit.> faced challenges in controlling the motion of specific cardiac substructures since they relied solely on the single sparse condition (LVEFs), while our approach aims to provide fine-grained control over the motion of each cardiac substructure.
Therefore, we extract motion curves for the key structures (e.g., Left Ventricle, Left Atrium, Mitral Valve, etc.) as conditions.

As illustrated in Fig. <ref>, the process of extracting motion curves is as follows: 
(a) We employ a well-trained anatomy detector (average accuracy=85%) to identify each substructure in each frame of the echocardiogram video. 
(b) Subsequently, we utilize the pixel coordinates of each substructure's bounding box (bbox) as the basis for encoding the motion curves, which form the basis for encoding the motion curves, represented as 
f_c^m∈ℝ^B × N × C × ( 4 × 2),
where N represents the number of frames and C represents the categories of the substructure. Notably, any missed detected structures would be treated as learnable parameters, initialized by the network.
(c) Since cardiac motion is periodic, we employ Fourier transformation (FT) to transform the pixel coordinates into a high-dimensional feature representation, denoted as 
f_c^m∈ℝ^B × N × C × E, 
where E represents the dimensionality of the features obtained from FT. 
(d) Finally, the motion embedding is passed through several multi-layer perceptron (MLP) layers. Overall, these motion curve features can be formulated as f_c^m∈ℝ^B × N × C × 1024.



 


 §.§ Structure-to-Motion Alignment Module


Although the motion curves of the echocardiography videos are captured, it is challenging for the model to distinguish the relationship between the motion curves and the semantic information of each substructure.
Recently, GLIGEN <cit.> has demonstrated its effectiveness in combining caption and bbox information, enhancing visual-language understanding to enable fine-grained control over specific objects in natural images.
However, our preliminary experiments suggested that GLIGEN struggles to effectively represent and interpret texts related to cardiac structures in echocardiography.



Innovatively, we replace texts with the cardiac structure features, denoted as f_i^s. 
As shown in Fig. <ref>, to obtain f_i^s, 
the regions of interest (ROI) that correspond to the cardiac substructures in the initial frame are cropped by the well-pretrained detector as mentioned above. 
Then, a pretrained CLIP image encoder <cit.> is utilized to transform the ROIs into structural embedding features.
Then, these features pass through several MLP layers,
resulting in an output denoted as f_c^s∈ℝ^B × N × C × 1024, which has the same shape with the extracted motion curve features f_i^m mentioned in Sec. <ref>.
Consequently, structural features and their corresponding motion curve features from the same category are concatenated to create aligned features. The aligned features are formulated as:

    F_c^a = Concat(f_c^s, f_c^m).


Note that any undetected structures are replaced with general features from the dataset.
Next, the concatenated features pass through an additional MLP layer for further integration.
Finally, to introduce the aligned motion curve features to guide the generation of echocardiography video, we then mapped 
F_c^a to the intermediate spatial layers of the UNet<cit.> via a cross-attention layer implementing cross attention, formulated as:

    CrossAtten(Q, K, V) = softmax( QK^T/√(d)) · V,

where Q, K and V represent the query, key and value respectively in the attention mechanism. Here, we regard noise latent feature F_l^m∈ℝ^B × 64 × 64 as query and the structure-to-motion embedding features as key and value. 






 §.§ Position-aware Attention Mechanism

Enhancing the consistency of cardiac motion is crucial in the task of cardiac video generation. 
To address this, we aim to inject the positional information of the cardiac structure into the cross-attention mechanism. Specifically, we design Gaussian masks that are generated based on the position of the cardiac structures. 
The Gaussian masks are defined as:

    M_g(x, y) = 1/2πσ^2exp(-(x - μ_x)^2 + (y - μ_y)^2/2σ^2),

where x and y represent the spatial positions of the mask.
μ_x and μ_y denote the center positions of the Gaussian distribution, which are the coordinates of the four corner points of the detected bbox. 
σ is the standard deviation to control the distribution width. 
Here, we set σ to 10 pixels as default.
Subsequently, the Gaussian masks are resized to match the dimensions of the latent feature map. 
The Gaussian-weighted cross-attention mechanism is as follows:

    CrossAttenMask(Q, K, V, M_g) = softmax( QK^T/√(d_k)⊙M_g ) V,

where ⊙ is the element-wise multiplication. Overall, this position-aware attention mechanism effectively integrates positional information of cardiac structures, enhancing the consistency and realism of generated cardiac motion.





§ EXPERIMENTS


Datasets and Implementations. 
To assess the performance of ECM, we gathered data from three sources: two private dataset from multiple hospitals and the publicly available dataset named EchoNet-Dynamic <cit.>.
The in-house dataset includes 144 apical four-chamber (A4C) and 100 apical two-chamber (A2C) heart videos, and the public one comprises 10,030 labeled A4C echocardiography videos.
For both datasets, the videos were randomly split into training (90%) and testing (10%) sets. During training, 12-frame clips were randomly sampled from each video, with a sampling interval ranging from 1 to 4 frames. 
For testing, videos were truncated according to different sampling intervals. 
The input videos from the private datasets were resized to 256x256 pixels, while those from the EchoNet-Dynamic dataset were kept at their original resolution of 112x112 pixels.
All methods were implemented in PyTorch using an NVIDIA RTX 4090 GPU under same settings. The Adam optimizer was used with a learning rate of 5e-3 and 60,000 training steps.

Evaluation Metrics.
Our evaluation metrics cover both image-level and video-level evaluation.
The image-level assessment includes Structural Similarity Index (SSIM) <cit.>, Mean Absolute Error (MAE) <cit.>, Peak Signal-to-Noise Ratio (PSNR) <cit.>, Fréchet Inception Distance (FID) <cit.> and Learned Perceptual Image Patch Similarity (LPIPS) <cit.>.
For video-level assessment, we only considered the commonly-used Fréchet Video Distance (FVD) <cit.>.
Notably, to assess the consistency of cardiac structures between the generated and target videos, we introduce a new indicator calculated by the Intersection over Union (IoU) between the bboxes of the original and synthesized videos.




Method Comparison.
The quantitative comparison of ECM and other methods is reported in Table <ref>.
The ECM model achieves the best performance across all metrics, indicating superior image quality and high fidelity compared to other methods.
It can be seen that the SEG Diffusion method, generating echocardiography videos without any control conditions, achieves poor quality.
Notably, ECM model markedly outperforms the SD method under bbox/text conditions, achieving 67.8%↓ in FVD. 
This demonstrates that synthesizing 2D images with rich conditions and stitching them into videos does not yield satisfactory outcomes. 
The integration of a position-aware attention mechanism significantly contributes to our superior performance and enhanced video consistency.
Furthermore, the ECM model shows better performance than 3D ControlNet. 
This disparity arises from the fact that natural images typically have clearly defined control conditions, whereas the motion in echocardiography videos is inherently more complex.
Similarly, for the EchoNet-Dynamic dataset, guided by motion curves, our proposed ECM outperforms the strong competitor <cit.> across most metrics, indicating its superior generation capacity.










Ablation Study.
We conducted the ablation study to test the contribution of each component in Table <ref>. 
The ECM model consistently outperforms others across most metrics for both the A4C and A2C datasets, achieving outstanding FVD scores of 189.27 and 208.12, and impressive IoU scores of 0.766 and 0.828, respectively.
This highlights that ECM effectively enhances video generation consistency while maintaining strong image quality.
Additionally, incorporating motion curves instead of traditional textual control leads to notable improvements. 
Specifically, using motion curves, the FVD decreased from 307.50 to 259.77, and the IoU increased from 0.691 to 0.748.
This indicates that motion curves provide precise control over the motion in echocardiography videos.
It can also be observed that the Structure-to-Motion alignment module (`+S2M') and the position-aware attention mechanism (`+Att') enhance image quality and video motion consistency, respectively. Consequently, the final ECM model demonstrates excellent performance at both the image and video levels.



Qualitative Results. 
Fig. <ref> demonstrates that ECM generates videos closely matching the target in both the A4C and EchoNet-Dynamic datasets by inputting the initial frame and motion curves. It effectively mimics heart structure motions such as chamber dilation, contraction, and diastolic opening and systolic closing
Furthermore, Fig. <ref> (a) illustrates ECM's controllability. Starting from the same initial frame, the second-row video uses the original motion curves from target video A, while the third-row video uses replaced curves from target video B. The mitral valve's systole and diastole differ after replacing the motion curves (as shown by yellow arrows), indicating ECM's effective control over motion curves.
More visualization results, including generated videos with replaced and scaled motion curves, can be found at http://106.53.160.189:8000/anonymous link.




§ CONCLUSION

In this study, We have presented ECM for generating echocardiography videos guided by motion curves, which reflect the movement of each cardiac structure. ECM enables to customize the generated videos by adjusting (scaling and replacing) the initial motion curve. Besides, to link the structure features with corresponding movement information, we propose the structure-to-motion alignment mechanism. Moreover, attention masks based on the position of the anatomical structures are introduced to enhance the motion consistency of each structure. Overall, our proposed ECM achieves state-of-the-art performance for generating echocardiography videos in terms of fidelity and consistency.




  §.§.§ 

This work was supported by the grant from National Natural Science Foundation of China (12326619, 62101343, 62171290), Science and Technology Planning Project of Guangdong Province (2023A0505020002), Science and Technology Development Fund of Macao (0021/2022/AGJ), and Shenzhen-Hong Kong Joint Research Program (SGDX20201103095613036).



  §.§.§ 

The authors have no competing interests to declare that are relevant to the content of this article.


splncs04




 Paper Title: Smoothed Energy Guidance: Guiding Diffusion Models with Reduced Energy Curvature of Attention.




Quantized electrical, thermal, and spin transports of non-Hermitian clean and dirty two-dimensional topological insulators and superconductors
    Bitan Roy
    August 5, 2024
==============================================================================================================================================




§ ABSTRACT 

Conditional diffusion models have shown remarkable success in visual content generation, producing high-quality samples across various domains, largely due to classifier-free guidance (CFG). Recent attempts to extend guidance to unconditional models have relied on heuristic techniques, resulting in suboptimal generation quality and unintended effects. In this work, we propose Smoothed Energy Guidance (SEG), a novel training- and condition-free approach that leverages the energy-based perspective of the self-attention mechanism to enhance image generation. By defining the energy of self-attention, we introduce a method to reduce the curvature of the energy landscape of attention and use the output as the unconditional prediction. Practically, we control the curvature of the energy landscape by adjusting the Gaussian kernel parameter while keeping the guidance scale parameter fixed. Additionally, we present a query blurring method that is equivalent to blurring the entire attention weights without incurring quadratic complexity in the number of tokens. In our experiments, SEG achieves a Pareto improvement in both quality and the reduction of side effects. The code is available at <https://github.com/SusungHong/SEG-SDXL>.






§ INTRODUCTION


Diffusion models <cit.> have emerged as a promising tool for visual content generation, producing high-quality and diverse samples across various domains, including image <cit.>, video <cit.>, and 3D generation <cit.>. The success of these models can be largely attributed to the use of classifier-free guidance (CFG) <cit.>, which enables sampling from a sharper distribution, resulting in improved sample quality. However, CFG is not applicable to unconditional image generation, where no specific conditions are provided, creating a disparity between the capabilities of text-conditioned sampling and sampling without text. This disparity results in a restriction in application, e.g., synthesizing images with ControlNet<cit.> without a text prompt (see the last two columns of Fig. <ref>).

Recent literature <cit.> has attempted to decouple CFG and image quality by extending guidance to general diffusion models, leveraging their inherent representations <cit.>. Self-attention guidance (SAG) <cit.> proposes leveraging the intermediate self-attention map of diffusion models to blur the input pixels and provide guidance, while perturbed attention guidance (PAG) <cit.> perturbs the attention map itself by replacing it with an identity attention map. Despite these efforts, these methods rely on heuristics to make perturbed predictions, resulting in unintended effects such as smoothed-out details, saturation, color shifts, and significant changes in the image structure when given a large guidance scale. Notably, the mathematical underpinnings of these unconditional guidance approaches are not well elucidated.




In this work, we approach the objective from an energy-based perspective of the self-attention mechanism, which has been previously explored based on its close connection to the Hopfield energy <cit.>. Specifically, we start from the definition of the energy of self-attention, where performing a self-attention operation is equivalent to taking a gradient step. In light of this, we propose a tuning- and condition-free method that reduces the curvature of the underlying energy function by directly blurring the attention weights, and then leverages the output as the negative prediction. We call this method Smoothed Energy Guidance (SEG).

SEG does not merely rely on the guidance scale parameter that cause side effects when its value becomes large. Instead, we can continuously control the original and maximally attenuated curvature of the energy landscape behind the self-attention by simply adjusting the parameter of the Gaussian kernel, with the guidance scale parameter fixed. Additionally, we introduce a novel query blurring technique, which is equivalent to blurring the entire attention weights without incurring quadratic cost in the number of tokens.

We validate the effectiveness of SEG throughout the various experiments without and with text conditions, and ControlNet <cit.> trained on canny and depth maps. Based on the attention modulation, SEG results in less structural change from the original prediction compared to previous approaches <cit.>, while achieving better sample quality.





§ PRELIMINARIES




 §.§ Diffusion models



Diffusion models <cit.> are a class of generative models that generate data through an iterative denoising process. The process of adding noise to an image 𝐱 over time t ∈ [0, T] is governed by the forward stochastic differential equation (SDE):

    d𝐱 = 𝐟(𝐱, t)dt + g(t)d𝐰,

where 𝐟 and g are predefined functions that determine the manner in which the noise is added, and d𝐰 denotes a standard Wiener process.

Correspondingly, the denoising process can be described by the reverse SDE:

    d𝐱 = [𝐟(𝐱, t) - g(t)^2 ∇_𝐱log p_t(𝐱)] dt + g(t)d𝐰̅,

where ∇_𝐱log p_t(𝐱) represents the score of the noisy data distribution and d𝐰̅ denotes the standard Wiener process for the reversed time.

Diffusion models are trained to approximate the score function with 𝐬_θ(𝐱, t) ≈∇_𝐱log p_t(𝐱). To generate an image based on a condition c, e.g., a class label or text, one simply needs to train diffusion models to approximate the conditional score function with 𝐬_θ(𝐱, t, c) ≈∇_𝐱log p_t(𝐱|c) and replace ∇_𝐱log p_t(𝐱) with it in the denoising process. To enhance the quality and faithfulness of the generated samples, classifier-free guidance (CFG) <cit.> is widely adopted. Accordingly, the reverse process becomes:

    d𝐱 = [𝐟(𝐱, t) - g(t)^2 (γ_cfg𝐬_θ(𝐱, t, c) - (γ_cfg-1) 𝐬_θ(𝐱, t))] dt + g(t)d𝐰̅.

Here, 𝐬_θ(𝐱, t) is learned by dropping the label by a certain proportion, and γ_cfg is a hyperparameter that controls the strength of the guidance. Intuitively, CFG helps us to sample from sharper distribution by conditioning on a class label or text.



 §.§ Energy-based view of attention mechanism



The attention mechanism <cit.>, which has been widely adopted in diffusion models <cit.>, has been interpreted through the lens of energy-based models (EBMs) <cit.>, especially through its close connection with the Hopfield energy <cit.>. In the modern (continuous) Hopfield network, the attention operation can be derived based on the concave-convex procedure (CCCP) from the following energy function <cit.>:

    E(ξ) = - lse(𝐗ξ^⊤) + 1/2ξξ^⊤,

where ξ∈ℝ^1× d, 𝐗∈ℝ^N× d, and lse stands for the log-sum-exp function, defined as lse(𝐯) := log(∑_i=1^N e^v_i). The quadratic term acts as a regularizer to prevent ξ from exploding <cit.>, while -lse(𝐗ξ^⊤) penalizes misalignment between 𝐗 and ξ.

Mathematically, it turns out that the attention mechanism is equivalent to the update rule of the modern Hopfield network <cit.>. Specifically, inspired by the Hopfield energy in (<ref>), and noticing that the first term depends on the attention weights, we propose the following energy function for entire self-attention weights in diffusion models:

Let 𝐐∈ℝ^(HW)× d be a matrix of query vectors and 𝐊∈ℝ^(HW)× d be a matrix of key vectors, where H, W, and d represent the height, width, and dimension, respectively. Let 𝐀∈ℝ^(HW)× (HW) := 𝐐𝐊^⊤. The energy function with respect to entire self-attention weights in diffusion models is defined as:

    E(𝐀) := ∑_i=1^H∑_j=1^W E'(𝐚_:(i,j)),
       E'(𝐚) := - lse(𝐚) = - log(∑_k=1^H ∑_l=1^W e^a_(k,l)).


Note that to explicitly denote the spatial dimension, we use the subscript (x, y) to represent the index of a row or column of the matrices. Despite using the definition in (<ref>) for the rest of the paper for simplicity, we additionally discuss the dual case, where we use the swapped indexing, in Appendix <ref>.

This view leads us to an important intuition: the attention operation can be seen as a minimization step on the energy landscape, considering that the first derivative represents the softmax operation which also appears in the attention operation. Building upon this intuition, we argue that Gaussian blurring on the attention weights modulates the underlying landscape to have less curvature, and we demonstrate this in the following sections by analyzing the second derivatives.



§ METHOD


Our aim is to theoretically derive the effect of Gaussian blur applied on the attention weights, which in the end attenuates the curvature of the underlying energy function. Then, utilizing this fact, we develop attention-based drop-in diffusion guidance that enhances the quality of the generated samples, regardless of whether an explicit condition is given. In Section <ref>, we claim some useful properties of Gaussian blur: that it preserves mean, reduces variance, and thus decreases the lse value. In Section <ref>, we find that the curvature of the energy landscape is attenuated by the attention blur operation, leading naturally to a blunter prediction for guidance. And finally, in Section <ref>, built upon this fact, we define Smoothed Energy Guidance (SEG) and propose the equivalent query blurring method, which can perform attention blurring while avoiding quadratic complexity in the number of tokens.



 §.§ Gaussian blur to attention weights



In this section, we derive some important properties of the Gaussian blur with the aim of figuring out the variation of the energy landscape. To this end, we start from some mathematical underpinnings on applying Gaussian blur to attention weights.

A 2D Gaussian filter is a convolution kernel that uses a 2D Gaussian function to assign weights to neighboring pixels. The 2D Gaussian function is defined as:

    G(x, y) = 1/2πσ^2 e^-(x-μ_x)^2 + (y-μ_y)^2/2σ^2

where μ_x and μ_y are the means in the x and y directions, and σ is the standard deviation.
The 2D Gaussian filter possesses symmetry, i.e., G(x, y) = G(-x, -y), and normalization, i.e., ∬ G(x, y)dxdy = 1. In practice, we use a discretized version of the Gaussian filter with a finite kernel size depending on σ, normalized to sum to 1.


Spatially applying a 2D Gaussian blur to the attention weights 𝐚 := 𝐐𝐤^⊤ preserves the average 𝔼_i, j[a_(i,j)]. In addition, the variance monotonically decreases every time we apply the Gaussian blur.



Proof sketch.
Applying a 2D Gaussian filter to the attention weights a_(i, j) yields the blurred values ã_(i, j):

    ã_(i,j) = ∑_m=-k^k∑_n=-k^k G(m, n) · a_(i+m, j+n)

where k is the filter size, G(m, n) is the Gaussian filter value at position (m, n), and a_(i+m, j+n) is the attention weight at position (i+m, j+n). Since the Gaussian filter is symmetric and normalized, it can be shown that the mean of the blurred attention weights is equal to the mean of the original attention weights. Similarly, we can show that the variance monotonically decreases when we apply a 2D Gaussian filter. See Appendix <ref> for the complete proof.

Note that this fact also implies that blurring with a Gaussian filter with a larger standard deviation causes a greater decrease in the variance of attention weights. This is because a Gaussian filter with a larger standard deviation can always be represented as a convolution of two filters with smaller standard deviations, due to the associativity of the convolution operation.

Finally, we show that applying a 2D Gaussian blur to attention weights increases the lse value in (<ref>), i.e., increases the energy in (<ref>). This provides a bit of intuition about the underlying energy landscape, yet it is more prominently utilized in the claims in the following sections.


Applying a 2D Gaussian blur to attention weights 𝐚 := 𝐐𝐤^⊤ increases the lse term when we consider the second-order Taylor series approximation of the exponential function around the mean μ := 𝔼_i,j[a_(i,j)]. Consequently, the maximum is achieved when the attention is uniform, i.e., a_(i,j) = a_(k,l) ∀ i,j,k,l. This corresponds to the case when we apply the Gaussian blur with σ→∞.



Proof sketch. Applying the second-order Taylor series approximation around the mean μ, and using Proposition <ref>, we show that the second-order approximation of lse(𝐚) is larger than or equal to that of lse(𝐚̃). Subsequently, we introduce Lagrange multipliers to find the maximum, which gives us the result, a_(i,j) = a_(k,l) ∀ i,j,k,l. We leave the full proof in Appendix <ref>.




 §.§ Analysis of the energy landscape



In this section, we demonstrate that applying a 2D Gaussian blur to the attention weights before the softmax operation results in computing the updated value with reduced curvature of the underlying energy function. To this end, we analyze the Gaussian curvature before and after blurring the attention weights. This is closely related to the Hessian of the energy function.


Let the attention weights be defined as 𝐚 := 𝐐𝐤^⊤. Consider the energy function in (<ref>). Then, applying a Gaussian blur to the attention weights 𝐚 before the softmax operation results in the attenuation of the Gaussian curvature of the underlying energy function where gradient descent is performed.



Proof sketch.
Let 𝐇 denote the Hessian of the original energy function, i.e., the derivative of the negative softmax, and 𝐇̃ denote the Hessian of the new energy function associated with blurred attention weights. Furthermore, let b_ij denote the i-th row, j-th column entry in the Toeplitz matrix 𝐁 representing the Gaussian blur. Calculating the derivatives, we have the elements of the Hessians, h_ij = (ξ(𝐚)_i - δ_ij)ξ(𝐚)_j and h̃_ij = (ξ(𝐚̃)_i - δ_ij)ξ(𝐚̃)_j b_ij. Using Lemmas <ref> and <ref> and under reasonable assumptions, we observe that |(𝐇)| > |(𝐇̃)|, which implies that the minimization step is performed on a smoother energy landscape with attenuated Gaussian curvature. The full proof is in Appendix <ref>.

To provide more intuition about what is actually happening and how we utilize this property in the later section, it is intriguing to consider the attenuating effect on the curvature in analogy to classifier-free guidance (CFG). CFG uses the difference between the prediction based on the sharper conditional distribution and the prediction based on the smoother unconditional distribution to guide the sampling process. By analogy, we propose a method to make the landscape of the energy function smoother to guide the sampling process, as opposed to the original (sharper) energy landscape.

From a probabilistic perspective, this process can be thought of as maximizing the likelihood of the attention weights in terms of the Boltzmann distribution conditioned on a given configuration, i.e., the feature map. Blurring the attention weights diminishes this likelihood as shown in Lemma <ref>, and also reduces the curvature of the distribution as shown in Theorem <ref>.




 §.§ Smoothed energy guidance for diffusion models



Based on the above observation that the Gaussian blur on attention weights attenuates the curvature of the energy function, we propose Smoothed Energy Guidance (SEG) in this section. For brevity, we redefine the unconditional score prediction as 𝐬_θ(𝐱, t), and the unconditional score prediction with the energy curvature reduced as 𝐬̃_θ(𝐱, t). Specifically, 𝐬̃_θ(𝐱, t) is the prediction with the attention weights blurred using a 2D Gaussian filter G with the standard deviation σ. We formulate the process as:

    (𝐐𝐊^⊤)_seg = G * (𝐐𝐊^⊤),

where * denotes the 2D convolution operator. Then, we replace the original attention weights with (𝐐𝐊^⊤)_seg and compute the final value as in ordinary self-attention.

For practical purposes when the number of tokens is large, we propose an efficient computation of (<ref>) using the property of a linear map, since the convolution operation is linear. Concretely, blurring queries is exactly the same as blurring the entire attention weights, and we propose the following proposition to justify our claim.


Let 𝐐 and 𝐊 be the query and key matrices in self-attention, and let G be a 2D Gaussian filter. Blurring the attention weights with G is equivalent to blurring the query matrix 𝐐 with G and then computing the attention weights.




Since the convolution operation is linear, we can always find a Toeplitz matrix 𝐁 such that:

    G ∗ (𝐐𝐊^⊤) = 𝐁(𝐐𝐊^⊤),

where ∗ denotes the 2D convolution operation. Using the properties of matrix multiplication, we can rewrite (<ref>) as:

    𝐁(𝐐𝐊^⊤) = (𝐁𝐐)𝐊^⊤ = (G ∗𝐐)𝐊^⊤.


Finally, SEG is formulated as follows:

    d𝐱 = [𝐟(𝐱, t) - g(t)^2 (γ_seg𝐬_θ(𝐱, t) - (γ_seg - 1) 𝐬̃_θ(𝐱, t))] dt + g(t)d𝐰̅,

where γ_seg denotes the guidance scale of SEG.

In a straightforward manner, as SEG does not rely on external conditions, it can be used for conditional sampling strategies such as CFG <cit.> and ControlNet <cit.>. For the combinatorial sampling with CFG, following <cit.>, we simply extend (<ref>) for improved conditional sampling with both SEG and CFG as follows:

    d𝐱 = [𝐟(𝐱, t) - g(t)^2 ((1-γ_cfg+γ_seg) 𝐬_θ(𝐱, t) + γ_cfg𝐬_θ(𝐱, t, c) - γ_seg𝐬̃_θ(𝐱, t))] dt + g(t)d𝐰̅,

which is an intuitive result, as the update rule moves x towards the conditional prediction while keeping it far from the prediction with blurred attention weights.

We are likely to get a result with saturation when using a large guidance scale, such as with classifier-free guidance (CFG) <cit.>, self-attention guidance (SAG) <cit.>, and perturbed attention guidance (PAG) <cit.>. This is a significant caveat since we need to increase the scale to achieve a maximum effect with these methods. Contrary to this, we can fix the scale of SEG as justified in Sec. <ref> and control its maximum effect through σ of the Gaussian blur, making the choice more flexible. For σ, two extreme cases are recognized. If σ→ 0, the blurred attention weights remain the same as the original, while when σ→∞, the attention weights merely adopt a single mean value across spatial axes. We find that even the latter extreme case results in a high-quality outcome, corroborating that we can control the quality to the limit without saturation.












§ DISCUSSION ON RELATED WORK




Classifier-free guidance (CFG) <cit.>, first proposed as a replacement for classifier guidance (CG) <cit.> is controlled by a scale parameter. The higher we set classifier-free guidance, the more we get faithful, high-quality images. However, it requires external labels, such as text <cit.> or class <cit.> labels, making it impossible to apply to unconditional diffusion models. Also, it requires specific traning procedure with label dropping and it is known that high CFG causes saturation <cit.>.

Tackling the caveats of CFG, unconditional approaches such as self-attention guidance (SAG) <cit.> and perturbed attention guidance (PAG) <cit.> have been proposed. SAG selectively blurs images with the mask obtained from the attention map and guides the generation process given the prediction. This indirect approach causes saturation and noisy images when given a large guidance scale, leading to the selection of a guidance scale less than or equal to 1. PAG guides images using prediction with identity attention, where the attention map is an identity matrix. However, the reliance on heuristics to make perturbed predictions results in unintended side effects. As an example of the side effects of replacing the attention map with identity attention, PAG changes the visual structure and color distribution of an image, as evidenced in Figs. <ref>, <ref>, and <ref>.

Contrary to these, we control the effect of SEG through the standard deviation of the Gaussian filter, σ. Moreover, while being theory-inspired, SEG is relatively free from unintended effects. In the following section, we corroborate our claim with extensive experiments.




§ EXPERIMENTS





 §.§ Implementation details


We build upon the current open-source state-of-the-art diffusion model, Stable Diffusion XL (SDXL) <cit.>, as our baseline, and do not change the configuration. To sample with SEG, we choose the same attention layers (mid-blocks) and guidance scale as PAG <cit.>. For SEG and PAG sampling, we use the Euler discrete scheduler <cit.>, while for SAG <cit.>, we instead use the DDIM scheduler <cit.> since the current implementation of SAG does not support the Euler discrete sampler. For SAG and PAG, we use the same configurations they used in the experiments with the previous version of Stable Diffusion, with guidance scales of 1.0 and 3.0, respectively. We set γ_seg to 3.0, except in the ablation study.















 §.§ Metrics


We use various metrics to evaluate quality (FID <cit.> and CLIP score <cit.>, calculated with 30k references from the MS-COCO 2014 validation set <cit.>) and to assess the extent of change due to applied guidance (LPIPS_vgg, alex <cit.>). The latter metric, calculated using the outputs of vanilla SDXL, measures the extent of side effects by comparing guided images to their unguided counterparts.



 §.§ Controlling image generation with the standard deviation


In this section, our aim is to demonstrate that with SEG, we can sample plausible images using vanilla SDXL <cit.> under various conditions and even without any conditions, as demonstrated in Fig. <ref>. Furthermore, without the risk of saturation, we can control the quality and plausibility of the samples. For the results, we use σ∈{1, 2, 5, 10}. Additionally, as mentioned in Sec. <ref>, we present two extreme cases, σ→ 0 (vanilla SDXL) and σ→∞ (uniform queries).



  
Unconditional generation

In this section, our aim is to demonstrate that with SEG, we can sample plausible images from the unconditional mode of the vanilla SDXL, which was originally trained on a large-scale text-to-image dataset. The results are presented in Fig. <ref>, Fig. <ref>, and Table <ref>. The results show a clear tendency to draw higher quality samples by utilizing the differences between the two energy landscapes with different curvatures derived from self-attention mechanisms.

In Fig. <ref> and Fig. <ref>, we show the effectiveness of generating more plausible images, while vanilla SDXL is unable to generate high-quality images without any conditions. The results show a clear tendency to draw higher quality samples by utilizing the differences between the two energy landscapes with different curvatures derived from self-attention mechanisms. When σ is larger, the definition and expression of the samples improve, as the difference in curvature becomes more pronounced.



  
Conditional generation

In Figs. <ref>, <ref>, <ref>, <ref>, and <ref>, we display sampling results conditioned on text, Canny, and depth map. Using text (Fig. <ref>), the vanilla SDXL without CFG is unable to generate high-quality images and produces noisy results. Canny and depth map conditioning on SDXL (Fig. <ref>, <ref>, and <ref>) is achieved through ControlNet <cit.>, trained on such maps. The results show that SEG enhances the quality and fidelity of the generated images while preserving the textual and structural information provided by the conditioning inputs. Notably, as σ increases, the generated images exhibit improved definition and quality without introducing significant artifacts or deviations from the original condition. The combination with higher CFG scales is shown in Figs. <ref>–<ref>.

In Table <ref>, we show the quantitative results for text-conditional generation in terms of σ. We observe a clear trade-off between image quality (represented by FID and CLIP score) and the deviation from the original sample (represented by LPIPS). We sample 30k images for each σ to compute the metrics.




 §.§ Comparison with previous methods

Since the results are visually favorable when we use σ = 10 and σ→∞, and they are the best in terms of CLIP score and FID, respectively, we adopt those configurations for comparison of unconditional guidance methods. The results are presented in Figs. <ref>, <ref>, <ref>, and Table <ref>. Notably, our method achieves better image quality in terms of FID, while remaining similar to the original output of vanilla SDXL as measured by LPIPS, implying a Pareto improvement.

r6.5cm



    < g r a p h i c s >



Ablation study on γ_seg and σ.








 §.§ Ablation study



In this section, we address two parameters, γ_seg and σ, and justify that fixing γ_seg is a reasonable choice. In Fig. <ref>, we present the results from our testing. The results reveal that increasing γ_seg does not generally lead to improved sample quality in terms of FID and CLIP score, due to various issues such as saturation. In contrast, increasing σ tends to improve sample quality and plausibility. This supports the claim that image quality should be controlled by σ, instead of the guidance scale parameter. We sample 30k images for each combination to calculate the metrics.







§ CONCLUSION, LIMITATIONS AND SOCIETAL IMPACTS




  
Conclusion

We introduce Smoothed Energy Guidance (SEG), a novel training- and condition-free guidance method for image generation with diffusion models. The key advantages of SEG lie in its flexibility and the theoretical foundation, allowing us to significantly enhance sample quality without side effects by adjusting the standard deviation of the Gaussian filter. We hope our method inspires further research on improving generative models, and extending the approach beyond image generation, for example, to video or natural language processing.



  
Limitations and societal impacts

The paper proposes guidance to enhance quality outcomes. Consequently, the attainable quality of our approach is contingent upon the baseline model employed. Furthermore, the application of SEG to temporal attention mechanisms in video or multi-view diffusion models is not addressed, remaining a promising avenue for future research. It is important to note that the improvements achieved through this method may potentially lead to unintended negative societal consequences by inadvertently amplifying existing stereotypes or harmful biases.


plain













§ FULL PROOFS




 §.§ Proof of Lemma <ref>



Let a_(i, j) denote the original attention weights and ã_(i, j) denote the blurred attention weights, as in the main paper. Assume that the original attention weights are properly padded to maintain consistent statistics. Then, the following shows that the mean of the blurred attention weights remains the same.

    𝔼_i,j[ã_(i,j)]    = 1/HW∑_i=1^H∑_j=1^Wã_(i,j) = 1/HW∑_i=1^H∑_j=1^W∑_m=-k^k∑_n=-k^k G(m, n) · a_(i+m,j+n)
       = ∑_m=-k^k∑_n=-k^k G(m, n) ·(1/HW∑_i=1^H∑_j=1^W a_(i+m, j+n)) 
       = ∑_m=-k^k∑_n=-k^k G(m, n) ·𝔼_i,j[a_(i,j)] = 𝔼_i,j[a_(i,j)] ·∑_m=-k^k∑_n=-k^k G(m, n) = 𝔼_i,j[a_(i,j)]


In addition, the variance of the blurred attention weights is smaller than or equal to the variance of the original attention weights.

    Var_i,j[ã_(i,j)]    = 1/HW∑_i=1^H∑_j=1^W (ã_(i,j) - 𝔼_i,j[ã_(i,j)])^2 
       = 1/HW∑_i=1^H∑_j=1^W(∑_m=-k^k∑_n=-k^k G(m, n) · (a_(i+m, j+n) - 𝔼_i,j[a_(i,j)]))^2 
       = ∑_m=-k^k∑_n=-k^k∑_r=-k^k∑_s=-k^k G(m, n) · G(r, s) ·Cov[a_(i+m, j+n), a_(i+r, j+s)]

Using the Cauchy-Schwarz inequality and the normalization property of the 2D Gaussian filter, we can show that the variance monotonically decreases when we apply Gaussian blur.

    Var_i,j[ã_(i,j)]    ≤∑_m=-k^k∑_n=-k^k∑_r=-k^k∑_s=-k^k G(m, n) · G(r, s) ·√(Var[a_(i+m, j+n)] ·Var[a_(i+r, j+s)])
       = (∑_m=-k^k∑_n=-k^k G(m, n) ·√(Var[a_(i+m, j+n)]))^2 
       ≤(∑_m=-k^k∑_n=-k^k G(m, n)) ·(∑_m=-k^k∑_n=-k^k G(m, n) ·Var[a_(i+m, j+n)]) 
       = ∑_m=-k^k∑_n=-k^k G(m, n) ·Var[a_(i+m, j+n)] 
       = Var_i,j[a_(i,j)]






 §.§ Proof of Lemma <ref>



Applying the second-order Taylor series approximation of e^x to our function f around the mean μ, we get:

    ∑_i=1^H∑_j=1^W e^a_(i,j)   ≈∑_i=1^H∑_j=1^W(e^μ + e^μ(a_(i,j) - μ) + 1/2e^μ(a_(i,j) - μ)^2) 
       = HW · e^μ + 1/2e^μ∑_i=1^H∑_j=1^W (a_(i,j) - μ)^2

In the last step, we used the fact that ∑_i=1^H∑_j=1^W (a_(i,j) - μ) = 0 because μ is the mean.

Similarly,

    ∑_i=1^H∑_j=1^W e^ã_(i,j)   ≈ HW · e^μ + 1/2e^μ∑_i=1^H∑_j=1^W (ã_(i,j) - μ)^2

Since Var[a] > Var[ã], we have:

    ∑_i=1^H∑_j=1^W (a_(i,j) - μ)^2 ≥∑_i=1^H∑_j=1^W (ã_(i,j) - μ)^2

Therefore, the second-order approximation of lse(𝐚) is larger than that of lse(𝐚̃).

Note that this fact also implies blurring with a Gaussian filter with a bigger variance causes more decrease in the variance of attention weights, because Gaussian filter with a larger variance can always be represented as a convolution of two filters with smaller variances, and the convolution operation is associative.

To find the maximum value subject to the constraint a_(1,1) + a_(1,2) + … + a_(H,W) = c for some constant c, we introduce Lagrange multipliers. Let g(a_(1,1), a_(1,2), …, a_(H,W)) = a_(1,1) + a_(1,2) + … + a_(H,W). The Lagrangian function is defined as:

    L(a_(1,1), a_(1,2), …, a_(H,W), λ) = e^a_(1,1) + e^a_(1,2) + … + e^a_(H,W) - λ(a_(1,1) + a_(1,2) + … + a_(H,W) - c)

Taking partial derivatives and setting them to zero yields:

    ∂ L/∂ a_(i,j) = e^a_(i,j) - λ = 0

Solving for a_(i,j), we obtain a_(i,j) = ln(λ) for all i = 1, 2, …, H and j = 1, 2, …, W
Summing these equations results in:

    λ = e^c/HW

Substituting λ back into a_(i,j) = ln(λ) gives a_(1,1) = a_(1,2) = … = a_(H,W) = c/HW.
Therefore, the minimum value of ∑_i=1^H∑_j=1^W e^a_(i,j) is achieved when a_(1,1) = a_(1,2) = … = a_(H,W).




 §.§ Proof of Theorem <ref>



Let 𝐚 = (a_1, …, a_n) denote the attention values before the softmax operation, and let 𝐚̃ = (ã_1, …, ã_n) denote the attention values after applying the 2D Gaussian blur. Let 𝐇 denote the Hessian of the original energy, i.e., the derivative of the negative softmax, and 𝐇̃ denote the Hessian of the underlying energy associated with the blurred weights.

The elements in the i-th row and j-th column of the Hessian matrices are given by:

    h_ij = (ξ(𝐚)_i - δ_ij)ξ(𝐚)_j,


    h̃_ij = (ξ(𝐚̃)_i - δ_ij)ξ(𝐚̃)_j b_ij,

respectively, where b_ij are the elements of the Toeplitz matrix corresponding to the Gaussian blur kernel, and δ_ij denotes the Kronecker delta.

Assuming ξ(𝐚̃)_i ξ(𝐚̃)_j ≈ 0 and ξ(𝐚)_i ξ(𝐚)_j ≈ 0 for all i and j, which is a reasonable assumption when the number of token is large, the non-diagonal elements of the Hessians approximate to 0 and the diagonal elements dominate. Therefore, the determinants of the Hessian matrices are approximated as the product of the dominant terms:

    |(𝐇)| ≈∏_i=1^n ξ(𝐚)_i,    |(𝐇̃)| ≈∏_i=1^n ξ(𝐚̃)_i b_ii


We have the following inequality:

    ∏_i=1^n ξ(𝐚̃)_i b_ii < ∏_i=1^n ξ(𝐚̃)_i    = e^∑_j=1^nã_j/(∑_j=1^n e^ã_j)^n
       ≤e^∑_j=1^na_j/(∑_j=1^n e^a_j)^n = ∏_i=1^n ξ(𝐚)_i,

where the first inequality follows from the property of the Gaussian blur kernel, 0 ≤ b_ii < 1, and the second inequality is derived from Lemmas <ref> and <ref>, which demonstrate the mean-preserving property and the decrease in the lse value when applying a blur. The monotonicity of the logarithm function implies that the denominator involving the blurred attention weights is smaller. Eventually, we obtain the following inequality:

    |(𝐇̃)| < |(𝐇)|.

This implies that the updated value is derived with attenuated curvature of the energy function underlying the blurred softmax operation compared to that of the original softmax operation.






§ DUAL DEFINITION



As we previously stated in Section <ref>, we have the dual definition regarding (<ref>), where we use swapped indexing. Importantly, the swapped indices can be interpreted as altering the definition of attention weights to 𝐀:=𝐊𝐐^⊤.

A similar conclusion can be drawn as in the main paper, except that query blurring becomes key blurring with this definition. To see this, Eq. <ref> changes slightly with this definition, using the symmetry of the Toeplitz matrix 𝐁:

    G ∗ (𝐊𝐐^⊤)    = 𝐁(𝐊𝐐^⊤) 
       = ((𝐊𝐐^⊤)^⊤𝐁^⊤)^⊤
       = (𝐐𝐊^⊤𝐁^⊤)^⊤
       = (𝐐(𝐁𝐊)^⊤)^⊤
       = (𝐐(G ∗𝐊)^⊤)^⊤
       = (G ∗𝐊)𝐐^⊤,

where ∗ denotes the 2D convolution operation. Empirically, this altered definition does not introduce a significant difference in the overall image quality, as shown in Fig. <ref>.




§ ADDITIONAL QUALITATIVE RESULTS


In this section, we present further qualitative results to demonstrate the effectiveness and versatility of our Smoothed Energy Guidance (SEG) method across various generation tasks and in comparison with other approaches.



  
Comparison with previous methods

Figs. <ref> and <ref> provide a qualitative comparison of SEG against vanilla SDXL <cit.>, Self-Attention Guidance (SAG) <cit.>, and Perturbed Attention Guidance (PAG) <cit.>. These comparisons highlight the superior performance of SEG in terms of image quality, coherence, and adherence to the given prompts. SEG consistently produces sharper details, more realistic textures, and better overall composition compared to the other methods.



  
Conditional generation with ControlNet

Figs. <ref> and <ref> showcase the application of SEG in conjunction with ControlNet <cit.> for conditional image generation. These results illustrate how SEG can enhance the quality and coherence of generated images while maintaining fidelity to the provided control signals. The images demonstrate improved detail, texture, and overall visual appeal compared to standard ControlNet outputs without prompts.



  
Unconditional and text-conditional generation

Fig. <ref> demonstrates the capability of SEG in unconditional image generation, showcasing its ability to produce high-quality, diverse images without text prompts. Fig. <ref> exhibits text-conditional generation results using SEG, illustrating its effectiveness in translating textual descriptions into visually appealing and accurate images.



  
Interaction with classifier-free guidance

Figs. <ref>–<ref> present a series of experiments exploring the combination of SEG with CFG. In these experiments, the SEG guidance scale (γ_seg) is fixed at 3.0, while the CFG scale is varied. The results demonstrate that SEG consistently improves image quality across different CFG scales without causing saturation or significant changes in the general structure of the images.














































 Paper Title: Operator-based semantics for choice programs: is choosing losing? (full version).
























 




















proofNestedLemma[1][]
  
  
  


arrows,positioning,shapes.misc,shapes.geometric
arrows.meta   
>=stealth',args/.style=circle,draw=black,minimum size=10mm







exampleExample
instantiationInstantiation
definitionDefinition
lemmaLemma
theoremTheorem
problemProblem
propositionProposition
extrapropositionExtra Proposition
factFact
corollaryCorollary
notationNotation
conventionConvention
remarkRemark
conjectureConjecture

propositionApProposition


























OMSzplmmn














































Operator-based semantics for choice programs: is choosing losing? (full version)
    
Jesse Heyninck


Open Universiteit, the Netherlands

University of Cape Town, South-Africa

    August 5, 2024
================================================================================================



§ ABSTRACT 
Choice constructs are an important part of the language of logic programming, yet the study of their semantics has been a challenging task. So far, only two-valued semantics have been studied, and the different proposals for such semantics have not been compared in a principled way. In this paper, an operator-based framework allow for the definition and comparison of different semantics in a principled way is proposed.




§ INTRODUCTION

Logic programming is one of the most popular declarative formalisms, as it offers an expressive, rule-based modelling language and efficient solvers for knowledge representation. An important part of this expressiveness comes from choice constructs <cit.>, that allow to state e.g. set constraints in the body or head of rules, and are, among others, part of the ASP-Core-2 standard <cit.>. For example, the rule 1{p,q,r} 2← s. expresses that if s is true, between 1 and 2 atoms among p, q and r are true. Choice constructs are non-deterministic, in the sense that there is more than one way to satisfy them. For example,  1{p,q,r} 2 can be satisfied by {p}, {p,q}, {r}, …. Formulating semantics for such non-deterministic rules has proven a challenging task <cit.>: several semantics have been proposed but no unifying framework for defining and comparing these semantics exists. Furthermore, attention has been restricted to two-valued semantics, in contradistinction to many other dialects for logic programming for which three- or four-valued semantics have been proposed. Moreover, many proposed semantics only allow for choice constructs in the body, but not in the head. Finally, relations with the non-deterministic construct disjunction remain unclear. 

In this paper, a unifying framework for the definition and study of semantics for logic programs with choice constructs in the head and body is provided.  This framework is based on immediate consequence operators, which are also useful for the design of explanatory tools and provide foundations for solvers <cit.>.








The contributions of this paper are the following: (1) we show how the famework of non-deterministic approximation fixpoint theory (AFT)  <cit.>  can be used to define a wide variety of supported and stable semantics for choice logic programs, (2) we introduce the constructive stable fixpoints, allowing to (3) generalize many existing semantics for choice programs, (4) compare these semantics by introducing postulates, (5) provide a principled comparison with disjunctive logic programs.







Outline of the Paper:
 In Section <ref>, the background on choice programs and  non-deterministic approximation fixpoint theory is given. In Section <ref>, approximation operators for choice programs are defined. In Section <ref>, we study the resulting supported semantics. In Section <ref>, we define stable semantics and show representation results, while giving a postulate-based study  in Section <ref>. In Section <ref>, we relate choice constructs and disjunctions. Related work and a conclusion follows in Sections <ref> and <ref>.





Relation with Previous Work: 

 This paper extends this previous work <cit.> by also considering choice construct in the body, which gives rise to different possible approximators, and comparing these operators using the notion of groundedness.




§ BACKGROUND AND PRELIMINARIES


We recall choice programs  and non-deterministic AFT.



 §.§ Choice Rules and Programs

A choice atom (relative to a set of atoms 𝒜) is an expression C=(,) where ⊆𝒜 and ⊆℘(). Intuitively,  denotes the domain of C, i.e. the atoms relevant for the evaluation of C, whereas  is the set of satisfiers of C. We also denote, for C=(,),  by (C) and  by (C).
For a concrete example, consider 1 {p,q,r} 2 which intuitively states that between 1 and 2 of the atoms p, q and r have to be true, corresponds to the choice atom ({p,q,r},{{p},{q},{r},{p,q},{p,r}, {q,r}}) (notice that {p,q,r} is the domain and not a satisfier). For such choice atoms, we assume the domain and satisfiers are clear and can be left implicit (and similarly for constructs such as {p,q}=1 or {p,q}≠ 1).



A choice rule has the form C← C_1,…,C_n where  C,C_1,…,C_n are choice atoms.

 C is called the head (denoted (r)) and C_1,…,C_n the body. If C_i is a literal (i.e. C_i=({α},{{α}}), abbreviated by α, or  C_i=({α},{∅}), abbreviated by ¬α) for every i=1,…,n, we call it a normal choice rule. If C is an atom, we call r an aggregate rule.
 A choice program is a set of choice rules, and is called normal[aggregate] if all of the rules are so. A choice atom C is monotone if  (C)∩ x∈(C) implies (C)∩ x'∈(C) for any x⊆ x'⊆𝒜, and it is convex if for any s.t. x⊆ y⊆𝒜, (C)∩ x∈(C) and (C)∩ y∈(C), (C)∩ z∈(C) for any x⊆ z⊆ y.








Following <cit.>, a set x⊆𝒜 satisfies a choice atom C if (C)∩ x∈ sat(C).  An interpretation x satisfies a rule r if x satisfies the head of r or does not satisfy some choice atom in the body of r. x is a model of 𝒫 if it satisfies every rule in 𝒫. A rule r∈𝒫 is x-applicable if x satisfies the body of r, and the set of x-applicable rules in 𝒫 is denoted by 𝒫(x). 

x⊆𝒜 is a  supported model of 𝒫 if it is a model and x⊆⋃_r∈𝒫(x)((r)).


For some x⊆𝒜, let 

 _𝒫(x)={(r)| r∈𝒫(x)} and  _𝒫(x)=

    {z⊆⋃_C∈_𝒫(x)(C)|∀ C∈_𝒫(x):z(C)= T}























Consider the program 𝒫={1{p,q} 2←{p,q}≠ 1}. 
The choice atoms behave as follows:




     ∅     {p}     {q}     {p,q}
 1{p,q}2     F      T      T      T

  {p,q}≠ 1     T      F      F      T




 This means that _𝒫(∅)=_𝒫({p,q})={{p},{q},{p,q}}, whereas _𝒫({p})=_𝒫({q})={∅}. The models of 𝒫 are {p}, {q} and {p,q}, and only {p,q} is supported.





 §.§ Approximation Fixpoint Theory


We first  recall some basic algebraic notions. 


A lattice is a partially ordered set (poset) ⟨ℒ,≤⟩ s.t. for every x,y∈ℒ, a least upper bound x⊔ y and a greatest lower bound x⊓ y exist. A lattice is complete if every X⊆ℒ has a least upper bound X and a greatest lower bound X.

ℒ is denoted by ⊤ and ℒ is denoted by .

A function f:X→ Y from a poset ⟨ X,≤_1⟩ to a poset ⟨ Y,≤_2⟩ is monotonic if x_1≤_1 x_2 implies f(x_1)≤_2 f(x_2), and x∈ X is a fixpoint of f if x=f(x)

 We define the ordinal powers of a function f: X→ X as f^0(x)=x, f^α+1(x)=f(f^α(x)) for a successor ordinal α, and   f^α(x)=_β<αf^β(x) for a limit ordinal α. The following notation will be used: [x,y]={z| x≤ z≤ y}. We say a pair (x,y) is consistent if x≤ y and total if x=y.
 
We now recall basic notions from non-deterministic approximation fixpoint theory (AFT) by <cit.>, which generalizes approximation fixpoint theory as introduced by <cit.> to non-deterministic operators. We refer to the original paper <cit.> for more details. 

 The basic idea behind non-deterministic approximation operators is that we approximate a non-deterministic operator O by generating, for a given lower bound x and upper bound y that approximates z, a set of lower bounds {x_1,x_2,…} and a set of upper bounds {y_1,y_2,…} that under- respectively over-approximate O(z)={z_1, z_2,…}.
Formally, a non-deterministic operator on ℒ is a function O : ℒ→℘(ℒ) ∖{∅}. 
 For example, the operator _𝒫 is a non-deterministic operator on the lattice ℘(𝒜),⊆.
As the ranges of non-deterministic operators are sets of lattice elements, one needs a way to compare them, such as the Smyth order and the Hoare order.
Let L = ℒ,≤ be a lattice, and let X,Y ∈℘(ℒ). Then: X ≼^S_L Y if for every y∈ Y there is an x∈ X such that x≤ y; and X ≼^H_L Y if for every x∈ X there is a y∈ Y such that x≤ y.              Given some X_1,X_2,Y_1,Y_2⊆ℒ, X_1× Y_1 ≼^A_i X_2× Y_2 iff X_1≼^S_L X_2 and Y_2≼^H_L Y_1; and X_1× Y_1 ≼^S_t X_2× Y_2 iff X_1≼^S_L X_2 and Y_2≼^S_L Y_1. 
The main orders, instantiated for the lattice of interest for this paper, ⟨𝒜,⊆⟩ are recalled in Table <ref>.










Let L=ℒ,≤ be a lattice.
Given an operator 𝒪:ℒ^2→ℒ^2, we denote by 𝒪_l  the projection operator defined by 𝒪_l(x,y)=𝒪(x,y)_1, and similarly for 𝒪_u(x,y)=𝒪(x,y)_2.
 An operator 𝒪:ℒ^2→℘(ℒ)∖∅×℘(ℒ)∖∅ 
is called a non-deterministic approximating operator (ndao, for short), if it is ≼^A_i-monotonic (i.e. (x_1,y_1)≤_i (x_2,y_2) implies 𝒪(x_1,y_1)≼^A_i 𝒪(x_2,y_2)),
and  is exact (i.e., for every x∈ℒ, 𝒪(x,x)=(𝒪_l(x,x), 𝒪_l(x,x))). (x,y) is a fixpoint of 𝒪 if x∈𝒪_l(x,y) and y∈𝒪_u(x,y).
A non-deterministic operator O:ℒ→℘(ℒ) is downward closed if for every sequence X={x_ϵ}_ϵ<α of elements in ℒ such that:
(1) for every ϵ<α, O(x_ϵ)≼^S_L {x_ϵ}, and
(2) for every ϵ <ϵ'<α, x_ϵ'< x_ϵ,
it holds that O( X)≼^S_L  X.





We also introduce the following generalisation of an approximation operator, as it will allow us to capture some further existing semantics.

An operator 𝒪:ℒ^2→℘(ℒ)∖∅×℘(ℒ)∖∅  is a semi-ndao iff:

(1) it is exact,
(2) 𝒪_l(·,y) is ≼^S_L-monotonic for any y∈ℒ, and 
(3) 𝒪_u(x,·) is ≼^H_L-anti-monotonic for any  x∈ℒ.


Any ndao is a semi-ndao (cf.  Lemma 1 by <cit.>). A (semi-)ndao 𝒪 approximates an operator O if 𝒪_l(x,x)=O(x,x) for every x∈ℒ. 

We recall now stable operators (given an ndao 𝒪).
The complete lower stable operator is defined by (for any y∈ℒ) C(𝒪_l)(y)  = 
 {x ∈ℒ| x∈𝒪_l(x,y) ¬∃ x'< x: x'∈𝒪_l(x',y)}.
The complete upper stable operator is defined by  (for any x∈ℒ) C(𝒪_u)(x)  = {y ∈ℒ| y∈𝒪_u(x,y) ¬∃ y'<y:  y' ∈𝒪_u(x,y')}.
 The stable operator is given by: S(𝒪)(x,y)=(C(𝒪_l)(y),C(𝒪_u)(x) ). (x,y) is a stable fixpoint of 𝒪 if (x,y)∈ S(𝒪)(x,y).

Other semantics (e.g. well-founded state, Kripke-Kleene fixpoints and state and semi-equilibrium semantics) have been defined (<cit.>) and can be immediately obtained once an ndao is formulated. Due to space limitations, they are not discussed or studied here.



































§ APPROXIMATION OPERATORS FOR CHOICE PROGRAMS

The central task is to define non-deterministic approximations of the immediate consequence operator _𝒫. As is usual, we conceive of pairs of sets of atoms (x,y) as four-valued interpretations, where atoms in x are true whereas those in y are not false. Thus, assuming x⊆ y, (x,y) represents an approximation of some set z∈[x,y].

The basic idea behind all the operators defined below is the same: given an input interpretation (x,y), we determine a set of rules that are to be taken into account when constructing the new lower (respectively upper bound), and then take as new lower (respectively upper bounds) the interpretations that make true the heads of all these rules.  As is well-known already in the case for aggregate programs, there are various ways to  give formal substance to this idea. 
We will consider four operators inspired by previous work on aggregate or choice programs, namely ^ GZ_𝒫 <cit.>,  ^ MR_𝒫 <cit.>,  ^ LPST_𝒫 <cit.>, and  ^𝒰_𝒫 <cit.>.
The study of further operators is left for future work.
Intuitively, the GZ-operator takes into account only rules of satisfied bodies whose domain remains unchanged in x and y. The lower bound of the LPST-operator takes into account heads of rules whose body is true in every interpretation between x and y, whereas the upper bound collects the results of applying _𝒫 to every “non-false” interpretation (i.e. every member of [x,y]).
 The lower bound of the  MR-operator  looks at all rules whose body is satisfied by the upper bound y and by some subset of the lower bound x, whereas its upper bound is identical to the LPST-operator upper bound. The 𝒰-operator, finally, has as a lower and upper bound the LPST-operator upper bound.


Given a choice program 𝒫 and pair of sets of atoms (x,y), let:
  
    ^ GZ,l_𝒫(x,y)={  C|∃ C← C_1,…,C_i∈𝒫, ∀ i=1… n:  
        x∩(C_i)=y∩(C_i)∈(C_i)  }, 
       ℋ𝒟^ LPST,l_𝒫(x,y)={C|∃ C← C_1,…,C_n∈𝒫, ∀ i=1… n: 
       ∀ z∈[x,y]: z(C_i)= T},
    
       ℋ𝒟^ MR,l_𝒫(x,y)={ C|∃  C← C_1,…,C_n ∈𝒫,  ∃ z⊆ x:
       ∀ i=1… n: y(C_i)= T z(C_i)= T},


 
 We then define (for x∈{ LPST,  MR, GZ})  the lower bound operators by ^ x,l(x,y)=
 
    {z⊆⋃_ C∈^ x,l_𝒫(x,y)(C)|∀ C∈^ x,l_𝒫(x,y):z∩(C) ∈ (C)}
 
Furthermore, we define (for †∈{ MR, LPST, 𝒰}): 
  
    ^𝒰,l_𝒫(x,y)    = ^†,u_𝒫(x,y)    =⋃_x⊆ z⊆ y_𝒫(z)


 whereas ^ GZ,u_𝒫(x,y)=^ GZ,l_𝒫(x,y).
Finally, we define  (for x∈{ LPST,  MR, GZ,𝒰}):
 
    ^ x(x,y)=(^ x,l(x,y),^ x,u(x,y)).

We also provide a handy summary of all operators in Table <ref>.
























We first illustrate the behaviour of these operators:

Consider again 𝒫 from Example <ref>.
We will consider the three-valued interpretation ({p},{p,q}). 
We observe that:


  * ^ MR,l_𝒫({p},{p,q})={{p},{q},{p,q}},

  * ^ LPST,l_𝒫({p},{p,q})= ^ GZ,†_𝒫({p},{p,q})={∅}  (for †=l,u), and

  * ^𝒰,l_𝒫({p},{p,q})=^𝒰,u_𝒫({p},{p,q}={∅,{p},{q},{p,q}}.



 Thus, ({p},{p,q}) is a fixpoint of the MR- and 𝒰-operators, but not of the LPST and GZ-operators.




Unsurprisingly, these ndaos are not well-defined for every program (inevitably so, as this already holds for _𝒫):

Let 𝒫={{p,q}≠ 2←; {p,q}=2←}. For this program, no set satisfying both heads exists, and thus ^ x_𝒫(x,y) is not defined (for any x,y⊆𝒜). 

We therefore making the following

Assumption:
We restrict attention to programs for which _𝒫^ x,†(x,y)≠∅ for any x,y⊆𝒜, x∈{ LPST,  MR, GZ,𝒰} and †∈{l,u}.




All operators are (semi-)approximators of _𝒫:[Proof of all results in the paper, as well as additional results can be found in the appendix.]

  ^ LPST_𝒫 and ^ MR_𝒫 are ndaos for _𝒫. ^ MR_𝒫 is a semi-ndao for _𝒫. ^ GZ_𝒫 is an ndao for _𝒫 when restricted to consistent inputs.


Consider an arbitrary but fixed choice program 𝒫.


  x=𝒰 Follows from Proposition 1 by <cit.>.

  x= LPST

For ≼^A_i-monotonicity, consider some (x_1,y_1)≤_i(x_2,y_2). We first show that †: ^l, LPST_𝒫(x_1,y_1)⊆^l, LPST_𝒫(x_2,y_2)

Suppose that C∈^l, LPST_𝒫(x_1,y_1), i.e. there is some C← C_1,…,C_n∈𝒫 s.t. for every z∈[x_1,y_1] and every i=1,…,n, z(C_i)= T. Then, since [x_2,y_2]⊆ [x_1,y_1], it also holds that  for every z∈[x_2,y_2] and every i=1,…,n, z(C_i)= T. We have thus shown that ^l, LPST_𝒫(x_1,y_1)⊆^l, LPST_𝒫(x_2,y_2).  


To show that   ^ LPST,l_𝒫(x_1,y_1)≼^S_L^ LPST,l_𝒫(x_2,y_2),
consider now some z_2∈^ LPST,l_𝒫(x_2,y_2). We show that z_2∩⋃{(C)| C∈^ LPST,l_𝒫(x_1,y_1)}∈^ LPST,l_𝒫(x_1,y_1). Indeed, consider some C∈^ LPST,l_𝒫(x_1,y_1). Then C∈^ LPST,l_𝒫(x_2,y_2) (with †) and thus (C)∩ z_2∈ sat(C). As z_2∩⋃{(C)| C∈^ LPST,l_𝒫(x_1,y_1)}⊆ z_2, this concludes the proof of ≼^S_L-monotonicity.  
≼^H_L-monotonicity of the upper bound ^ LPST,u_𝒫 follows immediately from the previous item as ^ LPST,u_𝒫=^ U,u_𝒫.











It can be easily observed that ^ LPST,l_𝒫(x,x)=^ LPST,u_𝒫(x,x)=_𝒫(x), which demonstrates exactness and that _𝒫 approximates _𝒫.



  x= MR For ≼^S_L-monotonicity of ^ MR,l_𝒫(·,y), consider some x_1⊆ x_2⊆ y⊆𝒜. We show that  ^l, MR_𝒫(x_1,y)⊆^l, LPST_𝒫(x_2,y), the proof of ≼^S_L-monotonicity is identical to that given in the previous item. Suppose thus that C∈^l, MR_𝒫(x_1,y), i.e. there is some C← C_1,…,C_n s.t. y(C_i)= T and there is some x⊆ x_1 s.t. x(C_i)= T (for every i=1,…,n). But then also x⊆ x_2 and thus C∈^l, MR_𝒫(x_2,y). ≼^H_L-monotonicity of the upper bound was established in the previous item.  Exactness and approximation of _𝒫 are immediate.

  x= GZ  For ≼^S_L-monotonicity, consider some  (x_1,y_1)≤_i(x_2,y_2) and x_2⊆ y_2 (recall that we restrict the input of the GZ-operator to consistent pairs). We show that ^l, GZ_𝒫(x_1,y_1)⊆^l, GZ_𝒫(x_2,y_2), the rest can be derived analogously to the first item. Consider thus some C∈^l, GZ_𝒫(x_1,y_1), which means that there is some C← C_1,…,C_n∈𝒫 s.t. x_1∩(C_i)=y_1∩(C_i)∈(C_i) for every i=1,…,n. As x_1⊆ x_2, we see that x_1∩(C_i)⊆ x_2∩(C_i) (and similarly we see that y_2∩(C_i)⊆ y_1∩(C_i)). As we restricted ourselves to consistent pairs (and thus x_2⊆ y_2), we see that x_1∩(C_i)= x_2∩(C_i)=y_2∩(C_i)= y_1∩(C_i). Exactness and approximation of _𝒫 are immediate.




The following example shows that ≼^A_i-monotonicity of _𝒫^ GZ is not guaranteed when considering inconsistent interpretations:

Consider 𝒫={p←{p,q}≠ 0}. We see that 

    ({p},{p})≤_i ({p,q},{p})  
       _𝒫^ GZ,l({p},{p})={{p}}⋠^S_L  _𝒫^ GZ,l({p,q},{p})={∅}


To see that _𝒫^ GZ,l({p,q},{p})={∅}, observe that ( {p,r}≠ 0)∩{p}≠( {p,r}≠ 0)∩{p,q}.


















Furthermore, the LPST and MR-operators coincide for normal choice programs:

For any normal choice program,  ℐ𝒞^ MR_𝒫(x,y)= ℐ𝒞^ LPST_𝒫(x,y). 


We have to show that ℋ𝒟^l, MR(x,y)=ℋ𝒟^l, LPST(x,y) for a normal choice program 𝒫 (the upper bound is already identical by definition), which can be seen by observing that for any atom α, z(α)= T for every z∈[x,y] iff α∈ x, and for any negated literal ¬α, z(¬α)= T for every z∈[x,y] iff α∉y.



It can be easily verified that the  LPST and MR-operators coincide with the four-valued Przymusinski operator for normal logic programs <cit.>, which implies, with Proposition 3, Proposition 11 and Theorem 6 by <cit.>, that the semantics based on these operators faithfully generalize the (partial) supported, (partial) stable and well-founded semantics of normal logic programs.

















§ SUPPORTED MODEL SEMANTICS

In this section, we look at the fixpoints of ^ x_𝒫, which give a three-valued generalisation of the supported model semantics by <cit.> (and the three-valued model semantics of normal logic programs). Intuitively, fixpoints of ^ x_𝒫 generalise the idea that only atoms supported by activated rules can be accepted.

A first insight is that the total fixpoints of all operators coincide with supported models  <cit.>.

Let a choice program 𝒫 and x∈{ LPST,  MR, GZ,𝒰}  be given. Then (x,x) is a fixpoint of ^ x_𝒫 iff x is a supported model of 𝒫.


Notice first that, as ^ x_𝒫 approximates _𝒫 (Proposition <ref>),  ^ x_𝒫(x,x)=_𝒫(x) for any x⊆𝒜.
For the ⇒-direction, suppose that x∈^ x,l_𝒫(x,x)= ^c_𝒫(x). We first show x is a model of 𝒫. Indeed, consider a rule C← C_1,…,C_n s.t. x satisfies C_i for i=1,…,n. Then x(C_i)= T and thus C∈^ x,l_𝒫(x,x)=_𝒫(x), which implies, since x∈^c,l_𝒫(x,x), that  x∩(C)∈(C). That x is supported follows immediately from the fact that  x∈_𝒫(x) implies z⊆⋃_ C∈^l_𝒫(x)(C).
The ⇐-direction is analogous.

However, our extension to three-valued semantics allows to give semantics to a wider class of programs:

Let 𝒫={{p,q}=1←{p}≠ 1; p← q. }. It can be easily checked that this program has no two-valued supported model. However, (∅,{p}) is a fixpoint of ^ x_𝒫 for x∈{ LPST, MR,𝒰}.

A more detailed investigation of conditions for the existence of fixpoints  is left for future work.








We now show that fixpoints of all four operators satisfy a notion of supportedness, relative to how rule bodies are evaluated according to their definition. For example, C can be said to be true in (x,y) according to the LPST-operator if z(C)= T for every z∈ [x,y]. We formalize this as follows. Given (x,y) and ℐ𝒞^ x_𝒫, a∈ x supported if a occurs in the domain of the head of a rule whose body is true, i.e. there is some C← C_1,…,C_n s.t. a∈(C), and if would replace C by the dummy atom p, p is the only consequence of ^ x,l_{p← C_1,…,C_n}(x,y) (and similarly for y).


Let a choice program 𝒫 and x∈{ LPST,  MR, GZ,𝒰}  be given. Then (x,y)∈^ x_𝒫(x,y) implies that for every a∈ y, there is some C← C_1,…,C_n with a∈(C) s.t. 

 ^ x_{p← C_1,…,C_n}(x,y)=({{p}∩ x},{{p}∩ y}).


We show this for x=𝒰,  LPST, the proof of the other cases are analogous.


  x=𝒰 Suppose that (x,y)∈^𝒰_𝒫(x,y) and p∈ y (and thus, also p∈ x as x=y by definition of ^𝒰_𝒫).  This means that there is some z∈ [x,y] s.t. p∈(C) for some C←ϕ for which z(ϕ)= T. But then ^𝒰_{p←ϕ}(x,y)=({{p}},{{p}}).


  x= LPST Suppose that (x,y)∈^ LPST_𝒫(x,y) and p∈ y. The case for the upper bound follows from the previous case. Suppose thus that p∈ x.
By definition of ^ LPST,

p∈⋃_C∈_𝒫^ LPST,l(x,y)(C), which means that there is some C← C_1,…,C_n s.t. for every i=1,…,n, and for every 
for every z∈[x,y], z(C_i)= T. But then  ^ x,l_{p← C_1,…,C_n}(x,y)={{p}}.
















  
*Normal choice programs
We can simplify the definition of supported models for normal choice programs. In order to do this, we first have to generalize the four-valued truth-assignments to choice constructs. The following forms a generalization of assignment of truth-values to choice constructs that forms a natural generalization of the assignment of atoms to choice constructs.
We first recall the bilattice FOUR,  
consisting of the elements T (true), F (false), U (undecided) and C (contradictory) and two order relations ≤_i and ≤_t:


0.2!
[node distance=1cm, auto, >=latex, scale=0.5]
		(a) ;
		(b) [above of=a, yshift=2.5cm] ≤_i;
		(c) [right of=a, xshift=2.5cm] ≤_t;
		[->] (a.center) – (b);
		[->] (a.center) – (c);
		dot= [circle, fill, minimum size=4pt,inner sep=0pt, outer sep=0pt]
		(third) [dot,above right of=a, node distance=1.2cm, xshift=1cm, label=below:U] ;
		(second) [dot,above left of=third, label=left:F] ;
		(fourth) [dot,above right of=third, label=right:T] ;
		(first) [dot, above right of= second,label=above:C] ;	

		[color=black] (second) edge (first);
		[color=black] (third) edge (second);
		[color=black] (fourth) edge (third);
		[color=black] (first) edge (fourth);
	
	


Given a choice construct C and an interpretation (x,y), we say that:
 

  * (x,y)(C)= T if x∩(c)∈ (C) and y∩(c)∈ (C);

  * (x,y)(C)= F if x∩(c)∉ (C) and y∩(c)∉ (C); 

  * (x,y)(C)= C if x∩(c)∈ (C) and y∩(c)∉ (C),

  * (x,y)(C)= U if x∩(c)∉ (C) and y∩(c)∈ (C).


We define three-valued models of a normal choice program 𝒫 as consistent interpretations (x,y) for which _≤_t{(x,y)(C_i)| i=1… n}≤_t (x,y)(C) for every C← C_1,…,C_n ∈𝒫, and supported[These models have been called weakly supported models <cit.> for disjunctive logic programs.] models as models (x,y) of 𝒫 s.t. for every p ∈  y, there is a rule C← C_1,…,C_n∈𝒫 with p∈(C) and _≤_t{(x,y)(C_i)| i=1… n}≥_t (x,y)(p). I.e., a model is supported if for every non-false atom p, we have a reason in the form of an activated rule  for accepting (or not rejecting) that atom.

Three-valued supported models of 𝒫 coincide with fixpoints of ^ LPST_𝒫 and ^ MR_𝒫, whereas those of ^ GZ_𝒫 are a subset of the three-valued supported models:

Let some normal choice program 𝒫 and x∈{ LPST, MR} be given.  Then  (x,y) is a three-valued supported model of 𝒫 iff (x,y) is a fixpoint of ^ x_𝒫. Furthermore, if (x,y) is a fixpoint of ^ GZ_𝒫 then (x,y) is supported, but not always vice-versa. Supported models might not be fixpoints of _𝒫^𝒰 and vice-versa.


Let 𝒫 be an arbitrary but fixed normal choice program.



  x= LPST, MR:
For the ⇒-direction, suppose that (x,y) is a three-valued supported model of 𝒫. As for every p ∈  y,   there is a rule C←ϕ∈𝒫 such that p∈(C)∩ x and (x,y)(ϕ)≥_t (x,y)(p), we see that x⊆⋃_C∈^ x,l_𝒫(x,y)(C) (and similarly for y). That for every C←ϕ∈^c,l_𝒫(x,y), (C)∩ x∈(C) follows from (x,y) being a model of 𝒫 (and similarly for y).
For the ⇐-direction, suppose (x,y)∈^ x_𝒫(x,y). That (x,y) is a model follows straightforwardly from the definition of ^ x_𝒫(x,y). Consider now some p∈ y. As y⊆⋃_C∈^ x,u_𝒫(x,y)(C), there is some C←⋀_i=1^nα_i⋀_i=1^m¬β_i∈𝒫 s.t. p∈(C)∩ y (and similarly for x) and α_i∈ y for i=1… n and β_j∉x for j=1… m.


  x= GZ: Suppose that (x,y)∈^ GZ_𝒫(x,y). That (x,y) is a model follows straightforwardly from the definition of ^ GZ_𝒫(x,y).  Consider now some p∈ y. As y⊆⋃_C∈^ GZ,u_𝒫(x,y)(C), there is some C←⋀_i=1^nα_i⋀_i=1^m¬β_i∈𝒫 s.t. p∈(C)∩ y (and similarly for x) and α_i∈  y for i=1… n and β_j∉x for j=1… m. Thus (x,y) is a supported model.

Supported models might not be fixpoints of the GZ-operator:
For this, consider 𝒫={p←¬ q; q←¬ p}. Then (∅,{p,q}) is a supported model. Yet as ∅∩{q}≠{p,q}∩{q} (and similarly for p), ^ GZ_𝒫(∅,{p,q})=(∅,∅).


  x=𝒰:   Supported models might not be fixpoints of the 𝒰-operator:
Consider the program 𝒫={p← q; q←¬ q}. Then (∅,{p,q}) is a supported model. However, as ⋃_z⊆{p,q}_𝒫(z)={{p,q}}, we see that ∅∉^l,𝒰_𝒫(∅,{p,q}). 

Fixpoints of the 𝒰-operator might not be supported models:
Consider the program 𝒫={ 1{p,q}← p,¬ q}. Then _𝒫^𝒰({p},{p,q})=({{p},{q},{p,q}},{{p},{q},{p,q}}) and thus ({p},{p,q}) is a fixpoint. However, it is not supported, as ({p},{p,q})(¬ q)= U yet ({p},{p,q})(p)= T (i.e. we cannot find any support for p being true).






  
*Models as pre-fixpoints
It is well-known that for many non-monotonic formalisms, pre-fixpoints of an operator can characterise models of the corresponding knowledge base. For the general case of choice constructs, this correspondence does not hold:

Consider 𝒫={{p,q}=1←}. Then ({p,q},{p,q}) is a pre-fixpoint of _𝒫 (as _𝒫({p,q},{p,q})={{p},{q}}×{{p},{q}}≼^S_t ({p,q},{p,q})) yet it is not a model (as the only models are {p} and {q}): there is no way to prove p and q.

For choice programs with monotone heads, this correspondence does hold:


Let some choice program 𝒫 s.t. for every C← C_1,…,C_n, C is monotone and some x∈{ LPST,  MR,  GZ, 𝒰} be given. Then  x is a model of 𝒫 iff ^l, x_𝒫(x,x)≼^S_L x. 



Suppose that 𝒫 is a choice program with monotone choice constructs in the head. For the ⇒-direction, suppose that x is a model of 𝒫. 
We show that x∩⋃_C∈(x)_𝒫(x)(C)∈_𝒫(x)=^ x,l_𝒫(x,x) for any x∈{ LPST,  MR,  GZ, 𝒰} (Notice that _𝒫(x)=^ x,l_𝒫(x,x) holds in view of Proposition <ref>), which implies ^ x,l_𝒫(x,x)≼^S_t x. Indeed, consider some C∈(x)_𝒫(x), i.e. there is some C← C_1,…,C_n ∈𝒫 with (C_i)∩ x∈(C_i) for i=1,…,n. As x is a model, also (C)∩ x∈(C). Thus x∩(C)∈_𝒫(x). The ⇐-direction is similar.










§ STABLE SEMANTICS

We now move to the stable semantics, whose main aim is favvoding the acceptance of self-supporting cycles, e.g. accepting ({p},{p}) as a reasonable model of the program {p← p}. For the deterministic case, this is done by looking at fixpoints of the stable operator, obtained by calculating a new lower bound as the least fixpoint of 𝒪_l(·,y), where y is the input upper bound (and similarly for the upper bound). Intuitively, we take the least information the upper bound obliges use to derive.  In contradistinction to the deterministic case, there are divergent options for how to define stable semantics for non-deterministic operators.
We first consider the minimality-based stable semantics known from non-deterministic AFT <cit.>, defined as the ≤-minimal fixpoints of 𝒪_l(.,y) (cf. Section <ref>), which where shown to generalize the (partial) stable model semantics for disjunctive (aggregate) programs  <cit.>. For choice constructs, this construction is overly strong:

Consider the program 𝒫={ 1 {p,q} 2←}. Intuitively, this rule allows to choose between one and two among p and q. The stable  version of ^ x_𝒫 (cf. Section <ref>) behaves as follows (for any x= LPST, MR, GZ):

    S(^ x,l_𝒫)(x)={{p},{q}}x⊆𝒜

 {p} and {q} are the two stable fixpoints of ^ x_𝒫. This is undesirable, as according to the intuitive reading of 𝒫, {p,q} should also be allowed as a stable interpretation.


If we take one step back, we can explain the choice for minimal fixpoints, and their shortcomings in the context of choice constructs, in stable non-deterministic operators <cit.> as follows. For deterministic operators, the stable version of an approximation operator 𝒪 is defined as the greatest lower bound (glb) of fixpoints of 𝒪_l(.,y). For deterministic operators over finite lattices, the minimal fixpoint of 𝒪_l(.,y) are identical to the  glb of fixpoints of 𝒪_l(.,y), and it is also identical to the fixpoint obtained by iterating 𝒪_l(.,y) starting from  (i.e. ⋃_i=1^∞𝒪_l^i(,y)). For non-deterministic operators, this correspondence does not hold. Indeed, the glb of fixpoints of 𝒪_l(.,x) is often too weak (e.g.  for the program 𝒫 from Example <ref> we get {p}∩{q}∩{p,q}=∅ as the glb of fixpoints). However, this still leaves a third choice: namely looking at fixpoints reachable by applications of 𝒪_l(.,y) starting from . 
E.g. for choice programs, we are interested in the fixpoints of ^ x,l_𝒫(.,y) that  can built them up from the ground up (i.e. from ∅) by a sequence of applications of ^ x,l_𝒫(.,y). 

We first generalize the notion of a well-founded sequence by <cit.>:

Given a non-deterministic operator O:ℒ→℘(ℒ) over a complete lattice, a sequence x_0,…,x_n⊆ℒ is well-founded relative to O if: (1) x_0=;
(2) x_i≤ x_i+1 and x_i+1∈ O(x_i) for every successor ordinal i≥ 0; and (3)  x_λ=({x_i}_i<λ) for a limit ordinal λ.
The well-founded sequences relative to O are denoted by wfs(O).




The assumption of a complete lattice in  Definition <ref> is needed since the greatest lower bound is used in point 3 of Definition <ref>.



Notice that, in contradistinction to the deterministic version of a well-founded sequence <cit.>, we require not merely that  x_i+1≼^S_L O(x_i) (or, in the case of deterministic operators O, x_i+1< O(x_i)) but  x_i+1∈ O(x_i). This is to ensure that x_i+1 can actually be constructed from x_i. For non-deterministic operators, this is not ensured by x_i+1≼^S_L O(x_i):

Let 𝒫={{p,q} = 2←}. If we would allow for x_i+1≼^S_L O(x_i) in Definition <ref>.(2), ∅, {p} would be a well-founded sequence according to ^l_𝒫(·,y) (for any y⊆{p,q}) as {p}≼^S ^l_𝒫({p},y)={{p,q}}. However, we have no way of deriving just p from the program 𝒫.


We now define the constructive stable operator:

Given an semi-ndao 𝒪 over a complete lattice ℒ with y∈ℒ, the c(onstructive)-complete lower bound operator is defined as:

    C^c(𝒪_l)(y)={x∈𝒪_l(x,y)|∃ x_0,..,x∈ wfs(𝒪_l(.,y))}

The c-complete upper bound operator C^c(𝒪_u) is defined analogously, and the c-stable operator is defined as S^c(𝒪)(x,y)=(C^c(𝒪_l)(y), C^c(𝒪_u)(x)).
 (x,y) is a c-stable fixpoint iff (x,y)∈ S^c(𝒪)(x,y).



[Example <ref> continued]
Consider again the program from Example <ref>. We see that (for any y⊆{p,q}), S^c(^ x,l_𝒫)(y)={{p},{q},{p,q}}, as ∅,{p}, ∅,{q} and ∅,{p,q} are all well-founded sequences. Thus, the total c-stable fixpoints of ^ x_𝒫 are ({p},{p}), ({q},{q}) and ({p,q},{p,q}) (for x= LPST, MR, GZ,𝒰).


It is easy to see that for a deterministic operator over a complete lattice, the c-stable operator coincides with the stable operator known from deterministic AFT.

The c-stable operator  generalizes the minimality-based one:

Let an (semi-)ndao 𝒪 over a complete lattice L=⟨ℒ,≤⟩ be given s.t. 𝒪_u(x,·) is ≼^S_L-monotonic for any x∈ℒ. Then C^c(𝒪_†)(y)⊇ C(𝒪_†)(y) for any y∈ℒ and †=l,u. Furthermore,  if  (x,y)∈ S(𝒪)(x,y) then  (x,y)∈ S^c(𝒪)(x,y).


We first show that (†): for any ≼^S_L-monotonic operator O, for every x∈min_≤{x∈ℒ| x∈ O(x)}, there is a well-founded sequence x_0,…,x_n,x relative to O.
Indeed, as O is ≼^S_L-monotonic, for any x_1≤ x, O(x_1)≼^S_L O(x)∋ x, i.e. there is some x_2∈ O(x_1) with x_2≤ x. This means we can define a ≤-monotonic operator O_s: {x∈ℒ| x'≤ x}→{x'∈ℒ| x'≤ x} by simply defining a selection function that selects some x_2∈ O(x_1) s.t. x_2≤ x for every x_1≤ x. We now show that {x '∈ℒ| x'≤ x} is a complete partially ordered set. Indeed, consider some X⊆{x'∈ℒ| x'≤ x}. As L is a complete lattice, X∈ℒ. As every x'∈ X, x'≤ x, also X≤ x, thus X∈{x'∈ℒ| x'≤ x}. 

As {x '∈ℒ| x'≤ x} is a complete partially ordered set and O_s is a monotone mapping, with 

Theorem 5.1 of <cit.>, we know that there is a unique well-founded sequence O_s^1(),O_s^2()… x' that results in a least fixpoint x' of O_s.

 Clearly also x' is a fixpoint of O. As x is a minimal fixpoint of O and x'≤ x, x=x'.



We can now show the first part of the proposition. 
We first show that C^c(𝒪_l)(y)⊇ C(𝒪_l)(y): since 𝒪 is ≼^A_i-monotonic, with Lemma 2 by <cit.> respectively the definition of a semi-ndao, 𝒪_l(.,y) is ≼^S_L-monotonic, which with † implies C^c(𝒪_l)(y)⊇ C(𝒪_l)(y). That C^c(𝒪_u)(y)⊇ C(𝒪_u)(y) follows with the assumption that 𝒪_u(x,·) is ≼^S_L-monotonic and †. 

For the second part, observe that if (x,y)∈ S(𝒪)(x,y) then with the first part, x∈  S^c(𝒪_l)(y) and y∈  S^c(𝒪_u)(x) and thus (x,y)∈ S^c(𝒪)(x,y).







Thus, what might appear to be a change to the previously formulated stable semantics for non-deterministic approximation operators <cit.>, is a mere generalization of these semantics. 








In general, non-deterministic operators might not admit a fixpoint, which means that the c-complete operator is not always well-defined (e.g. C^c(O_l)(x)=∅ or C^c(O_u)(y)=∅). We illustrate this for C^c(O_u)(y), by giving an example of a ≼^H_L-monotonic operator that does not admit a well-founded sequence.

 Let O:ℕ∪{∞} where O(i)={i+1} for any i∈ℕ and ∞=ℕ. Observe that O is ≼^H_L-monotonic, as for any finite i,j s.t. i≤ j, O(i)={i+1} and i+1≤ j+1 ∈ O(j), and for any finite i, i∈ O(∞). However, it is clear that O admits no fixpoint and thus no well-founded sequence.

To ensure well-definedness of the c-stable operator

we will assume that the lower bound is downwards closed and the upper bound satisfies the following, analogous, notion, called upwards closedness:

An operator O is upwards closed if for every sequence X={x_ϵ}_ϵ<α of elements in ℒ s.t.

  * for every ϵ<α, x_ϵ≼^H_L O(x_ϵ), and

  * for every ϵ' <ϵ<α, x_ϵ'<x_ϵ,
  

it holds that X≼^H_L O( X).



It can be easily observed that the operator from Example <ref> is not upwards closed. Downwards closedness of 𝒪_l(·,y) and upwards closedness of 𝒪_l(x,·) guarantee well-definedness of the c-stable operator:kuratowski1922methode,zorn1935remark

For any (semi-)ndao 𝒪 over a complete lattice s.t. 𝒪_l(.,y) is downwards closed and 𝒪_u(x,.) is upwards closed for any x,y∈ℒ, C^c(𝒪_l)(y)≠∅ and C^c(𝒪_y)(x)≠∅.


We first show that C^c(𝒪_l)(y)≠∅.
Suppose first that 𝒪 is an ndao. Then the proposition is immediate as C(𝒪_l)(y)≠∅ and C(𝒪_y)(x)≠∅ in view of Proposition 13 by <cit.> and C^c(𝒪_l)(y)⊇ C(𝒪_l)(y) and C^c(𝒪_u)(x)⊇ C(𝒪_u)(x) (Proposition <ref>). 
In fact, it can be checked that the proof of Proposition 13 by <cit.>, which means the proofs generalize to semi-ndaos.


We now show that C^c(𝒪_y)(x)≠∅. For this, we show some intermediate results (Lemma <ref> and Lemma <ref>). It will be useful to define, given a non-deterministic operator O over ℒ, a post-fixpoint of O is any x∈ℒ s.t. x≼^H_L O(x). 

Let O be an upwards closed ≼^H_L-monotonic operator over a complete lattice. Then any ≤-maximal post-fixpoint of O is also a fixoint of O.


Suppose that x is a ≤-maximal post-fixpoint of O. Thus, {x}≼^H_L O(x), i.e. there is some z∈ O(x) s.t. x≤ z. As O is ≼^H_L-monotonic, this implies O(x)≼^H_L O(z), and, as z∈ O(x), this means that {z}≼^H_L O(z), which means that z is a post-fixpoint. As x is a ≤-maximal post-fixpoint and x≤ z, this implies z=x. Thus, x∈ O(x) which means x is a fixpoint of O.



Any upwards closed ≼^H_L-monotonic operator over a complete lattice admits a fixpoint. 


Consider a complete lattice L=⟨ℒ,≤⟩ and suppose that O is a  ≼^H_L-monotonic operator.
Clearly, the set of post-fixpoints of O is a partially ordered set. With upwards closedness, every chain of post-fixpoints has an upper bound (which is also a post-fixpoint), and thus, by the  Kuratowski-Zorn lemma <cit.>, there is a maximal post-fixpoint. With Lemma <ref>, this maximal post-fixpoint is a fixpoint. 













We have established thus that 𝒪_u(x,·) admits a fixpoint (as 𝒪_u(x,·) is a ≼^H_L-monotonic operator for any (semi)-ndao 𝒪 and any x∈ℒ). We can show, analogous to the proof of Proposition <ref>, that this means a well-founded sequence exists.






























We now show that the c-stable operator is well-defined for all of the ndao's considered in this paper for choice programs 𝒫 built up from choice atoms whose domain is finite. This is a sufficient condition: other conditions might also guarantee well-definedness. For most applications, this assumption seems warranted.




Let a choice program 𝒫 s.t. for every C_1← C_2,…,C_n∈𝒫, (C_i) is finite for i=1… n,

and x∈{ MR, LPST,𝒰} and x⊆ y ⊆𝒜 be given.  Then S^c(_𝒫^ x)(x,y)≠∅. Furthermore, C^c(^ GZ,l_𝒫)(y)≠∅.[Notice that, as ^ GZ,u_𝒫 is only ≼^H_L-monotonic for consistent inputs, the complete operator for the upper-bound will not be well-defined as it starts from _𝒫^ GZ,u(x,∅). This means that the GZ-operator is only useful for total stable fixpoints.]


We first show C^s(_𝒫^ x,l)(·,y)≠∅ for any y ⊆𝒜.
We have to show that ^l, x_𝒫(·,y) is downward closed
 ^u, x_𝒫(x,·) is upwards closed. 


We now show downward closedness. For this, let {x_ϵ}_ϵ≺α be a descending chain of sets of atoms that are subsets of y and are post-fixpoints of ℐ𝒞^ x,l_𝒫(·,y) for some y⊆𝒜,
and let x=⋂{x_ϵ}_ϵ≺α.  We show that ℐ𝒞^ x,l_𝒫(x,y)≼^S_L {x}. If the chain is finite, this is trivial. 
Suppose therefore that {x_ϵ}_ϵ≺α  is infinite.


  x= MR  Consider some C← C_1,…,C_n∈𝒫 s.t. for every i=1,…,n, y(C_i)= T and there is some z'⊆ x s.t. z'(C_i). As x=⋂{x_ϵ}_ϵ≺α, also z'⊆ x_ϵ for every ϵ≺α. Thus, as ℐ𝒞^ x,l_𝒫(x_ϵ,y)≼^S_L {x_ϵ}, there is some z_ϵ⊆ x_ϵ s.t. z_ϵ∩(C)∈(C) for every ϵ<α. As  {x_ϵ}_ϵ≺α is a ⊆-descending chain, and (C) is finite, there is some β<α s.t. for every ϵ,ϵ'<β, (C)∩ z_ϵ=(C)∩ z_ϵ'∈(C). But then also (C)∩ z_ϵ'⊆ x, and thus ℐ𝒞^ x,l_𝒫(x,y)≼^S_L {x}.

  x= LPST Consider some C← C_1,…,C_n∈𝒫 s.t. for every i=1,…,n, and every z∈ [x,y], z(C_i)= T. As x⊆ x_ϵ for every ϵ<α, C∈_𝒫^ LPST,l(x_ϵ,y) (for every ϵ<α), and thus, as ℐ𝒞^ x,l_𝒫(x_ϵ,y)≼^S_L {x_ϵ}, there is some z_ϵ⊆ x_ϵ s.t. z_ϵ∩(C)∈(C) for every ϵ<α. The rest of the proof is similar to the previous item.

  x= GZ  Consider some C← C_1,…,C_n∈𝒫 s.t. for every i=1,…,n, x∩(C_i)=y∩(C_i)∈(C_i). As x⊆ y, for every z∈ [x,y], z∩(C_i)=x∩(C_i)∈(C_i). As ℐ𝒞^ x,l_𝒫(x_ϵ,y)≼^S_L {x_ϵ}, there is some z_ϵ⊆ x_ϵ s.t. z_ϵ∩(C)∈(C) for every ϵ<α. The rest of the proof is similar to the previous item.

  x=𝒰  We have to show that for every z∈ [x,y] there is some z'∈_𝒫(z) s.t. z'⊆ x. As z∈ [x_ϵ,y] for any ϵ<α, for every ϵ<α there is some z_ϵ⊆ x_ϵ with z_ϵ∈_𝒫(z). For every such z_ϵ, it holds that for every C∈_𝒫(z), z_ϵ∩(C)∈(C). As (C) is finite,  there is some β<α s.t. for every ϵ,ϵ'<β, z_ϵ∩(C)=z_ϵ'∩(C)∈(C) for every ϵ<α. But then also z_ϵ∩(C)⊆ x.


We now show upwards closedness of ^𝒰,u_𝒫(x,·)  (for an arbitrary but fixed x⊆𝒜).
For this, let {y_ϵ}_ϵ<α be an ascending chain of sets of atoms that are supersets of x and postfxipoints of ^𝒰,u_𝒫(x,·) and let y=⋃{y_ϵ}_ϵ<α. We show that ^𝒰,u_𝒫(x,y)≼^H_L {y}. If the chain is finite, this is trivial. Suppose thus that {y_ϵ}_ϵ<α is infinite. We show that there is some z∈ [x,y] s.t. y∈_𝒫(y). Indeed, consider some C← C_1,…,C_n s.t. (C_i)∩ y∈(C_i) for every i=1,…,n. 
As y=⋃{y_ϵ}_ϵ<α, and (C_i) is finite (for every  i=1,…,n), there is an ϵ<α s.t. for every ϵ'∈ [ϵ,α], (C_i)∩ y_ϵ'∈(C_i) (for i=1,…,n). Thus, for every ϵ'∈ [ϵ,α], (C)∩ y_ϵ'∈(C) (as {y_ϵ'}≼^H_L _𝒫^𝒰,u(x,y_ϵ')).
Again in view of the finitude of (C), there is some ϵ<α s.t. for every ϵ'∈ [ϵ,α], (C)∩ y_ϵ=(C)∩ y_ϵ', which implies that  (C)∩ y_ϵ=(C)∩ y, and, since 
 (C)∩ y_ϵ∈(C), shows that (C)∩ y∈(C).


That C^c(^ x,u_𝒫)(x)≠∅ follows from Lemma <ref> and the fact that _𝒫^ x,u(x,·) is ≼^H_L-monotonic (Proposition <ref>) for any x ⊆𝒜.



Upwards closedness is not guaranteed when allowing choice constructs with infinite domains:

Consider the set of atoms x={a_i| i∈ℕ} and the choice atom C=(A, {A'⊆ A| A'}). We let 𝒫={C←}. 
Let x_j= A∖{a_j| j≤ i} for any i∈ℕ. Then {x_j}_j<∞ is a ⊆-descending chain of infinite sets. We see that for any j<∞, _𝒫(x_j)={A'⊆ A| A'}≼^S_L {x_j} as x_j∈{A'⊆ A| A'}. However, ⋂_j<∞{x_j}=∅, whereas _𝒫(∅)={A'⊆ A| A'}≼^S_L {x_j} and thus  _𝒫(∅)⋠^S_L {∅}.



Another property that stable fixpoints of both deterministic and non-deterministic operators have in common is that they are  ≤_t-minimal. As is to be expected and as can be seen from the constructive stable fixpoint ({p,q},{p,q}) in Example <ref>, constructive stable fixpoints are not necessariy ≤_t-minimal, in contradistinction to minimiality-based stable fixpoints (Proposition 14 by <cit.>).


We now show a host of representation results: the LPST-operator allows to generalize 

 the semantics of Liu et al <cit.> from two- to three-valued. The GZ-operator allows to adapt the semantics of  <cit.> (defined for disjunctive programs) to choice programs, and the semantics of <cit.> from two- to three-valued.  



Let a choice program 𝒫 be given.



  * x is a stable model according to <cit.> iff (x,x) is a stable fixpoint of ℐ𝒞^ LPST_𝒫. 

  * x is a stable model according to <cit.> iff (x,x) is a stable fixpoint of ℐ𝒞^ MR_𝒫. 

  * If 𝒫 is a aggregate program then x is a stable model according to <cit.> iff 
 x∈ C^c(ℐ𝒞^ GZ,l_𝒫)(x).




For 1., we recall first some necessary preliminaries from <cit.>: 

Given some sets x, y⊆𝒜 and a choice atom C:


  * y▹^spt_x C iff y is an x-trigger for C.

  * 𝒫^▹^spt_x(y)={C← C_1,…,C_n| y▹^spt_x C_ii=1… n},

  * T^nd, ▹^spt_x_𝒫(y)={z⊆⋃_C∈𝒫^▹^spt_x(y)(C)| z∩(C)∈(C) ∀ C←ϕ∈𝒫^▹^spt_x(y)},

  * A sequence  ∅=x_1,…,x_n=x is a self-justified  ▹^spt_x-computation if x_i+1∈ T^nd, ▹^spt_x_𝒫(x_i) for every i≥ 1, and x∈ T^nd, ▹^spt_x_𝒫(x).



We first show that (†): a sequence ∅=x_1,…,x_n=x is well-founded relative to ^ LPST,l_𝒫(·,x) iff it is a self-justified weak ▹^spt_x-computation. That this is the case is easily seen by observing that _𝒫^ LPST,l(x',x)=T^nd, ▹^spt_x_𝒫(x') for any x',x⊆𝒜 (as x' is an x-trigger for C iff for every z∈[x',x], z(C)= T). 
We now show the main claim.
Suppose for the ⇒-direction that x is a stable model according to <cit.>. With Proposition 12 from <cit.>, there is a self-justified   ▹^spt_x-computation for x. With †, there is a well-founded sequence  relative to ^ LPST,l_𝒫(·,x) for x. Thus, x∈  S^c(ℐ𝒞^ LPST_𝒫)(x,x). Suppose now for the ⇐-direction that  x∈  S^c(ℐ𝒞^ LPST_𝒫)(x,x), i.e. there is a well-founded sequence  relative to ^ LPST,l_𝒫(·,x) for x. With †,  this sequence  is a self-justified weak ▹^spt_x-computation. With Proposition 11 by <cit.>, this is a founded computation. By Definition 5 by <cit.>, this means x is an answer set according to <cit.>.

2. We first recall the relevant definitions <cit.> (adapted to our notation):

Given a choice atom C=(,), and a set of atoms x, C_y={z⊆ y|∃ z'∈(C)z⊆ z'}. A perhaps helpful alternative description is: C_y=⋃_z∈(C)^z⊆ y[z,y]. 
Given a choice program 𝒫 and a set of atoms x, (𝒫,y) is obtained by:


  * Removing any C← C_1,…,C_n s.t. y(C_i)= F for some i=1,…,n,

  * For every rule r=C← C_1,…,C_n not removed in the first step, replace r by {α←C_1_x,…,C_n_x|α∈(C)}.

 <cit.> show that for any choice program 𝒫, (𝒫,y) has a unique least model obtained by iteratively applying the one-step provability operator  T_(𝒫,y)(x)={α|α←C_1_x,…,C_n_x∈(𝒫,y)∀  i=1… n: x∩( C_i_x)∈(C_i_x)}. Call this least model N_𝒫,y. Then y is a MR-stable model of 𝒫 iff y is a model of 𝒫 and y=N_𝒫,y.

 We show that (†): T_(𝒫,x)(z)∈^ MR,l_𝒫(z,x) for any z⊆ x. For this, consider some C← C_1,…,C_n∈𝒫. We distinguish two cases:
 

  * 
x(C_i)= F for some i=1,…,n. Then C∉^ MR,l_𝒫(z,x) for any z⊆ x and thus we are done.

  * 
x(C_i)= T for every  i=1,…,n.  We distinguish two cases:


  * 
Suppose first that there is no z⊆ x s.t. z(C_i)= T for some i=1,…,n. Then C∉^ MR,l_𝒫(z,x) and thus we are done. 

  * 
Suppose now that there is a z'⊆ z s.t. z'(C_i)= T for every i=1,…,n.  Then C∈^ MR,l_𝒫(z,x).
Also, for every i=1,…,n, z∈(C_i_x) and, as α←C_1_x,…,  C_n_x∈(𝒫,x) for every α∈(C)∩ x, (C)∩ x⊆  T_(𝒫,x)(z). As x is a model of 𝒫 and x(C_i)= T for every i=1,…,n, x∩(C)∈(C). 

  

We now show that x∈ S^c(^ MR,l_𝒫)(x,x) iff x is a MR-stable model of 𝒫.
For the ⇒-direction, observe first that as x is a fixpoint of ^ MR,l_𝒫, it is, with Proposition <ref>, a model of 𝒫. With †, T_(𝒫,x)(z)∈^ MR,l_𝒫(z,x) for any z⊆ x, which implies
(with a simple induction on the well-founded sequence ∅,…,x) that x=N_𝒫,x.
For the ⇐-direction, suppose that x is an MR-stable model. Notice that as x=N_𝒫,x, there is a sequence ∅=x_0, x_1,…,x_n=x s.t. x_i+1= T_(𝒫,x)(x_i) for any i=0,…,n. With †, this is a well-founded sequence relative to _𝒫^ MR,l(·,x). 

3. follows from the fact that for choice programs with only atoms in the head, ^ GZ,l coincides with ℐ^ GZ_𝒫,x(y), used by <cit.> to characterize the semantics of <cit.>.

Furthermore,  for normal logic programs, all operators besides the ultimate coincide, and the stable fixpoints respectively partial stable fixpoint coincide with the stable respectively partial stable models (and similarly for the well-founded semantics) (recall Remark <ref>). Thus, the semantics studied in this paper do not only generalize existing semantics for choice or aggregate programs, but also the well-known semantics for normal logic programs.




§ GROUNDEDNESS

 We  introduce several postulates to facilitate a comparison between semantics for choice programs (thus solving an open question in the literature <cit.>) formalizing in different ways the idea of groundedness. Furthermore, we show that for every notion of groundedness, there exist examples which have been argued in the literature to be counter-intuitive.












Intuitively, the idea behind groundedness is that models should be derivable from the ground up, i.e. they should be supported by non-cyclic arguments. For choice programs,  what constitutes a cycle becomes less clear:
[<cit.>]
Consider the program 𝒫={{p,q}=2←{p,q}≠ 1}.
There are two candidates for stable models: ∅ and {p,q} (as the only satisfier of the head of the only rule is {p,q}), which we discuss:

(1) If we choose ∅, then we see that the only rule in the program is applicable but not applied.

(2)
 If we choose {p,q}, we could justify this intuitively by the sequence ∅,{p,q}: at the first step, ∅ makes {p;q}≠ 1 true and thus we derive {p,q}. At the second step, however, we can only justify our choice by a self-supporting justification: {p,q} is true since it is derivable using the head of the rule {p,q}=2←{p;q}≠ 1 and since {p,q}({p;q}≠ 1)= T.

 We observe that {p,q} is stable fixpoint for the 𝒰- and MR-operator, but not for the LPST- and GZ-operators. 

The benefit of our operator-based framework is that we do not have to choose for a single “best” semantics: the choices outlined above will be obtainable by using different operators. 
We now carry forward our study of self-support on a more general level by introducing three postulates of decreasing strength.
The first notion (inspired by <cit.>) requires that we can find a stratification of the program (i.e. an assignment κ of natural numbers representing levels) s.t. the entire domain of the head of a rule is strictly higher stratified then the domain of the choice constructs in the body.

 A set x is d(omain)-grounded (for 𝒫) if there is some κ:x→ℕ s.t. every a∈ x there is some r=C← C_1,…,C_n∈𝒫 s.t. a∈(C), κ(a)>max{κ(b)| b∈⋃_i=1^n(C_i)}. 

As already observed by <cit.>, this requirement might be overly strong:

Consider  𝒫={b← 1 {a,b}; a←}. Then one might expect {a,b} to be a good candidate for a stable model. Indeed, a is a fact and allows to support b. However, {a,b} is not d-grounded, as there is no κ:{a,b}→ℕ s.t. κ(b)>max{κ(a),κ(b)}. 

 The following weaker notion (inspired in name and idea by  <cit.>) requires that at some point in a sequence, there is support for the body of a rule, and this support persists in every following step:

(1) A set x is an y-trigger for C← C_1,…,C_n if for every z∈ [x,y], z(C_i)= T for every i=1,…,n.
(2) A set x is s(trongly)-grounded (for 𝒫) if there is some κ:x→ℕ s.t. every a∈ x, there is some r=C← C_1,…,C_n∈𝒫 s.t. a∈(C) and there is an x-trigger z for r s.t. κ(a)>max{κ(b)| b∈ z}.

Intuitively, x is s-grounded if for every atom a∈ x, we can find a rule that that has an x-trigger in a strictly lower level. 

D-grounded sets are s-grounded, but not vice-versa:

Consider 𝒫 as in Example <ref>. 
We see that {a,b} is strongly grounded. Indeed, if we take κ(a)=0 and κ(b)=1, we see that 
b← 1 {a,b} has a κ-lower {a,b}-trigger in {a} as {a}(1 {a,b})={a,b}(1 {a,b})= T. 


The following example, first introduced by <cit.>, shows that in some circumstances, s-groundedness might be overly strong:

Consider 𝒫={a←{a,b}≠ 1; b←{a,b}≠ 1}. One might expect the set {a,b} to be acceptable. Suppose it is s-grounded. Then there is an {a,b}-trigger y for a s.t. y⊆{b}. This is impossible as {b}∈ [∅,{a,b}] and {b}∈ [{b},{a,b}] and {b}∩({a,b}≠ 1)∉({a,b}). Thus, {a,b} cannot be s-grounded.


We thus introduce an even weaker notion of groundedness which we call antecedent groundedness, which requires that for every accepted atom, we can find a rule that is activated by a lower κ-level:

A set x is a(ntecedent)-grounded if there is some κ:x→ℕ s.t. every a∈ x, there is some r=C← C_1,…,C_n∈𝒫 s.t. a∈(C) and  for every i=1,…,n there is some z⊆{b|κ(b)< κ(a)} s.t. z(C_i)= T .

S-grounded sets are a-grounded, but not vice-versa:

Consider again 𝒫 from Example <ref>. We first observe that {p,q} is antecedent grounded as ∅({p,q}≠ 1)= T and thus we have found our antecedent justification for {p,q}. However, ∅ is not a {p,q}-trigger, as {p}∈ [∅,{p,q}] and {p}({p,q}≠ 1)= F. Likewise, it can be seen that {a,b} in Example <ref> is a-grounded.


The reader might think that a-groundedness is the most suitable notion of groundedness, as it avoids the behavior (deemed problematic by <cit.> in Example <ref>). However,  <cit.> argued that, for 𝒫 from Example <ref>, {p,q} is not a feasible candidate for an answer set. Altogether, we see that for every notion of groundedness, one can find examples deemed as counter-intuitive in the literature.

Notice that all notions of groundedness generalize a well-known notion of groundedness proposed for normal logic programs <cit.>:

Let 𝒫 a normal logic program 𝒫. Then x is a-grounded 𝒫 then x is grounded according to <cit.>, namely: there is a ranking κ:x→ℕ s.t. for every a∈ x, there is some a← a_1,…,a_n, ¬ b_1,…,¬ b_m∈𝒫 s.t. for every i=1,…,n, κ(a)>κ(a_i).


Suppose that x is a-grounded for 𝒫, i.e. there is some κ:x→ℕ s.t. for every a∈ x, there is some r=a←⋀_i=1^n a_i⋀_i=1^m ¬ b_i s.t. for every i=1,…,n, there is some z⊆{b|κ(b)<κ(a)} with z(a_i)= T. As z(a_i)= T iff a_i∈ z, this means that κ(a)>κ(a_i) for every i=1,…,n.




Notice that the other direction of Proposition <ref> does not necessarily hold: take e.g. 𝒫={b←¬ c., c←¬ b.}. Then {b,c} is grounded according to <cit.> (trivially, as there are no rules with positive bodies) but not a-grounded.




The 𝒰-operator does not satisfy any notion of groundedness, whereas all other operators give rise to a-grounded stable models, but only the LPST-operator and GZ-operator give rise to s-grounded stable models and only the GZ-operator gives rise to d-grounded stable models:

Let 𝒫 be a choice program and †∈{ GZ, LPST, MR}. Then if (x,y)∈ S^c(ℐ𝒞_𝒫^†)(x,y):


  * x and y are a-grounded (for 𝒫), and

  * x is s-grounded (for 𝒫) if †∈{ GZ, LPST}.

  * x and y are d-grounded (for 𝒫) if †= GZ.

There are programs 𝒫 s.t. (x,y)∈ S^c(ℐ𝒞_𝒫^𝒰)(x,y) yet x and y are not antecedent grounded.


Let 𝒫 be a choice program and †∈{ GZ, LPST, MR}. 






Ad 1.: 
We first consider C^c(ℐ𝒞^l, MR_𝒫).
Suppose that x∈ C^c(ℐ𝒞^l, MR_𝒫)(y), i.e. there is a well-founded sequence ∅=x_0,…,x_n,x relative to ^ MR,l_𝒫(·,y).
We let κ(a)=min{i| a∈ x_i}. 
 Consider some a∈ x and let i be the ordinal s.t. a∈ x_i+1∖ x_i. Then there is some C← C_1,…,C_n∈𝒫 s.t. a∈(C)∩ x_i+1 and for every j=1… n, y(C_j)= T and there is some w⊆ x_i s.t. w(C_j)= T. But then w(C_j)= T and κ(b)≤ i < κ(a)=i+1.
 
 We now consider C^c(ℐ𝒞^ GZ,u_𝒫)=C^c(ℐ𝒞^ LPST,u_𝒫)=C^c(ℐ𝒞^𝒰,u_𝒫).  




Suppose that  y∈ C^c(ℐ𝒞^u, MR_𝒫)(x), i.e. there is a well-founded sequence ∅=y_0,…,y_n,y relative to ^𝒰,u_𝒫(x,·).
We let κ(a)=min{i| a∈ y_i}. 
Consider some a∈ y and let i be the ordinal s.t. a∈ y_i+1∖ y_i. Then there is some z∈ [x,y_i] and some C← C_1,…,C_n s.t. z(C_j)= T for every j=1,…,n and a∈(C). As z⊆ y_i, z⊆{a∈𝒜|κ(z)≤ i}, whereas κ(a)=i+1. Thus, we have found our justification for a.

 
 
 Ad 2.: Suppose that x is a constructive stable fixpoint of ℐ𝒞^ LPST_𝒫, i.e. there is a well-founded sequence relative to ^ LPST,l_𝒫(·,y) ∅=x_0,…,x_n,x. We let κ(a)=min{i| a∈ x_i}.  Consider some a∈ x and let i be the ordinal s.t. a∈ x_i+1∖ x_i.

Then there is some r=C← C_1,…,C_n∈𝒫 s.t. a∈(C)∩  x_i+1 and for every z∈ [x_i,y] and every j=1,…,n, z(C_j)= T. But then clearly x_i is an y-trigger for r and for every b∈ x_i, κ(b)≤ i<κ(a)=i+1.


Ad 3.: suppose that x is a constructive stable fixpoint of ℐ𝒞^ GZ_𝒫,  i.e. there is a well-founded sequence relative to ^ GZ,l_𝒫(·,y) ∅=x_0,…,x_n,x.  We let κ(a)=min{i| a∈ x_i}. Consider now some a∈ x and let i be the ordinal s.t. a∈ x_i+1∖ x_i. This means that a∈(C) for some C∈^ GZ,l_𝒫(x_i,x). Suppose towards a contradiction that for every C← C_1,…,C_n∈𝒫 with a∈(C), there is some i=1,…,n s.t. for some b∈(C_i), κ(b)≥κ(a). For any such b, b∉x_i, which implies that b∉(C_i)∩ x_i yet b∈(C_i)∩ x, i.e. (C_i)∩ x_i≠(C_i)∩ x, contradiction to C∈^ GZ,l_𝒫(x_i,x).

Regarding the ultimate operator, consider 𝒫={{p,q}=2←{p,q}=2}. Then we see that ^𝒰_𝒫(∅,{p,q})=(⋃_z⊆{p,q}_𝒫(z),⋃_z⊆{p,q}_𝒫(z))=({∅,{p,q}}, {∅,{p,q}}). Thus, the sequence ∅,{p,q} is well-founded relative to ^𝒰,l_𝒫(·,{p,q}). However, it is clearly not antecedent grounded as  we cannot justify {p,q} in view of ∅. 


Summing up, we see that the different semantics satisfy different notions of groundedness, and that no notion of groundedness is uncontested. It seems that it depends on the application at hand which notion of groundedness is the most suitable.



The results of this section are summarized in Table <ref>.




§ DISJUNCTIONS ARE CHOICE CONSTRUCTS

Our study allows us to give a principled account of the relation between stable semantics for disjunctive logic programs (DLPs) and choice programs. Indeed, in this section, we show that for DLPs <cit.> are a special case of the operator for choice programs. This means that all semantics obtained on the basis of these operators coincide, thus explaining the difference between semantics for stable models of disjunctive logic programs and choice logic programs in terms of the choice of stable operator (minimality-based versus constructive). In the rest of this section, we show this claim in more detail, first providing the necessary background on DLPs and then showing DLP-operators are a special case of the operators studied in this paper.



  
*Preliminaries on disjunctive logic programs
We first recall some necessary preliminaries on disjunctive logic programming. For simplicity, we restrict attention to disjunctive logic programs whose bodies consist solely of literals, leaving the generalization of these results to stronger languages for future work. 

In what follows we consider a propositional[We restrict ourselves to the propositional case.] language 𝔏, 
whose atomic formulas are denoted by p,q,r (possibly indexed), and that contains the propositional constants T (representing truth), F (falsity), 
U (unknown), and C (contradictory information). The connectives in  𝔏 include negation , conjunction ∧ and disjunction ∨. Formulas are denoted by ϕ, ψ, δ (again, possibly indexed). A (propositional) disjunctively normal logic program 𝒫 in 𝔏 (a dlp in short) is a finite set of rules of the form 
             ⋁_i=1^n p_i ← ⋀_i=1^n a_i⋀_i=1^m ¬ b_i,  where the head ⋁_i=1^n p_i is a non-empty disjunction of atoms, and the body ψ
             is a formula in 𝔏.  We furthermore recall that a formula of the form ⋀_i=1^n a_i⋀_i=1^m ¬ b_i can be evaluated over the bilattice FOUR relative to (x,y) by assuming the involution - defined by - T= F, - F= T, - U= U and - C= C,
 and by defining truth assignments to complex formulas recursively as follows:
 
    
  * (x,y)(p)=
 T    if p∈ x  and p∈ y,  
 U    if p∉x  and p∈ y,  
 F    if p∉x  and p∉y,  
 C    if p∈ x  and p∉y. 
  * (x,y)(¬ϕ)=- (x,y)(ϕ),  

  * (x,y)(ψϕ)=⊓_≤_t{(x,y)(ϕ),(x,y)(ψ)}, 

  * (x,y)(ψϕ)= ⊔_≤_t{(x,y)(ϕ),(x,y)(ψ)}. 

Notice that this is equivalent to the evaluations introduced in Section <ref>.

We recall the following ndao ^d_𝒫 introduced by <cit.> and defined as follows (given a dlp 𝒫 and an interpretation (x,y)):


  * ^d,l_𝒫(x,y) = {Δ|⋁Δ←ϕ∈𝒫, (x,y)(ϕ)≥_t  C}, 

  * ^d,u_𝒫(x,y) = {Δ|⋁Δ←ϕ∈𝒫, (x,y)(ϕ)≥_t  U}, 
 
  * ^d,†_𝒫(x,y)={x_1⊆⋃^d,†_𝒫(x,y) |∀Δ∈^d,†_𝒫(x,y),  x_1 ∩Δ≠∅} (for †∈{l,u}),



  * ^d_𝒫(x,y)=(^d,l_𝒫(x,y), ^d,u_𝒫(x,y)). 


The operator ^d_𝒫 faithfully represents the semantics of dlps:
In general, total stable fixpoints of 𝒫 correspond to stable models of 𝒫 <cit.>, and weakly supported models of 𝒫 <cit.> correspond to fixpoints of ^d_𝒫 <cit.>.





  
*Disjunctions as Choice Constructs
We now show how the operator ^d_𝒫 can be seen as a special case of the operators for choice constructs. In more detail, 
for a DLP 𝒫, we define D2C(𝒫)={1Δ←ϕ|⋁Δ←ϕ∈𝒫}. E.g. D2C({p q←})= {1{p,q}←}.  In other words, we replace every disjunction by the choice atom that requires at least one element of Δ is true. 
We will show here that the operator defined for disjunctive logic programs <cit.> then coincides with the operator ^ x_ D2C(𝒫) (for x= MR, LPST). 
This implies that all AFT-based semantics coincide for DLPs and their conversion into choice rules. We can now point in a very exact way to the difference between DLPs and choice programs: it lies not in how the constructs of disjunction and choice atoms are treated (i.e. when they should be made true or false), but rather in how the stable semantics is defined: for disjunctions, typically (e.g. in the most popular solvers <cit.>), a minimality-based stable operator is used, whereas for choice constructs, the c-stable operator is more apt. Thus, disjunctive and choice programs use the same (approximation) operators, but differ in how the corresponding stable operator is constructed.



We now show that the operator ^d_𝒫 is a special case of the MR- and LPST-operators from this paper, when applied to the choice program D2C(𝒫).

For any disjunctive normal logic program 𝒫, _𝒫=^ MR_ D2C(𝒫)=^ LPST_ D2C(𝒫).


We first notice that, as D2C(𝒫) does not change the bodies of rules in 𝒫, if 𝒫 is a disjunctively normal logic program, D2C(𝒫) is a normal choice program. This means that,  with Proposition <ref>, ^ MR_ D2C(𝒫)=^ LPST_ D2C(𝒫).

Furthermore, it can be straightforwardly shown that for any disjunctively normal logic program, ^d,†_𝒫(x,y)=^ MR,†_𝒫(x,y).
Finally, it suffices to observe that z∩(1≤Δ)∈(1≤Δ) iff Δ∩ z≠∅ for any z⊆𝒜, which means that z∈_𝒫(x,y) iff z∈^ MR_ D2C(𝒫)(x,y) for any x,y⊆𝒜.

From this, we immediately obtain that all the major semantics for disjunctive logic programming are special cases of the semantics introduced in this paper. Namely, weakly supported models of 𝒫 coincide with fixpoints of ^ MR_ D2C(𝒫) whereas minimality-based stable fixpoints of 𝒫 coincide with stable models of 𝒫. Since, the c-stable semantics  does not enforce minimality, it will in general not correspond to the stable models semantics:

Let 𝒫={p q←}. Then ({p,q},{p,q}) is a c-stable fixpoint of ^ MR_ D2C(𝒫) but it is not a stable model of 𝒫 according to  <cit.>.



This also gives an answer to the question of how to combine disjunctions and choice constructs in logic programs: a choice as to which stable semantics are used has to be made: either preserve the minimality of answer sets as in DLPs and loses some reasonable models, or one gives up the minimality requirement by using the constructive stable semantics. In this context, it is perhaps interesting to note that the constructive stable semantics still coincides with the standard stable semantics for normal logic programs. In that case, all stable models are minimal. On the other hand, we can now also use minimality-based stable semantics on more complicated choice construct than simple disjunctions. 





§ RELATED WORK

To the best of our knowledge, this is the first application of AFT to the semantics of choice programs. 
We have shown how major semantics for choice programs <cit.> can be characterized in our framework. Other well-known semantics for (disjunctive) aggregate programs are those by <cit.>, which are the ones used in the solvers DLV respectively clingo <cit.>.

Another semantics, the FLP-semantics <cit.>, were originally not defined for programs with choice constructs in the head, but were generalized to allow choice constructs in the head by <cit.>. Nevertheless, these semantics only allow for two-valued stable models, and were shown  to differ from the semantics by <cit.>, <cit.> and <cit.> already for aggregate programs <cit.>, which means, in view of Proposition <ref>, that the stable semantics induced by the LPST-, MR- and GZ-operators also differ from the FLP-semantics. This comparison also holds for the semantics by <cit.> as the latter coincide with the FLP-semantics for aggregates with positive atoms <cit.>. 

Another line of work that is relevant for this paper is the application of AFT to logic programs with aggregates.
<cit.> introduce several approximation operators for aggregate programs, whereas <cit.> introduces operator-based semantics for disjunctive aggregate programs. These semantics were generalized by <cit.> in the framework of non-deterministic AFT. Generalizing the operator by   <cit.> to choice programs is left for future work. An overview of semantics that are (non-)representable in the deterministic AFT-framework is given by <cit.>. This work severed as an important foundation of our paper, as the operators ^ GZ_ P and ^ MR_ P are generalizations of the operator-based characterisations by <cit.> of the corresponding semantics. 




§ CONCLUSION

The main contributions of this paper are the definition of several ndaos for choice programs, the definition of the constructive stable operator, the characterisation of several existing semantics for various dialects of logic and choice programs and the introduction and study of postulates for choice programs. We provide a principled view of choice programs versus disjunctive programs.


This paper is subject to one restriction: we assume ^c_𝒫(x,y)≠∅ for every interpretation (x,y). In future work, we will generalize our results beyond this assumption. We also plan to study the complexity of the resulting semantics, device implementations and study other AFT-based semantics, such as the Kripke-Kleene and well-founded states and semi-equilibrium semantics <cit.> and study other operators, e.g. inspired by <cit.>.

























cousot1979constructive



  
*Acknowledgements
I thank Bart Bogaerts and Wolfgang Faber for interesting discussions on this topic, and the reviewers for their dilligent reviews.
named









 Paper Title: SentenceVAE: Faster, Longer and More Accurate Inference with Next-sentence Prediction for Large Language Models.






Scaling Backwards:
Minimal Synthetic Pre-training?

    Ryo Nakamura1,*
0009-0009-0858-1229 
Ryu Tadokoro2,*
0009-0001-9473-3832
Ryosuke Yamada1
0000-0002-2154-8230


Yuki M. Asano 30000-0002-8533-4020
Iro Laina40000-0001-8857-7709
Christian Rupprecht40000-0003-3994-8045
Nakamasa Inoue50000-0002-9761-4142
Rio Yokota50000-0001-7573-7873
Hirokatsu Kataoka10000-0001-8844-165X

    August 5, 2024
====================================================================================================================================================================================================================================================================================================================================



§ ABSTRACT 
Contemporary large language models (LLMs) primarily rely on next-token prediction method for inference, which significantly impedes their processing speed. In this paper, we introduce a novel inference methodology termed next-sentence prediction, aimed at enhancing the inference efficiency of LLMs. We present Sentence Variational Autoencoder (SentenceVAE), a tiny model consisting of a Sentence Encoder and a Sentence Decoder. The encoder effectively condenses the information within a sentence into a singular token, while the decoder reconstructs this compressed data back into its original sentential form. By integrating SentenceVAE into the input and output layers of LLMs, we develop Sentence-level LLMs (SLLMs) that employ a sentence-by-sentence inference approach, markedly accelerating inference speeds. SentenceVAE also maintains the integrity of the original semantic content by segmenting the text into sentences, thereby improving accuracy while boosting inference speeds. Compared to published LLMs, SLLMs process fewer tokens over equivalent context lengths, significantly reducing memory demands for self-attention computations and facilitating the handling of longer contexts. Our experimental findings reveal that this method can accelerate inference speeds by 204∼365%, reduce perplexity (PPL) to 46∼75% of its original metric, and decrease memory overhead by 86∼91% for the same context length, compared to the token-by-token method. Moreover, the benefits of this approach become even more pronounced as model parameters increase.












§ INTRODUCTION




Large language models (LLMs) have demonstrated remarkable proficiency in understanding human intent through the acquisition of knowledge from vast and diverse datasets. These models provide rich and accurate responses, making them essential for applications such as machine translation <cit.>, chatbots <cit.>, and question-answering systems <cit.>. The predominant method to inference in LLMs involves next-token prediction, where the model generates tokens sequentially. While this token-by-token generation method has achieved significant performance, it is inherently limited in producing only one token per inference step. This limitation results in substantial computational overhead and increases the overall inference time, hindering the scalability and efficiency of LLMs.  Consequently, there is a critical need to explore alternative methods that can mitigate these limitations while preserving or enhancing the accuracy and responsiveness of LLMs.

To enhance the inference efficiency of large language models (LLMs), Gloeckle et al. <cit.> proposed training LLMs to predict multiple tokens simultaneously, aiming to accelerate the inference process. However, the number of tokens predicted simultaneously is fixed, resulting in a rigid and inflexible partitioning of the input text. For example, when set to five, the input is invariably segmented into non-overlapping sequences of five tokens, regardless of the text's underlying structure, potentially compromising inference accuracy.

To enhance inference speed while preserving or enhancing accuracy, we explore methods for adaptively selecting the optimal number of predicted tokens based on the semantic content of the text. Consequently, we introduce Sentence Variational Autoencoder (SentenceVAE), which comprises a sentence encoder and a sentence decoder. The schematic of SentenceVAE is illustrated in Fig. <ref>. The sentence encoder is designed to condense the information of an entire sentence into a single token, while the sentence decoder reconstructs this compressed token back into a sentence. By integrating the encoder and decoder of SentenceVAE into the input and output layers of LLMs, these models can perform next-sentence prediction, thereby accelerating the reasoning process. Moreover, since SentenceVAE segments text at the sentence level, the semantic integrity is better preserved, ensuring the accuracy of the inferrence. Specifically, prior to inputting text into LLMs, SentenceVAE segments the text into sentences and utilizes the sentence encoder to compress the information contained within each sentence into a single token. These compressed tokens are then fed as input to the LLMs. The LLMs predict their output based on these compressed tokens. Finally, the sentence decoder decodes the predicted token into the final output of the LLMs.

The contribution of the proposed method can be described as follows:



  * We propose SentenceVAE, a model capable of compressing the information content of a sentence into a single token, and conversely, expanding such tokens to reconstruct sentences with high informational fidelity.

  * We embed the encoder and decoder components of SentenceVAE into the input and output layers of LLMs, enabling these models to implement an effective inference method for next-sentence prediction.

  * Extensive experiments conducted on various LLMs have demonstrated that the proposed method significantly accelerates inference speed while preserving the accuracy of these models. 






§ RELATED WORK


Large language models: in recent years, LLMs have achieved remarkable performance in the field of natural language processing (NLP). However, these models typically possess a vast number of parameters from billions to hundreds of billions, exemplified by models such as TeleChat (1B-52B) from TeleAI <cit.>, Llama (7B-405B) from Meta <cit.>, InternLM (1.8B-20B) from Shanghai AI Lab <cit.>, etc. The sheer magnitude of these parameters renders the inference process computationally intensive and time-consuming. Moreover, most LLMs employ a one-token prediction method, where the model predicts only one token at a time. This token-by-token approach further exacerbates the inference time of LLMs. As a result, accelerating the inference speed of LLMs while preserving or enhancing their accuracy presents a significant challenge for researchers.

Multi token prediction: The transformer model enables the parallel processing of sequences through its self-attention mechanism, providing a technical foundation for the implementation of multi-token prediction <cit.>. The BERT model utilizes bidirectional encoder representations to capture contextual information, further solidifying the basis for the implementation of multi-token prediction <cit.>. The T5 model incorporates multi-token prediction during the training process for the first time, significantly enhancing the coherence and quality of the generated text <cit.>. The Better & Faster LLMs apply multi-token prediction to LLMs, allowing them to directly predict multiple tokens in a single inference step, thereby reducing the number of inference iterations and associated computational costs <cit.>. However, the number of predicted tokens is fixed, which may inadvertently combine unrelated tokens. This can result in the merging of unrelated tokens, disrupting the original sentence structure and consequently reducing inference accuracy.

Encoder-decoder model: The concept of an encoder is introduced by Yann LeCun in his doctoral dissertation, wherein he posited that an encoder could transform input data into shorter representations without losing the core information, thereby providing a foundational approach to compressing high-dimensional data. The Variational Autoencoder (VAE) model introduces the use of hidden variables between the encoder and decoder to learn the data distribution, laying the groundwork for deep-learning-based encoders and decoders <cit.>. Sutskever et al. introduces the encoder-decoder framework into the field of deep learning, utilizing recurrent neural networks (RNNs) as both the encoder and decoder for sequence-to-sequence (Seq2Seq) learning, demonstrating exceptional performance in machine translation tasks <cit.>.
The SegNet applies the encoder-decoder model to the field of image segmentation, significantly enhancing segmentation performance <cit.>. Subsequently, the encoder-decoder model has achieved great success in various domains including machine translation <cit.>, text-to-speech conversion <cit.>, computational imaging <cit.>, and beyond, fully demonstrating the capability of the encoder-decoder architecture to efficiently compress and restore data.

By leveraging the advantages of multi-token prediction and the encoder-decoder model, we propose SentenceVAE. SentenceVAE is capable of compressing the information contained within a sentence into a single token, enabling next-sentence prediction and significantly accelerating the inference speed of LLMs.  Additionally, our method divides the text into sentences without compromising the original semantics, accelerating the inference speed of LLMs while ensuring inference accuracy.




§ METHOD


In this section, we introduce the SentenceVAE framework. Embedding this framework into a LLM enables the model to implement a sentence-by-sentence prediction method, significantly accelerating its inference speed. The inference process of the proposed method compared to published LLMs is illustrated in Fig. <ref>. It is evident that for the same text, an LLM embedded with SentenceVAE requires only three inference steps to obtain the inference result, whereas traditional LLMs necessitate twenty-free inference steps. Furthermore, the proposed method divides the text into sentences without compromising the original semantics, thereby enhancing the inference speed of LLMs while preserving or enhancing accuracy.



 §.§ Sentence Variational Autoencoder (SentenceVAE)


Our SentenceVAE primarily consists of a Sentence Encoder and a Sentence Decoder. The ecoder encodes multiple word-level tokens from a sentence into a single sentence-level token, while the decoder reconstructs this sentence-level token back into the original sequence of word-level tokens. To ensure that the multiple tokens input to the encoder come from individual sentences, we propose a specialized sentence segmentation mechanism. Additionally, we introduce a feature fusion mechanism, which enable the encoder to encode variable-length token sequences into a single sentence-level token.

Sentence Segmentation Mechanism: To ensure that all word-level tokens input to the sentence encoder per time come from a single sentence, we use regular matching to split sentences by punctuation marks such as , , ,  before tokenizing. Let the original string be S, and after partitioning, the set {s_i|1≤ i ≤ n} is obtained, satisfying S=s_1 s_2 s_3 ... s_n.

Sentence Encoder: Taking s = s_t (1≤ t ≤ n) as an example, after tokenization, we obtained a sequence of L token ids D=[d_1, d_2, d_3, ..., d_L] representing a sentence. This sequence was then passed through an embedding layer, resulting in word-level embedding vectors of E, as shown in Eq.<ref>. 


    E = (e_i)_L×hidden_size = Embed(D)


These embeddings were subsequently input into self-attention based encoder blocks, yielding hidden features H, as shown in Eq. <ref>. 


    H = (h_i)_L×hidden_size = EncoderBlocks(E)


To derive the sentence embedding vector, we fused these L features into a single vector Ω_t ∈ℝ^hidden_size. This sentence embedding vector is then fed into the decoder-only LLM, which generates a new sentence embedding vector Ω_t+1 at each time step.

Feature Fusion Mechanism: After the encoder blocks generates H, we propose a method to fuse them into a single sentence embedding vector Ω_t. This method involves accumulating the L vectors and normalizing them using Layer Normalization <cit.>, as shown in Eq. <ref>. 


    Ω_t = LayerNorm(∑_i=1^Lh_i)


Sentence Decoder: The Sentence Decoder contains a combination of masked self-attention and cross-attention blocks, where the cross-attention mechanism uses the individual sentence embedding vector Ω_t+1 as key (K) and value (V).

In the prediction phase, given Ω_t+1, initialize the input token d_0=<bos>, the decoder outputs d_1, and then input [d_0, d_1] and Ω_t+1 into the decoder. Repeat this process until d_l'+1=<eos>, and terminate the iteration, as shown in Eq.<ref>.


    init:     d_0= <bos>, l' = 0 
    do:    
       D_l' = [d_0, d_1, d_2, ..., d_l'] 
       P_l' = (p_i)_(l'+1) ×hidden_size
       = DecoderBlocks(D_l', Ω_t+1) 
        d_l' = argmax(p_l') 
       D_l'+1 = concat(D_l', d_l') 
        l' = l' + 1 
    until:     d_l' = <eos> 
    ret:    D_L'=l'


After the iteration is completed, the detokenization process is used to convert the output D_L' back into a string.

In the training phase, parallel training is adopted, based on masked self-attention mechanism, to ensure that only tokens before d_l' can be seen when inferring d_l'+1. Given an input sequence of D_train=[d_0, d_1, ..., d_L'], model output logits of P∈ℝ^(L'+1) ×hidden_size, and ground truth of D_groud_truth=[d_1, d_2, ..., d_L', <eos>], focal loss <cit.> is taken as the loss function, as shown in Eq. <ref>.


    P   = DecoderBlocks(D_train, Ω_t+1) 
    P'    = softmax(P) 
    Loss   = ∑_i=1^L'-(1-p_d_i)^γlog (p_d_i), γ=2




 §.§ Sentence-level Large Language Models (SLLMs)


Currently, almost all decoder-only LLMs consist of the following components: an embedding layer EMB_llm, N decoder-only blocks DB_llm, and a fully connected layer FC_llm for outputting logits. As shown in Eq. <ref>, given context tokens D_context=[d_0, d_1, ..., d_l'], the model generates the next token d_l'+1.


    E_context   = EMB_llm(D_context) 
    H_llm   = DB_llm(E_context) 
    P_llm   = (p_llm,i)_(l'+1) ×hidden_size
        = FC_llm(H_llm) 
    
            d_l'+1   = argmax(p_llm,l')


In this section, we introduce a unified grafting method to graft SentienceVAE onto the beginning and end of any LLM. This approach transforms the LLMs into SLLMs, enabling it to operate effectively in the sentence-level embedding space.

Discard Embedding Layer: In a SLLM, the N decoder-only blocks no longer receives word-level tokens directly. Instead, it receives sentence embedding vectors encoded by the sentence encoder. Consequently, the traditional embedding layer in the LLM architecture is removed. Therefore, the model follows Eq. <ref>.


    E_sllm   =  [Ω_1, Ω_2, ..., Ω_t] 
    H_sllm   = (h_sllm,i)_t ×hidden_size
       = DB_llm(E_sllm)


Termination judgment layer: In traditional LLMs, the final fully connected layer FC_llm typically converts an H_llm∈ℝ^t ×hidden_size to a probability vector (logits) P_llm∈ℝ^t × V, where V represents the size of the vocabulary. This allows the model to determine the next token output using sampling algorithms, e.g., Greedy Search,  Beam Search <cit.>, Random Sampling, Temperature Sampling <cit.>, Top-K Sampling <cit.>, and Nucleus Sampling <cit.>. If the current token output is a special token like eos (end-of-sequence), the iteration terminates. However, in SLLMs, the output of DB_llm is a sentence-level hidden state vector H_sllm. Therefore, traditional token generation methods are not applicable. Instead, we use a new fully connected layer FC_sllm called termination judgment layer that converts the H_sllm into a 2-dimensional boolean vector B_stop_flag, as shown in Eq. <ref>. This vector helps determine whether the current sentence-level hidden state vector signifies the end of a sentence (stop flag) or needs further decoding by the sentence decoder. If the vector indicates an end flag, the iteration terminates. Otherwise, the sentence decoder processes the embedding to generate corresponding tokens, as shown in Eq. <ref>.


    B_stop_flag   = (b_i)_t × 2
       = FC_sllm(H_sllm)



    Ω_t+1 = 
        
            <eos>    , argmax(b_t) = 1 
    h_sllm, t   , argmax(b_t) = 0


During the training phase, calculate the focal loss for B_stop_flag as part of the global loss.

Inferencing sentence by sentence:  In the SLLM architecture, we use a tiny SentenceVAE model to encode multiple tokens within a sentence into a single sentence embedding vector. This enables the LLM to conduct inference on a sentence-by-sentence basis rather than the traditional token-by-token method.



§ EXPERIMENT


To validate our idea, we first trained individual SentienceVAEs using self supervised methods, demonstrating that multiple tokens in a sentence can be compressed into a single sentence vector by an encoder and restored by a decoder. Subsequently, we grafted the encoder and decoder at the beginning and end of the open-source LLMs. By such simple modifications, LLMs can be upgraded to SLLMs and work in sentence embedding space, while preserving PPL and improving inference speed. Meanwhile, by observing the loss curve, we found that SLLMs still follows the Scaling Law. 



 §.§ Experimental Setting


The experiments described in this paper were conducted using either a single 4-card RTX 4090 (24G) or a 4-card A100 (40G) (for SLLM-1.3B) setup, with training carried out in a data parallel distribution.

Base Models: We use the 125M, 350M, and 1.3B models of the OPT series <cit.> as the base LLMs, and demonstrate the feasibility of extending to larger scale models through Scaling Law <cit.>.

Dataset: We use the English subset (EN/WebText) of the Wanjuan-1.0 dataset <cit.> as the training dataset. The number of training samples depends on the number of iterations, with SentenceVAE training samples consisting of approximately 153.6M sentences (approximately 1.7B tokens); The training sample size of SLLM is about 6.4M paragraphs (approximately 5.6B tokens). The validation set consists of 1,000 random sentences or paragraphs that are mutually exclusive with the training set. 




Hyperparameters: When training SentenceVAE, we assume each sentence has a maximum of 64 tokens, with a batch size of 128/card, a base learning rate of 1e-7/batch/card, and a maximum of 300K iterations. 

For training SLLM, we assume each sentence has a maximum of 64 tokens, each paragraph has a maximum of 64 sentences (due to memory limitations), a batch size of 1/card, a base learning rate of 1e-6/batch/card, and a maximum of 1.6M iterations.

All experiments were conducted using the AdamW optimizer <cit.> with AMP enabled <cit.>, a weight decay coefficient of 0.01, and a maximum L2 norm of 1 for gradient clipping.

The first 5K iterations utilized a linear learning rate schedule, starting at 1/1000 of the base learning rate. After 5K iterations, a cosine annealing learning rate schedule was employed with a period of 20K iterations. Additionally, all experiments implemented the Exponential Moving Average (EMA) strategy.

Metrics: We use PPL as an evaluation metric. The formula for calculating PPL for output logits P of Sentence Decoder and ground truth D_groud_truth is as Eq. <ref>.


    PPL = exp( 1/L'∑_i=1^L'-log(p_d_i))




 §.§ A series of word-level tokens can be represented by a sentence-level embedding with strong robustness


In this experiment, we trained individual SentienceVAEs using a self-supervised approach. If the Sentence Encoder can compress multiple tokens from the input sentence into a single sentence embedding vector and the Sentence Decoder can use this vector to reconstruct the original token sequence, it would validate the feasibility of our theory.

We trained SentenceVAEs with hidden sizes of 768, 1024, and 2048, corresponding to OPT-125M, OPT-350M, and OPT-1.3B, respectively. For each hidden size, we varied the number of block layers to 1, 2, and 4. We evaluated the models using cross-entropy loss and perplexity (PPL) metrics on the validation set. The experimental results are presented in Table <ref>.







The evaluation results demonstrate that SentenceVAEs fulfill their intended purpose, confirming the theoretical validity. Table <ref> presents several test cases. Among these, Samples 1, 2, 3, 5, 6 are convention samples. To assess the model's robustness, we also included several out-of-distribution samples, highlighted in blue in the table:


    
  * Sample 4, 7:   Data preprocessing involves segmenting sentences based on punctuation marks, such as  and . Consequently, these samples will be split into multiple inputs during both training and practical application. Directly inputting these samples without preprocessing is considered invalid input for the model.
    
  * Sample 8:  This is the pinyin of a famous Chinese saying, not English, so it is illegal input.


From the output results of the model, we can analyze that:


    
  * In the convention samples, except for Sample 6, the model accurately restored the sentences. The reason for the error in Sample 6 may be related to the bottleneck of the sinusoidal position encoding mechanism <cit.>.
    
  * Although Sample 4 contains the illegal input, the model has accurately restored it with only punctuation marks. Sample 7 still maintains the meaning of the sentence. In Sample 8, as the hidden size of the model increases, the model also accurately restores the illegal input “yao yao ling xian”. These phenomena indicate that the model is robust to illegal inputs.




 §.§ LLMs can work in setence-level embedding space with faster inference speed, more accurate PPL, and longer context, resulting SLLMs


After verifying the feasibility of Sentence level embedding, we grafted the encoder and decoder of SentenceVAE onto the beginning and end of the OPT series models, corresponding to the relationship shown in Table <ref>.





We did not test SLLM-1.3B-H4 due to memory overflow issues encountered during data-parallel distributed training. The benchmark test results of OPTs and SLLMs are shown in Table <ref>.

Faster inference speed:  We assessed the throughput rate of SLLMs using the PyTorch framework <cit.>, without incorporating optimization methods such as PagedAttention <cit.> or acceleration engines like TensorRT-LLM or LMDeploy <cit.>. The throughput rate of the original OPT model served as the baseline for comparison. The experimental results show that SLLMs is 2∼3× faster on average than LLMs. The larger the number of model parameters, the smaller the proportion of inferencing cost occupied by SentenceVAE, and the more obvious the speed benefit.

More accurate PPL:  We observed that the PPL metric for SLLMs was better than that of the baseline OPT models. This improvement is likely due to the SLLM paradigm enabling the model to process natural language at a higher sentence level, leading to enhanced performance. Additionally, the attention computation in the sentence embedding space allows SLLMs to manage shorter relative contexts for texts of the same length, which further contributes to the improved model performance.

Longer context length:  The context length that a model can handle is primarily constrained by GPU memory. Under the SLLM framework, multiple original tokens can be compressed into a single token, allowing the model to use less memory when processing contexts of the same length. This effectively increases the context length that the model can handle given the same hardware resources.



 §.§ SLLMs still follow the Scaling Law


By analyzing the loss curve of SLLM during the training process, we observed that it adheres to the Scaling Law <cit.>, as shown in Fig. <ref>. This suggests that the SLLM framework can be effectively extended to larger-scale language models through scaling.



§ CONCLUSION & FUTURE TRENDS


In this section, we summarize our work and suggest several potential directions for future development.



 §.§ Scaling up SLLMs with better structures


In this work, to rapidly validate our ideas, we utilized the classic Transformer architecture <cit.> implemented in the PyTorch framework <cit.>. Due to constraints in computing power and time, we limited our validation to models ranging from 125M to 1.3B parameters. However, by verifying the Scaling Law <cit.>, we inferred the applicability of our approach to larger scale models. Numerous state-of-the-art architectural optimizations can be integrated into the SLLM framework, such as Rotational Position Encoding (RoPE) <cit.>, which may address the order problem observed in Sample 6. Currently, our models are trained solely on English corpora, but there is potential to add multilingual support in the future. Overall, while the SLLM framework is still in its early stages, it holds significant potential for further optimization.



 §.§ SLLMs assist in hybrid edge-cloud inference
 

The SLLM framework is inherently a hybrid architecture combining small and large models. By identifying the optimal balance point, the small model (SentenceVAE) and the large model (LLM) can be effectively distributed between the edge and cloud, balancing computational load and enhancing user experience.



 §.§ SLLMs assist in embodied intelligence
 

At present, embodied intelligence based on the LLM Agent paradigm struggles to interact directly with underlying hardware. Instead, they rely on intermediary components to translate high-level LLM instructions into traditional real-time control logic. The primary limitation is the insufficient rate of “token” generation on the edge side.  In contrast, SLLM's capacity to process and generate more “tokens” within the same computational and temporal constraints, offering the potential for embodied intelligent large models to interface directly with underlying hardware.



 §.§ SLLMs assist in multimodal large model
 

The “frames” in videos and “trunks” in audio are, to some extent, equivalent to “tokens” in language models. Therefore, the SLLM paradigm can similarly be applied to multimodal large models to increase the processing “frame rate”, enhance user experience, and  potentially attain performance on par with or exceeding.



§ ACKNOWLEDGMENTS


This research was supported by the China National Key R&D Program (2022YFC2808003) and the Natural Science Basic Research Program of Shaanxi (2024JC-YBMS-468).





 Paper Title: Multi-Level Querying using A Knowledge Pyramid.


Two-Component γ-ray Emission Spectrum and X-Ray Polarization of the Radio Galaxy Pictor A
    [
    August 5, 2024
=========================================================================================

§ ABSTRACT 

This paper addresses the need for improved precision in existing Retrieval-Augmented Generation (RAG) methods that primarily focus on enhancing recall. 

We propose a multi-layer knowledge pyramid approach within the RAG framework to achieve a better balance between precision and recall. 

The knowledge pyramid consists of three layers: Ontologies, Knowledge Graphs (KGs), and chunk-based raw text. 

We employ cross-layer augmentation techniques for comprehensive knowledge coverage and dynamic updates of the Ontology schema and instances. 

To ensure compactness, we utilize cross-layer filtering methods for knowledge condensation in KGs. 

Our approach, named PolyRAG, follows a waterfall model for retrieval, starting from the top of the pyramid and progressing down until a confident answer is obtained. 

We introduce two benchmarks for domain-specific knowledge retrieval, one in the academic domain and the other in the financial domain. 

The effectiveness of the methods has been validated through comprehensive experiments by outperforming 19 SOTA methods.

An encouraging observation is that the proposed method has augmented the GPT-4, providing 395% F1 gain by improving its performance from 0.1636 to 0.8109.






§ INTRODUCTION





Significant advancements have been made in Large Language Models (LLMs), including proprietary models like ChatGPT and GPT-4, as well as open-source variants like FLAN <cit.> and LLaMA <cit.>. 

These models have demonstrated remarkable achievements across a wide range of general knowledge tasks in areas such as language comprehension <cit.>, logical reasoning <cit.>, and complex question-answering <cit.>.

However, the one-size-fits-all nature of general LLMs fails to meet the specific demands for professional or personalized knowledge, such as law <cit.> and finance domains <cit.>. 

A straightforward solution is to utilize Supervised Fine-Tuning (SFT) to tailor LLMs to domain-specific tasks <cit.>. 





Examples include AlpaCare <cit.>, Mental LLaMA <cit.> and Zhongjing <cit.> in medical domain, BloombergGPT <cit.>, Pixiu <cit.>, and DISC-FinLLM <cit.> in financial domain, and DISC-LawLLM <cit.> in law domain.

Nonetheless, this risks catastrophic forgetting <cit.> of the general knowledge when tuning the original model, and is prone to model hallucination <cit.>. 



To address this challenge, the prevalent alternative of Retrieval Augmented Generation (RAG) is introduced as a means to enhance the domain knowledge comprehension of LLMs <cit.>. 

Rather than solely relying on the generation capabilities of LLMs to produce answers, RAG takes a different approach by incorporating information retrieval techniques. 

It retrieves relevant information from existing resources and utilizes this information to enrich the context of the prompt <cit.>. 

This enables in-context learning <cit.> or few-shot learning <cit.> to be initiated based on the enhanced context.

It makes the LLMs' output more stable, accurate, traceable, and interpretable. 

However, early implementations of RAG have relied on unstructured textual data chunks, which are often obtained by partitioning each document in the corpora database into segments with a predefined chunk size and thus not organized in a specific way <cit.>.

To address this limitation, Knowledge Graphs (KGs) have been incorporated into RAG <cit.>.



The incorporation of KGs in RAG is a logical choice, as KGs have long been recognized as a promising augmentation method for retrieval tasks.



In fact, in traditional retrieval tasks, another solution called Ontologies has been found to be suitable for domain-specific tasks due to their formally and strictly defined schemas <cit.>. 

However, the use of Ontologies in RAG has been underexplored, partly because the process of defining the Ontology schema requires significant human effort.

In previous RAG methods, the integrated KGs or Ontologies primarily serve the purpose of incorporating additional domain-specific information, thus enhancing the “recall” by increasing the likelihood of locating relevant information.

However, the aspect of “precision” has not received specific attention in these methods. For instance, Ontologies are recognized for their precision due to the stringent regulations governing schema and SparQL syntax. 

Nonetheless, the Ontology search process does not actively participate in the querying process within existing RAG methods.

Figure <ref> illustrates that different knowledge bases exist at varying levels of specificity, forming a pyramid-like structure. 

The higher layers of the pyramid contain more specific and structured information, providing a greater likelihood of accurately answering domain-specific questions. Hence, these higher layers are considered more “precision”-friendly.

On the other hand, the lower layers of the pyramid are more inclined towards “recall”-friendliness. 

They possess more flexible definitions and can store richer information, albeit potentially sacrificing precision.














The observation serves as motivation for us to build a multi-layer knowledge pyramid and devise a customized querying strategy within the RAG framework.

The main objective of this approach is to achieve a better balance between “precision” and “recall”.

Specifically, we organize a knowledge pyramid with three distinct layers of Ontologies, knowledge graphs, and chunk-based raw text.

During the construction of the knowledge pyramid, we ensure comprehensive coverage of knowledge by incorporating cross-layer augmentation techniques.

This approach reduces the need for human intervention in knowledge completion, as it allows for dynamic updates of the Ontology schema and instances.

Moreover, we maintain compactness by employing cross-layer filtering methods, which remove the redundant triplets in the KG for knowledge condensation.

The retrieval process follows a waterfall model, starting from the top of the pyramid and progressively moving down until the model reaches a confident answer. 

If certainty is not achieved, the search continues to the next layer down, and so on.

We name it as PolyRAG due to its nature of multi-layer knowledge organization and multi-level querying strategy. 




Furthermore, this paper presents a contribution by introducing two benchmarks for domain-specific knowledge retrieval. 

The first benchmark is specifically designed for the academic domain and has been meticulously constructed by the authors themselves. 

It encompasses extensive information regarding 1,319 staff members, 2,061 courses, 31 departments, as well as classrooms and facilities at XXX university.

In addition, the authors have extended an existing public dataset called FiQA <cit.> in the financial domain to create the second benchmark. 

We restructured the knowledge from this dataset into a similar pyramid structure, allowing for the application of the proposed methodology.

Both benchmarks will be made available to the community. 































































§ RELATED WORKS





 §.§ Domain-Specific Large Language Models







LLMs have become integral in various applications, such as chatbots, writing assistants, customer service automation <cit.>.

Domain-specific LLMs are a subset of LLMs that have been tailored to understand and generate text within a particular area of expertise, such as law <cit.>, medicine <cit.>, or finance <cit.>. These models aim to provide higher accuracy and more relevant outputs than general-purpose LLMs when dealing with specialized content <cit.>.



A common approach is to adapt instruction fine-tuning based on the general LLMs <cit.> using domain-specific copra. 



However, research has shown that the generality of the fine-tuned model will diminish, resulting in catastrophic forgetting <cit.>. 

In order to tackle the limitations, the approach of prompting LLMs with extra domain knowledge as contexts is widely adopted.



 §.§ Retrieval-Augmented Generation


Retrieval-Augmented Generation (RAG) enhances the generative capabilities of language models by incorporating retrieved knowledge for in-context learning <cit.>. 







NaiveRAG represents the most basic architecture within this framework, in which the system retrieves the top-k documents that are most relevant to the query and integrate them into the prompt, thereby grounding the responses in more relevant information <cit.>. 

Expanding on NaiveRAG, advanced RAG incorporates additional modules or structures to improve retrieval precision. Reranking is a notable example, where a reranker is employed to refine the initial ranked list (e.g., Re2G <cit.> and bge-reranker <cit.>, both are based on BERT <cit.>).

Furthermore, studies have indicated that excessive noise and lengthy context can have a negative impact on inference performance. To address this, prompt compression methods such as Selective Context <cit.> and LLMLingua <cit.> have been developed. 

These methods emphasize key information while reducing noise and context length, as discussed in <cit.>.





 §.§ Knowledge-Augmented Language Models

The Knowledge-Augmented Language Model (LM) approach involves integrating LMs with additional knowledge bases to facilitate in-context learning <cit.>. 

In addition to incorporating knowledge in the form of raw texts <cit.>, knowledge graphs (KGs) <cit.> have gained popularity, and Ontologies <cit.> have also been utilized, albeit less frequently.

In most KG augmentation methods <cit.>, the RAG framework is followed, wherein the context is expanded by incorporating retrieved KG triples instead of raw text chunks. 

KAPING <cit.> serves as an early example of this approach, which has been later refined in RRA <cit.>.

Regarding Ontologies, instead of being used as an individual knowledge base, they are often employed as assistants for generating KG triplets <cit.> (e.g., Text2KGBench <cit.>) or for augmenting textual corpora (e.g., EKGs <cit.>, OntoChatGPT <cit.>).

The integration of various forms of knowledge bases poses a significant challenge and remains an area that has received limited exploration.










§ KNOWLEDGE PYRAMID CONSTRUCTION



Let us establish the knowledge pyramid as the foundational base for PolyRAG.

The construction process begins by creating three distinct knowledge banks, each guided or supported by an Ontology, a knowledge graph, and the raw texts, following common practices.

These banks (denoted as 𝒪, 𝒦, and 𝒯, respectively) form the initial layers of the pyramid.

The essence of our proposed methodology lies in fostering interactions between these layers, aimed at enhancing the overall comprehensiveness and compactness of the knowledge base.





 §.§ Construction of Initial Layers












Ontology Layer: 

The Ontology layer 𝒪={𝒪_s,𝒪_i} consists of a schema 𝒪_s and corresponding instances 𝒪_i.

Defining an Ontology schema typically requires significant time and effort from human experts, and achieving a comprehensive schema can be challenging.

To simplify the process, one approach is to initially extract a sub-domain schema from general Ontologies like WordNet <cit.> or ConceptNet <cit.>.

This extracted schema can serve as a starting point and be refined using the semi-automatic approach presented in Section <ref>.



With the schema 𝒪_s that we extracted, we can guide LLMs to extract instances from the raw text layer 𝒯 for each of the concept-attribute pair (c,a)∈𝒪_s, where c is a concept (e.g., professor) and a is one of its attributes (e.g., research_interest).

This can be written as a prompt function

[boxrule=0pt, frame empty]
    f_ins(c, a; p): Given a paragraph {p} from the {domain} domain, please identify instances of the Ontology relationship where a class {c} has the attribute of {a}. Note that the attribute may consist of multiple entities.


Executing this function results in instances that fulfill the specified relationship. 

By repeatedly applying this function to each paragraph, the set 𝒪_i is constructed as


    𝒪_i={f_ins(c, a; p) | ∀(c, a)∈𝒪_s, ∀ p∈𝒯}.


It is important to note that the specific implementation of the prompt may vary among LLMs.

Additionally, in certain cases, it may be beneficial to provide examples to initiate few-shot learning for extracting high-quality information, depending on the capabilities of the LLMs.

Our implementations are available in the Github repository.

Knowledge Graph Layer: 

We adopt Open Information Extraction (OpenIE) <cit.> to extract KG triples from raw texts. 

However, direct extraction often results in noisy output, including irrelevant or duplicate entities. To address this, we draw inspiration from the multi-round prompting approach of LLM2KG <cit.> and reimplement it by introducing four functions: f_par(p) for paraphrasing, f_ent() for entity extraction, f_rel() for relation completion, and f_dis() for disambiguation.

These functions form a cascade for KG triplet extraction and refinement as


    f_kg(p)=f_dis(f_ent(f_par(p)), f_rel(f_par(p))).


It constructs the initial knowledge graph layer as


    𝒦={f_kg(p) | ∀ p∈𝒯}.


Due to space limitations, the implementations of these functions are provided in the Appendix <ref>.






 §.§ Knowledge Completion



As shown in the pyramid of Figure <ref>, the higher layers are more structured but not easy to define or extract, resulting in limited coverage.

One common issue is the absence of important classes or attributes from the expert-defined Ontology schema, which has historically hindered the ease of defining Ontologies.

By utilizing the pyramid framework, we can address this issue in a data-driven manner.

The idea is to identify noteworthy concepts and relations that exist in the lower layers but are absent from the higher layers. 

These identified elements are then incorporated into the higher layers to enhance knowledge completion.

For instance, our research reveals that the relationship publications_in_important_journals appears frequently in both the knowledge graph and raw texts but is absent from the Ontology.

This is due to the tendency of human-defined schema to overlook this specific relationship, as the presence of the publications attribute may make it seem redundant.

However, users often query this specific relation, and it is explicitly mentioned in professors' profiles.

It is thus worth including it in the Ontology.

Our approach involves modeling the semantic distributions of the Ontology and knowledge graph to identify concepts and relations that exhibit significant divergence between the two layers. 

To achieve this, we begin by transforming class-attribute pairs in the Ontology layer 𝒪 into subject+relation phrases that align with the format in the knowledge graph layer 𝒦.

Instructor embedding <cit.> of phrases at both layers are then encoded so as to project them into a common semantic space.

We then learn for each layer a multivariate Gaussian using


    F(𝐗)   =1/(2π)^n/2|Σ^1/2|e^-1/2(𝐗-μ)^⊤Σ^-1(𝐗-μ),


where n is the dimension of the space, 𝐗 denotes a phrase embedding, and μ and Σ are the mean and covariance matrix, respectively.

The semantic distributions of the Ontology and knowledge graph are then modeled as 𝐗_𝒪∼𝒩(μ_𝒪,Σ_𝒪) and 𝐗_𝒦∼𝒩(μ_𝒦,Σ_𝒦), respectively.

The Kullback–Leibler (KL) divergence can then be measured on each given phrase embedding 𝐗 as


    D_𝐗(𝒪∥𝒦)   =-F_𝒪(𝐗)log(F_𝒦(𝐗))
    
        D_𝐗(𝒦∥𝒪)   =-F_𝒦(𝐗)log(F_𝒪(𝐗)),


where D_𝐗(𝒪∥𝒦) measures the relative divergence at the point 𝐗 by the distribution of 𝒦 as the reference, while in D_𝐗(𝒦∥𝒪), the distribution of 𝒦 is used as the reference. 

At a specific location 𝐗, we define a function to rate the priority to extract entities from the knowledge graph to the Ontology as


    Pr(𝐗)=D_𝐗(𝒪∥𝒦)-D_𝐗(𝒦∥𝒪).


The rating outcomes are depicted in Figure <ref>, illustrating that this function effectively identifies areas where the knowledge graph contains dense information that is comparatively lacking in the Ontology.

We then apply k-medoids clustering on knowledge graph triplets within these area to locate new Ontology class and attributes that can be integrated into the Ontology schema, along with corresponding instances.

This process can be performed iteratively, allowing for continuous refinement and improvement. 

As a result of these interactions between the two layers, the comprehensiveness of the pyramid is progressively enhanced.


















 §.§ Knowledge Condensation


The process of Knowledge Completion aims to enhance the richness of information in a bottom-up manner. 

In addition, we perform Knowledge Condensation to improve the compactness of the pyramid in a top-down approach. 

The principle is to leverage the well-structured knowledge in the higher layers to eliminate redundant information present in the lower layers.

This is implemented by using each class-attribute in the Ontology layer as an anchor and explore its neighborhood to identify a set of k nearest knowledge graph triplets {𝐗_𝐤}.

A prompt function is then applied to instruct the LLM to summarize them into compact ones as

[boxrule=0pt, frame empty]
    f_con({𝐗_𝐤}): Please condense the set of knowledge graph triplets {𝐗_𝐤} obtained from the {domain} domain by eliminating redundant triplets and summarizing the remaining ones in a more concise manner. Here are some examples for your reference: {examples}. The condensation process should follow the logic of {CoT}.


By repeating this process for all anchors, we obtain the condensed knowledge graph layer, as depicted in Figure <ref>. 

The original triplets such as “Prof. A works in Z” and “Prof. A is with Z institue” have been merged into more compact form as a relation of “is_a_member_of”.

It is evident that the compactness of the pyramid has been significantly improved.






























































































































































§ MULTI-LEVEL QUERYING


The pyramid enables PolyRAG in a straightforward top-down querying manner, following the flow Ontology→Knowledge Graph→Raw Texts, in which the results are returned if answers are found at higher layers; otherwise, the querying continues to the next layer.

The querying flow is depicted in Algorithm <ref>.

Two essential components to build to support the logic are the ways of searching at each layer and conditions to dive next.

The Raw Texts layer can be effectively addressed by employing the NaiveRAG approach.

Regarding the design of the approaches for the remaining two layers, we propose the following strategies.






 §.§ Search at Ontology Layer

At the Ontology Layer, we can utilize SparQL as the query language. 

SparQL is a well-defined language that provides precise results when queries are properly structured.

To achieve this, we can guide the LLM to generate the SparQL query by using the following prompt function

[boxrule=0pt, frame empty]
Query_onto(Q, schema): Given a input question {Q} within the domain {domain}, please formulate a SPARQL query to retrieve the answer based on the provided Ontology schema. The namespace is {schema.base}, and the classes are {schema.class}. The object properties between classes are {schema.op}, and the classes may also have data properties such as {schema.dp}. 





For the example of a query question like “Who is currently working in CS Department and was graduated from Cambridge University?”, the LLM extracts the related attributes of “works_in” and “graduated_from” with the given schema, and generate a SPARQL query to search a staff who satisfies the relevant condition in 𝒪. The executing results may include a list of names, or simply empty due to either the lack of knowledge in Ontology layer, or the mistake of providing unavailable SPARQL due to the limitation of LLM itself.



 §.§ Search at Knowledge Graph Layer


At the knowledge graph layer, we employ a retrieval approach similar to the embedding-based retrieval utilized in NaiveRAG. 

However, instead of using chunks, we work with triplets.

Once the matching triplets are retrieved, we utilize a prompt function to assess the Language Model's agreement on whether the question has been answered adequately as

[boxrule=0pt, frame empty]
Query_kg(Q, triples): Given a question {Q} and the context information provided by the matched triples from the knowledge graph {triples}, please justify whether the provided information is sufficient to accurately answer the question. Respond with either “Yes” or “No” to provide your justification.





For example, given a query of “which CS staff has interest in cloud computing?”, the LLM may justify a set of triples as “agree” if they include information like “Prof. A works in CS Department, Prof. A published on cloud computing journal, ... etc.”; inversely, the justification might be “disagree” if the triples do not provide the key knowledge related to this query, thus the searching will process to the next layer for a more relevant context in order to answer the query.























































































§ EXPERIMENTS




 §.§ Experimental Setup

Our experimentation involves two distinct domain-specific benchmarks. 

The first benchmark is Academia Challenge (AcadChall), which has been built by ourselves and focuses on the academic domain. 

It encompasses a comprehensive collection of data obtained from XXX university, including information about 31 departments, 1,319 faculty members, and 2,061 courses. 

Specifically, AcadChall consists of 512 MCQ and MAQ questions that cover topics related to teaching and research. 

These questions are intentionally designed to be more challenging compared to existing benchmarks, as each question presents eight answer choices. 

This design aims to provide a more rigorous assessment of the models' precision.

The second benchmark is FiQA, which is widely recognized and utilized in the finance domain. Our experimentation follows the dataset split provided by FLUE benchmark <cit.>, resulting in the creation of the R-FLUE-FiQA dataset comprising 1,705 open-ended questions.

To enhance knowledge completion and condensation, we employ the methods proposed in Section <ref> to construct Ontologies, knowledge graphs, and pyramids based on the two datasets. 
For more detailed information about the two benchmarks, please refer to the Appendix <ref>.


We employ instructor-xl <cit.> as our vector base for embedding search and the cosine similarity is used as the metric <cit.>.

The k values for KG layer and raw texts layer are empirically set to 10 and 5, respectively.


















 §.§ Evaluation Metrics

Our evaluation process encompasses several metrics to assess the performance, including: 

1) For MCQs and MAQs, we employ metrics such as Precision, Recall, and F1 score. 

2) For open-ended questions, we utilize BLEU and BERT similarity, calculated based on MiniLM embeddings <cit.>.

3) We introduce a novel metric called HitRate (HR), which quantifies the proportion of correct entities present in the response.

These metrics collectively allow for a comprehensive assessment of the system's effectiveness across different question types and response formats. 
































 §.§ Comparison to SOTA Methods


We compare PolyRAG with five groups of state-of-the-art (SOTA) methods, including 1) Frozen-LLMs that are pretrained LLMs of frozen parameters and  no external knowledge has been used for querying.

2) SFT (Supervised Fine-Tuning) that are LLMs that undergo supervised fine-tuning on domain-specific datasets.

We include two latest implementation of Finance-LLM and Finance-Chat <cit.>, both of which are trained on financial corpora. 

Additionally, we explore variants of SFT that employ LoRA <cit.> instead of the entire models.

3) NaiveRAG proposed in <cit.>.

4) Advanced RAG that include two recent models, namely ColBERTv2 <cit.> and Bge-reranker <cit.>.

5) KG-Augmented LLMs that consists of models proposed in <cit.> that integrate knowledge graphs (KGs) into the LLMs.

Various LLMs have been explored as the backbones, including LLaMA <cit.>, LLaMA2 <cit.>, Vicuna <cit.>, Gemini <cit.>, GPT-3.5 and GPT-4.

By combining them with the five groups and PolyRAG, this results in a comprehensive set of 23 methods for performance comparison.

The results are shown in Table <ref>. 

It is clear that PolyRAG surpasses other SOTA methods across the five groups, different backbones, and both benchmarks.

An encouraging finding is that PolyRAG demonstrates a superior balance between precision and recall.

Among the improvements of PolyRAG over others, the gain on the precision (22.0%± 17.3) is more pronounced than that of recall (18.4%± 18.6).

With the incorporation of PolyRAG, GPT-4 (GPT-3.5) has achieved the highest gains by 338.6% (133.2%) over its frozen model and 73.1% (69.8%) over its NaiveRAG run on the AcadChall benchmark.

The improvement may result from the fact that around 43.7% (38.4%) of questions are addressed at the Ontology layer, and around 42.3% (44.8%) are at the KG layer on the AcadChall (R-FLUE-FiQA) benchmark.




















































































































































































































































 §.§ Knowledge Completion and Condensation


We have conducted experiments to evaluate the effectiveness of knowledge completion (CPL) and condensation (CND) by integrating them with a baseline model (PolyRAG without these two modules).

The results are presented in Table <ref>.

Please note that due to space limitations, we only list results for four specific backbones.

For a comprehensive performance analysis with Vicuna-13B and LLaMA2-70B backbones, please refer to the Appendix <ref>.

The results clearly demonstrate the effectiveness of both completion and condensation in enhancing performance, whether used individually (CPL: 3.8%± 6.1 performance gain, CND: 2.2%± 3.3 performance gain) or in combination (6.5%± 6.4 performance gain over the baseline).

Our statistics indicate that with completion, queries resolved at the Ontology layer is increased by 5%.

With condensation, more than 27% of queries are resolved at the KG layer.








































 §.§ Influence of Knowledge Layers


We have examined the influence of different knowledge layers on various backbones.

The results are presented in Table <ref>.

It is obvious that the Ontology layer plays a prominent role as a knowledge base in enhancing precision, as evidenced by a precision gain of 25.8%± 7.4 when combined with the raw text layer and 13.7%± 10.3 when combined with the KG layer.

On the other hand, the KG layer demonstrates a balanced impact on improving either precision or recall.

These observations align with our early discussions.








































































































































































§ CONCLUSION

This research has described a preliminary investigation that provides a multi-level query approach to domain-specific question answering. In this context, we addressed two significant challenges: the problem of prioritizing knowledge in a certain area and the presence of noise in retrieval situations. In order to overcome the limitations, we introduce Knowledge Pyramid multi-level retrieval framework as PolyRAG. PolyRAG utilizes a sequential retrieval approach and prioritizes knowledge extraction. This approach is designed to tackle the challenges posed by the high demand for dense knowledge in domain-specific scenarios and the distractions caused by noisy contexts. The accuracy of PolyRAG has been validated through extensive tests done in AcadChall and R-FLUE-FiQA, surpassing earlier approaches and generating state-of-the-art results.









§ LIMITATIONS

One limitation to consider is that Language Models (LLMs) do not strictly adhere to the SparQL syntax, which can result in typographical errors in the queries.

Further investigation into this aspect is warranted.



§ ETHICS STATEMENT

This research paper adheres to ethical considerations throughout its methodology and findings. The study primarily focuses on the development and evaluation of a retrieval-augmented generation framework, PolyRAG, for domain-specific knowledge retrieval. The research involves the use of existing datasets and benchmarks, as well as the creation of two new benchmarks in the academic and financial domains.

The authors ensure proper citation and acknowledgment of the data sources to maintain transparency and intellectual property rights. In terms of human subjects, this research does not involve any direct human participation, personal data collection, or sensitive information. Therefore, ethical concerns related to informed consent, privacy, and confidentiality are not applicable in this context.











60


[Abu-Rasheed et al.(2024)Abu-Rasheed, Weber, and Fathi]1-kg-recom
Hasan Abu-Rasheed, Christian Weber, and Madjid Fathi. 2024.
https://arxiv.org/abs/2403.03008 Knowledge graphs as context sources for llm-based explanations of learning recommendations.
Preprint, arXiv:2403.03008.

[Anderson et al.(2022)Anderson, Wilson, and Richardson]2-1-lingua
Nathan Anderson, Caleb Wilson, and Stephen D Richardson. 2022.
Lingua: Addressing scenarios for live interpretation and automatic dubbing.
In Proceedings of the 15th Biennial Conference of the Association for Machine Translation in the Americas (Volume 2: Users and Providers Track and Government Track), pages 202–209.

[Baek et al.(2023a)Baek, Aji, and Saffari]1-14-kaping
Jinheon Baek, Alham Fikri Aji, and Amir Saffari. 2023a.
Knowledge-augmented language model prompting for zero-shot knowledge graph question answering.
arXiv preprint arXiv:2306.04136.

[Baek et al.(2023b)Baek, Aji, and Saffari]KGRAG2023Baek
Jinheon Baek, Alham Fikri Aji, and Amir Saffari. 2023b.
Knowledge-augmented language model prompting for zero-shot knowledge graph question answering.
In Proceedings of the 1st Workshop on Natural Language Reasoning and Structured Explanations (NLRSE), pages 78–106.

[Baldazzi et al.(2023)Baldazzi, Bellomarini, Ceri, Colombo, Gentili, and Sallinger]baldazzi2023finetuneontologicalreasoning
Teodoro Baldazzi, Luigi Bellomarini, Stefano Ceri, Andrea Colombo, Andrea Gentili, and Emanuel Sallinger. 2023.
Fine-tuning large enterprise language models via ontological reasoning.
In International Joint Conference on Rules and Reasoning, pages 86–94. Springer.

[Carta et al.(2023)Carta, Giuliani, Piano, Podda, Pompianu, and Tiddia]3-3-llm2kg
Salvatore Carta, Alessandro Giuliani, Leonardo Piano, Alessandro Sebastian Podda, Livio Pompianu, and Sandro Gabriele Tiddia. 2023.
Iterative zero-shot llm prompting for knowledge graph construction.
arXiv preprint arXiv:2307.01128.

[Chang et al.(2024)Chang, Wang, Wang, Wu, Yang, Zhu, Chen, Yi, Wang, Wang et al.]chang2024evaluation_survey
Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, et al. 2024.
A survey on evaluation of large language models.
ACM Transactions on Intelligent Systems and Technology, 15(3):1–45.

[Chen et al.(2023)Chen, Wang, Long, Zhang, Lu, Li, Wang, Xu, Bai, Huang, and Wei]4-3-discfinllm
Wei Chen, Qiushi Wang, Zefei Long, Xianyin Zhang, Zhongtian Lu, Bingxuan Li, Siyuan Wang, Jiarong Xu, Xiang Bai, Xuanjing Huang, and Zhongyu Wei. 2023.
https://arxiv.org/abs/2310.15205 Disc-finllm: A chinese financial large language model based on multiple experts fine-tuning.
Preprint, arXiv:2310.15205.

[Cheng et al.(2024)Cheng, Huang, and Wei]financeChatLLM2024Cheng
Daixuan Cheng, Shaohan Huang, and Furu Wei. 2024.
Adapting large language models via reading comprehension.
In The Eleventh International Conference on Learning Representations.

[Chiang et al.(2023)Chiang, Li, Lin, Sheng, Wu, Zhang, Zheng, Zhuang, Zhuang, Gonzalez, Stoica, and Xing]4-3-vicuna
Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023.
https://lmsys.org/blog/2023-03-30-vicuna/ Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality.

[Cui et al.(2023)Cui, Li, Yan, Chen, and Yuan]2-chatlaw
Jiaxi Cui, Zongjian Li, Yang Yan, Bohua Chen, and Li Yuan. 2023.
https://arxiv.org/abs/2306.16092 Chatlaw: Open-source legal large language model with integrated external knowledge bases.
Preprint, arXiv:2306.16092.

[Dong et al.(2023)Dong, Li, Dai, Zheng, Wu, Chang, Sun, Xu, Li, and Sui]1-incontext-survey
Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu Sun, Jingjing Xu, Lei Li, and Zhifang Sui. 2023.
https://arxiv.org/abs/2301.00234 A survey on in-context learning.
Preprint, arXiv:2301.00234.

[Etzioni et al.(2008)Etzioni, Banko, Soderland, and Weld]3-3-openie
Oren Etzioni, Michele Banko, Stephen Soderland, and Daniel S Weld. 2008.
Open information extraction from the web.
Communications of the ACM, 51(12):68–74.

[Gao et al.(2023)Gao, Xiong, Gao, Jia, Pan, Bi, Dai, Sun, and Wang]2-1-ragreview
Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, and Haofen Wang. 2023.
Retrieval-augmented generation for large language models: A survey.
arXiv preprint arXiv:2312.10997.

[Ge et al.(2024)Ge, Hua, Mei, Tan, Xu, Li, Zhang et al.]2-domainspecific
Yingqiang Ge, Wenyue Hua, Kai Mei, Juntao Tan, Shuyuan Xu, Zelong Li, Yongfeng Zhang, et al. 2024.
Openagi: When llm meets domain experts.
Advances in Neural Information Processing Systems, 36.

[Glass et al.(2022)Glass, Rossiello, Chowdhury, Naik, Cai, and Gliozzo]2-1-re2g
Michael Glass, Gaetano Rossiello, Md Faisal Mahbub Chowdhury, Ankita Rajaram Naik, Pengshan Cai, and Alfio Gliozzo. 2022.
Re2g: Retrieve, rerank, generate.

[Hadi et al.(2023)Hadi, Qureshi, Shah, Irfan, Zafar, Shaikh, Akhtar, Wu, Mirjalili et al.]hadi2023llmsurvey
Muhammad Usman Hadi, Rizwan Qureshi, Abbas Shah, Muhammad Irfan, Anas Zafar, Muhammad Bilal Shaikh, Naveed Akhtar, Jia Wu, Seyedali Mirjalili, et al. 2023.
A survey on large language models: Applications, challenges, limitations, and practical usage.
Authorea Preprints.

[Hu et al.(2022)Hu, Shen, Wallis, Allen-Zhu, Li, Wang, Wang, and Chen]4-3-lora
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022.
Lora: Low-rank adaptation of large language models.

[Huang et al.(2023a)Huang, Yu, Ma, Zhong, Feng, Wang, Chen, Peng, Feng, Qin et al.]1-10-hallucination
Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, et al. 2023a.
A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions.
arXiv preprint arXiv:2311.05232.

[Huang et al.(2023b)Huang, Tao, Zhang, An, Jiang, Chen, Wu, and Feng]2-lawyer
Quzhe Huang, Mingxu Tao, Chen Zhang, Zhenwei An, Cong Jiang, Zhibin Chen, Zirui Wu, and Yansong Feng. 2023b.
https://arxiv.org/abs/2305.15062 Lawyer llama technical report.
Preprint, arXiv:2305.15062.

[Kenton and Toutanova(2019)]2-1-bert
Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. 2019.
Bert: Pre-training of deep bidirectional transformers for language understanding.
1:2.

[Kojima et al.(2022)Kojima, Gu, Reid, Matsuo, and Iwasawa]kojima2022zero_shot_reasoner
Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022.
Large language models are zero-shot reasoners.
Advances in neural information processing systems, 35:22199–22213.

[Lewis et al.(2020a)Lewis, Perez, Piktus, Petroni, Karpukhin, Goyal, Küttler, Lewis, Yih, Rocktäschel et al.]1-11-lewis2020rag
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020a.
Retrieval-augmented generation for knowledge-intensive nlp tasks.
Advances in Neural Information Processing Systems, 33:9459–9474.

[Lewis et al.(2020b)Lewis, Perez, Piktus, Petroni, Karpukhin, Goyal, Küttler, Lewis, Yih, Rocktäschel et al.]2-1-rag
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020b.
Retrieval-augmented generation for knowledge-intensive nlp tasks.
Advances in Neural Information Processing Systems, 33:9459–9474.

[Litman et al.(2020)Litman, Anschel, Tsiper, Litman, Mazor, and Manmatha]2-1-selective
Ron Litman, Oron Anschel, Shahar Tsiper, Roee Litman, Shai Mazor, and R Manmatha. 2020.
Scatter: selective context attentional scene text recognizer.
In proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11962–11972.

[Liu et al.(2019)Liu, Du, and Stoyanov]Liu2019KnowledgeAugmentedLM
Angli Liu, Jingfei Du, and Veselin Stoyanov. 2019.
https://api.semanticscholar.org/CorpusID:104292554 Knowledge-augmented language model and its application to unsupervised named-entity recognition.
In North American Chapter of the Association for Computational Linguistics.

[Luo et al.(2023)Luo, Yang, Meng, Li, Zhou, and Zhang]1-9-forget
Yun Luo, Zhen Yang, Fandong Meng, Yafu Li, Jie Zhou, and Yue Zhang. 2023.
An empirical study of catastrophic forgetting in large language models during continual fine-tuning.
arXiv preprint arXiv:2308.08747.

[Ma et al.(2023)Ma, Gong, He, hai zhao, and Duan]2-1-naiverag
Xinbei Ma, Yeyun Gong, Pengcheng He, hai zhao, and Nan Duan. 2023.
Query rewriting in retrieval-augmented large language models.
In The 2023 Conference on Empirical Methods in Natural Language Processing.

[Mihindukulasooriya et al.(2023)Mihindukulasooriya, Tiwari, Enguix, and Lata]mihindukulasooriya2023text2kgbench
Nandana Mihindukulasooriya, Sanju Tiwari, Carlos F Enguix, and Kusum Lata. 2023.
Text2kgbench: A benchmark for ontology-driven knowledge graph generation from text.
In International Semantic Web Conference, pages 247–265. Springer.

[Miller(1995)]miller1995wordnet
George A Miller. 1995.
Wordnet: a lexical database for english.
Communications of the ACM, 38(11):39–41.

[Ouyang et al.(2022)Ouyang, Wu, Jiang, Almeida, Wainwright, Mishkin, Zhang, Agarwal, Slama, Ray et al.]1-2-openaiSFT
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022.
Training language models to follow instructions with human feedback.
Advances in Neural Information Processing Systems, 35:27730–27744.

[Palagin et al.(2023)Palagin, Kaverinskiy, Litvin, and Malakhov]1-ontochatgpt
Oleksandr Palagin, Vladislav Kaverinskiy, Anna Litvin, and Kyrylo Malakhov. 2023.
Ontochatgpt information system: Ontology-driven structured prompts for chatgpt meta-learning.
arXiv preprint arXiv:2307.05082.

[Pan et al.(2024)Pan, Luo, Wang, Chen, Wang, and Wu]3-llmandkg
Shirui Pan, Linhao Luo, Yufei Wang, Chen Chen, Jiapu Wang, and Xindong Wu. 2024.
Unifying large language models and knowledge graphs: A roadmap.
IEEE Transactions on Knowledge and Data Engineering.

[Radford et al.(2018)Radford, Narasimhan, Salimans, Sutskever et al.]radford2018openaiGPT
Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. 2018.
Improving language understanding by generative pre-training.

[Rebedea et al.(2023)Rebedea, Dinu, Sreedhar, Parisien, and Cohen]2-app2
Traian Rebedea, Razvan Dinu, Makesh Sreedhar, Christopher Parisien, and Jonathan Cohen. 2023.
Nemo guardrails: A toolkit for controllable and safe llm applications with programmable rails.
arXiv preprint arXiv:2310.10501.

[Sallam(2023)]2-app1
Malik Sallam. 2023.
Chatgpt utility in healthcare education, research, and practice: systematic review on the promising perspectives and valid concerns.
In Healthcare, volume 11, page 887. MDPI.

[Santhanam et al.(2022)Santhanam, Khattab, Potts, and Zaharia]4-3-colbertv2
Keshav Santhanam, Omar Khattab, Christopher Potts, and Matei Zaharia. 2022.
Plaid: an efficient engine for late interaction retrieval.

[Shah et al.(2022)Shah, Chawla, Eidnani, Shah, Du, Chava, Raman, Smiley, Chen, and Yang]4-1-fiqa
Raj Sanjay Shah, Kunal Chawla, Dheeraj Eidnani, Agam Shah, Wendi Du, Sudheer Chava, Natraj Raman, Charese Smiley, Jiaao Chen, and Diyi Yang. 2022.
When flue meets flang: Benchmarks and large pretrained language model for financial domain.
In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics.

[Speer et al.(2017)Speer, Chin, and Havasi]speer2017conceptnet
Robyn Speer, Joshua Chin, and Catherine Havasi. 2017.
Conceptnet 5.5: An open multilingual graph of general knowledge.
In Proceedings of the AAAI conference on artificial intelligence, volume 31.

[Su et al.(2022)Su, Shi, Kasai, Wang, Hu, Ostendorf, Yih, Smith, Zettlemoyer, and Yu]INSTRUCTOR
Hongjin Su, Weijia Shi, Jungo Kasai, Yizhong Wang, Yushi Hu, Mari Ostendorf, Wen-tau Yih, Noah A. Smith, Luke Zettlemoyer, and Tao Yu. 2022.
https://arxiv.org/abs/2212.09741 One embedder, any task: Instruction-finetuned text embeddings.

[Team et al.(2023)Team, Anil, Borgeaud, Wu, Alayrac, Yu, Soricut, Schalkwyk, Dai, Hauth et al.]geminiteam2024gemini
Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al. 2023.
Gemini: a family of highly capable multimodal models.

[Touvron et al.(2023a)Touvron, Lavril, Izacard, Martinet, Lachaux, Lacroix, Rozière, Goyal, Hambro, Azhar et al.]1-4-llama
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023a.
Llama: Open and efficient foundation language models.
arXiv preprint arXiv:2302.13971.

[Touvron et al.(2023b)Touvron, Martin, Stone, Albert, Almahairi, Babaei, Bashlykov, Batra, Bhargava, Bhosale et al.]4-3-llama2
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023b.
Llama 2: Open foundation and fine-tuned chat models.

[Vamsi et al.(2024)Vamsi, Kommineni, and Samuel]1-13-onto
Krishna Kommineni Vamsi, Vamsi Krishna Kommineni, and Sheeba Samuel. 2024.
From human experts to machines: An llm supported approach to ontology and knowledge graph construction.

[Wang et al.(2023)Wang, Yang, and Wang]4-3-fingpt
Neng Wang, Hongyang Yang, and Christina Dan Wang. 2023.
Fingpt: Instruction tuning benchmark for open-source large language models in financial datasets.
NeurIPS Workshop on Instruction Tuning and Instruction Following.

[Wang et al.(2020a)Wang, Wei, Dong, Bao, Yang, and Zhou]MiniLM2020Wang
Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, and Ming Zhou. 2020a.
Minilm: deep self-attention distillation for task-agnostic compression of pre-trained transformers.
In Proceedings of the 34th International Conference on Neural Information Processing Systems.

[Wang et al.(2020b)Wang, Yao, Kwok, and Ni]1-fewshot-survey
Yaqing Wang, Quanming Yao, James T Kwok, and Lionel M Ni. 2020b.
Generalizing from a few examples: A survey on few-shot learning.
ACM computing surveys (csur), 53(3):1–34.

[Wei et al.(2021)Wei, Bosma, Zhao, Guu, Yu, Lester, Du, Dai, and Le]1-3-flan
Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. 2021.
Finetuned language models are zero-shot learners.
arXiv preprint arXiv:2109.01652.

[Wilkinson and Hingston(1991)]cosineSimilarity1991
Ross Wilkinson and Philip Hingston. 1991.
Using the cosine measure in a neural network for document retrieval.
In Proceedings of the 14th annual international ACM SIGIR conference on Research and development in information retrieval, pages 202–210.

[Wu et al.(2023a)Wu, Irsoy, Lu, Dabravolski, Dredze, Gehrmann, Kambadur, Rosenberg, and Mann]1-8-bloomberggpt
Shijie Wu, Ozan Irsoy, Steven Lu, Vadim Dabravolski, Mark Dredze, Sebastian Gehrmann, Prabhanjan Kambadur, David Rosenberg, and Gideon Mann. 2023a.
Bloomberggpt: A large language model for finance.
arXiv preprint arXiv:2303.17564.

[Wu et al.(2023b)Wu, He, Liu, Sun, Liu, Han, and Tang]1-5-chatgptReview
Tianyu Wu, Shizhu He, Jingping Liu, Siqi Sun, Kang Liu, Qing-Long Han, and Yang Tang. 2023b.
A brief overview of chatgpt: The history, status quo and potential future development.
IEEE/CAA Journal of Automatica Sinica, 10(5):1122–1136.

[Wu et al.(2023c)Wu, Hu, Bi, Qi, Ren, Xie, and Song]Wu2023RRARetrieveRewriteAnswerAK
Yike Wu, Nan Hu, Sheng Bi, Guilin Qi, J. Ren, Anhuan Xie, and Wei Song. 2023c.
https://api.semanticscholar.org/CorpusID:262054223 Retrieve-rewrite-answer: A kg-to-text enhanced llms framework for knowledge graph question answering.
ArXiv, abs/2309.11206.

[Wu et al.(2023d)Wu, Hu, Qi, Bi, Ren, Xie, and Song]1-15-kgretrieve
Yike Wu, Nan Hu, Guilin Qi, Sheng Bi, Jie Ren, Anhuan Xie, and Wei Song. 2023d.
Retrieve-rewrite-answer: A kg-to-text enhanced llms framework for knowledge graph question answering.
arXiv preprint arXiv:2309.11206.

[Xiao et al.(2023)Xiao, Liu, Zhang, and Muennighoff]4-3-bge
Shitao Xiao, Zheng Liu, Peitian Zhang, and Niklas Muennighoff. 2023.
https://arxiv.org/abs/2309.07597 C-pack: Packaged resources to advance general chinese embedding.
Preprint, arXiv:2309.07597.

[Xie et al.(2023)Xie, Han, Zhang, Lai, Peng, Lopez-Lira, and Huang]2-fin-pixiu
Qianqian Xie, Weiguang Han, Xiao Zhang, Yanzhao Lai, Min Peng, Alejandro Lopez-Lira, and Jimin Huang. 2023.
https://arxiv.org/abs/2306.05443 Pixiu: A large language model, instruction data and evaluation benchmark for finance.
Preprint, arXiv:2306.05443.

[Yang et al.(2024a)Yang, Zhang, Kuang, Xie, Huang, and Ananiadou]1-7-mentalllama
Kailai Yang, Tianlin Zhang, Ziyan Kuang, Qianqian Xie, Jimin Huang, and Sophia Ananiadou. 2024a.
Mentallama: interpretable mental health analysis on social media with large language models.
pages 4489–4500.

[Yang et al.(2024b)Yang, Zhao, Zhu, Zhou, Xu, Jia, and Zan]2-med-zhongjing
Songhua Yang, Hanjie Zhao, Senbin Zhu, Guangyu Zhou, Hongfei Xu, Yuxiang Jia, and Hongying Zan. 2024b.
Zhongjing: Enhancing the chinese medical capabilities of large language model through expert feedback and real-world multi-turn dialogue.

[Yue et al.(2023)Yue, Chen, Wang, Li, Shen, Liu, Zhou, Xiao, Yun, Huang, and Wei]2-disclawllm
Shengbin Yue, Wei Chen, Siyuan Wang, Bingxuan Li, Chenchen Shen, Shujun Liu, Yuxuan Zhou, Yao Xiao, Song Yun, Xuanjing Huang, and Zhongyu Wei. 2023.
https://arxiv.org/abs/2309.11325 Disc-lawllm: Fine-tuning large language models for intelligent legal services.
Preprint, arXiv:2309.11325.

[Zhai et al.(2023)Zhai, Tong, Li, Cai, Qu, Lee, and Ma]2-forget-investigating
Yuexiang Zhai, Shengbang Tong, Xiao Li, Mu Cai, Qing Qu, Yong Jae Lee, and Yi Ma. 2023.
Investigating the catastrophic forgetting in multimodal large language models.
arXiv preprint arXiv:2309.10313.

[Zhang et al.(2023)Zhang, Tian, Yang, Chen, Li, and Petzold]1-6-alpacare
Xinlu Zhang, Chenxin Tian, Xianjun Yang, Lichang Chen, Zekun Li, and Linda Ruth Petzold. 2023.
Alpacare: Instruction-tuned large language models for medical application.
arXiv preprint arXiv:2310.14558.















§ APPENDIX





§ KNOWLEDGE GRAPH LAYER CONSTRUCTION


This section provides the detailed implantations of the four functions for constructing the knowledge graph layer in Section <ref>.

[boxrule=0pt, frame empty]
    f_par(p): Determine the factual claims from a given paragraph {p}. Put these facts into short phrases with basic grammar. Remember that every statement should have a distinct meaning, and pronouns should be avoided.

[boxrule=0pt, frame empty]
    f_ent(f_par(p)): Extract the noun entities in phrases from the given sentences f_par(p). The entities should not contain any comma, and each entity should be unique during extraction.

[boxrule=0pt, frame empty]
    f_rel(f_par(p)): Given the reference context f_par(p) and relevant entities, complete the relations between two entities. Notice that a triple should only contain one entity as head, one verb or verb phrase as relation, and one entity as tail. Separate the head, relation, and tail with a comma.

[boxrule=0pt, frame empty]
    f_dis(f_ent(), f_rel()): You are given several triples f_rel() with their entities f_ent(). These triples consist of subject-predicate-object elements, separated with a comma, but may contain ambiguities or inaccuracies. Your task is to refine and disambiguate these triples to ensure that they accurately reflect the entities and relationships described in their source texts without duplication or omissions. If the relationships have the same semantic meaning, rewrite the triples with the same relation. If the triple has already been mentioned with the same meaning as previous triples, delete it.






§ ACADCHALL AND R-FLUE-FIQA BENCHMARKS




Table <ref> lists the data statistics of two benchmarks. We use Vicuna-13B <cit.> to build the knowledge pyramid for two datasets according to the methods illustrated in Section <ref>. Specifically, the AcadChall dataset is created by crawling public information from the official websites of various departments at XXX University. It encompasses comprehensive information about 1,319 staff members, 2,061 courses, 31 departments, classrooms, and facilities, with over 5k corpus entries. Taking the staff member category as an example, it includes not only personal information, contact details, positions held at the university, and department affiliations but also specific academic knowledge, such as staff research interests, alma maters, degrees, research experience, and published papers, which is hard to be captured by general LLMs. 
In addition, we manually create 512 multi-choice QA pairs. 
Each question corresponds to eight options, and the answer could be one or more options. 
For example, for the question Which staff member in the Art department graduated from Cambridge University?, there could be multiple staff in the provided options that meet the information need.
For the R-FLUE-FiQA dataset, we adopt the test set from the FLUE benchmark <cit.>, which consists of 1,705 open-ended long-form QA pairs. 

















  
  

  
  

























§ DISCUSSION ON KNOWLEDGE COMPLETION AND CONDENSATION





Table <ref> verifies the effectiveness of our proposed knowledge completion and condensation techniques over multiple LLM backends. 
Specifically, the multi-level querying on the knowledge pyramid without completion and condensation techniques is used as the baseline. 
In the baseline, around 30% of questions are answered by more structured data (i.e., Ontology or KG). After applying the completion and condensation techniques, the percentage increases to 64%. 
For example, the relation has published in, which frequently appears in the KG layer, has been used to complete the Ontology layer. It has resulted in those questions related to publications being answered by the Ontology layer. 
Similarly, with knowledge condensation, an increase of around 27% of questions are answered by the KG layer. For example, as has published research in, has published research articles on, has published a paper in, has work published in has been condensed to has published research in  or has published research articles in, the number of tokens that sent to LLM for QA is decreased. 
Having a compact prompt has eased a smaller LLM backend, such as Vicuna-7B, in understanding the context, and we found that it could lead to a better question-answering result.
Overall, with the completion/condensation technique, precision is consistently improved over the baseline across different LLM backends. 
With the collaboration of completion and condensation techniques, the performance boost is more significant. 

















    

    
    

    
    
    























§ INFLUENCE OF KNOWLEDGE LAYERS OVER BACKBONES



Table <ref> lists the impact of different knowledge layers on more LLM backbones than the results presented in Section <ref>. 
The proposed knowledge pyramid significantly improves the precision of the raw text layer across different backbones. With more powerful backbones, such as Genmini-Pro, GPT-3.5, and GPT-4, the performance gains are larger (i.e., over 0.3). 
Besides, higher precision could be obtained when combining the Ontology layer with the KG or raw texts layer. 
A similar observation happens when combing the KG layer with raw text or the Ontology layer across different backbones. These are consistent with the observations in Section <ref>. 































 Paper Title: A Performance Study of LLM-Generated Code on Leetcode.






















breaklines
bgcolor=LightGray









cdfCDFCumulative Distributed Functions
llmLLMLarge Language Model










formal
  
    
    width 2pt
    width 4pt
    
  -
  
  7pt
  




  


resultcounter
result
	resultcounter
	




2024
2024
28th International Conference on Evaluation and Assessment in Software Engineering (EASE 2024), June 18–21, 2024, Salerno, Italy
10.1145/3661167.3661221
979-8-4007-1701-7/24/06

none







    Univ. Lille, CNRS, Inria
    France

 0009-0003-2525-3637
tristan.coignion@inria.fr



    Univ. Lille, CNRS, Inria
    France

0000-0003-3203-6107
clement.quinton@inria.fr



    Univ. Lille, CNRS, Inria
    France

0000-0003-1771-8791
romain.rouvoy@inria.fr


§ ABSTRACT 
This study evaluates the efficiency of code generation by llm and measures their performance against human-crafted solutions using a dataset from Leetcode.
We compare 18 llm, considering factors such as model temperature and success rate, and their impact on code performance. 
This research introduces a novel method for measuring and comparing the speed of llm-generated code, revealing that llm produce code with comparable performance, irrespective of the adopted llm.
We also find that llm are capable of generating code that is, on average, more efficient than the code written by humans. 
The paper further discusses the use of Leetcode as a benchmarking dataset, the limitations imposed by potential data contamination, and the platform's measurement reliability. 
We believe that our findings contribute to a better understanding of llm capabilities in code generation and set the stage for future optimizations in the field.





A Performance Study of llm-Generated Code on Leetcode
    Romain Rouvoy
    
1Fusion@ÖAW, Institute for Theoretical Physics - Computational Physics, Graz University of Technology, 8010 Graz, Austria

2 Max-Planck-Institut für Plasmaphysik, 17489 Greifswald, Germany

3Institute of Plasma Physics, National Science Center “Kharkov Institute of Physics and Technology”, 61108 Kharkov, Ukraine

===============================================================================================================================================================================================================================================================================================================================





§ INTRODUCTION

llm have recently increased in popularity, especially with the advent of ChatGPT <cit.>.
While llm have been spreading over various application domains, such as text or image generation, certain types of llm are being developed solely for code-related purposes.
These llm aim to assist the developers by saving time and effort through the generation of code, documentation, unit tests, etc.
Many of these llm only come in a “raw" form, that is, they do not integrate themselves into the developer's coding process.
These include models, such as CodeGen <cit.>, 
StarCoder <cit.>, 
WizardCoder <cit.>, 
CodeT5 <cit.>, and Incoder <cit.>.
On the other hand, some llm are already seamlessly integrated into the developer's IDE as code assistants, like GitHub Copilot,[<https://github.com/features/copilot>] Amazon CodeWhisperer,[<https://aws.amazon.com/fr/codewhisperer/>] and Tabnine.[<https://www.tabnine.com/>]

There has been a significant amount of work dedicated to comprehending how these llm perform in various situations and defining their limits. 
For instance, several works address the security of the code generated by such models <cit.> or the prevalence of bugs in the generations <cit.>. 
Many researchers are also investigating how developers interact with llm and how such models fit into the programming workflow <cit.> 
There is also a broad research effort to measure the actual efficiency of these llm by creating common benchmarks for comparison <cit.> or by actually measuring different qualities related to the generations, such as the success rate <cit.> or the robustness of the model regarding variations <cit.>.

To the best of our knowledge, there is no research work evaluating the performance of the code generated by llm.
Yet, having code that runs faster is an often sought-after characteristic of a program. 
Indeed, programming efficiency is paramount, especially when resources are scarce or programs are deployed on a large scale. 
In today's context where the energy consumption of software systems has become a major concern,
improving software efficiency is particularly relevant since increasing the performance of a program can also lead to energy consumption reduction <cit.>.

The process of code optimization is lengthy and intricate, requiring careful attention and a certain level of expertise, especially when searching for the best-performing algorithm, selecting the most appropriate data structure, or struggling with memory hierarchy.
Yet, this process is necessary to identify opportunities for improvement that may result in minor reductions in execution time.
llm can be used as a way to make this process easier, by generating performance-improving code edits <cit.>. 
Garg  <cit.> also presented RAPGen, a method for generating zero-shot prompts to enhance performance.
However, while users seem to put a lot of trust in code generated by llm, they still have trouble reviewing it <cit.>, which can lead to slow code being shipped in production, especially if an llm generates inefficient code.

The key contributions of this paper are as follow :
(i) We study the performance of the code generated by 18 llm on 204 problems and investigate performance differences across models using a novel method for measuring and comparing the performance of llm-generated code.
(ii) We compare the performance of the code generated by llm to the code written by humans.
(iii) Incidentally, we evaluate the usability of Leetcode,[<https://leetcode.com/>] a public repository of algorithmic problems that we use as a dataset.




From <ref> to <ref>, we describe the tasks dataset and the models we selected, outline our experiment setup, and explain the methodology we followed to analyze the obtained results, respectively.
We report in <ref> on the results of our evaluation and provide a critical discussion in <ref>. 
Finally, <ref> presents the related works, and <ref> concludes the paper.



§ METHODOLOGY





 §.§ Research questions

This paper covers the performance of code generated by various llm.
In particular, we aim to answer the following research questions:

RQ1: Can Leetcode be used as a dataset and a benchmark platform for evaluating llm?
Leetcode can serve as both a dataset of problems and a tool to evaluate and measure solutions to the problems.
Particularly, we study if the dataset is subject to recitation and if the measures Leetcode provides are reliable.

RQ2: Are there notable differences between the performance of the code generated by different llm?
llm differ greatly in terms of generating correct code, so we want to know if they also differ in terms of generating efficient code.

RQ3: Is there an effect of the success rate and the temperature of the llm on the code's performance?
Having a higher temperature decreases the capacity of the llm to generate valid code, so we aim to study if this also applies to the performance of the code. 
In the same way, we also study if an llm that is very good at generating valid code on one problem, is going to generate efficient solutions to this problem.

RQ4: How efficient are the solutions generated by the llm compared to human solutions?
Comparing the llm to a set of human-authored solutions can provide insights into their position relative to humans in terms of code performance.




 §.§ Tasks & Dataset


Task selection.
The input tasks—i.e., problems specified by a prompt—we consider to generate code from various llm has to meet the following requirements:

    
  * A given problem, should offer multiple candidate solutions, whose generated code performs differently.
    This ensures one can observe differences across the various llm;
    
  * Generated solutions should exhibit variable execution times. 
    Given more complex inputs, one should differentiate 𝒪(n) from 𝒪(2n) and 𝒪(n^2) algorithms.

As a result, task datasets, such as HumanEval or Mostly Basic Python Programming (MBPP), which are classically used when evaluating llm for code assessments <cit.>, cannot be considered for our purpose.
Indeed, while they do provide unit tests to drive the generations, the size of the inputs in these unit tests remains small and fails to scale to appreciate performance issues.
Moreover, the solutions that need to be generated are often very short, which would lead to fewer possible variations between implementations.
Also, the fact that many problems are not algorithmic by nature makes them less prone to inefficient practices and performance variation.
To face such issues, we used input prompts that were built from Leetcode problems.
Leetcode is an online judge platform that suggests programming problems to registered users.
It addresses the above limitations as it provides algorithmic problems with varying levels of difficulty and test cases with large input sizes.
Leetcode also exposes a GraphQL API[<https://pypi.org/project/python-leetcode/>] to fetch relevant metadata on the problems, such as exercise instructions, code snippets containing the signature of the function to generate, as well as the difficulty and topics of the problem.

We followed an experimental design similar to the one used by Döderlein  <cit.>, while using a different set of Leetcode questions. 
To avoid data contamination, which happens when an llm is tested on data it was trained on,[You can find more details on data contamination https://web.archive.org/web/20231020160612/https://bdtechtalks.com/2023/07/17/llm-data-contamination/here.] we only considered problems that were published after January 1^st, 2023. 
As all the llm (except GitHub Copilot) we evaluated were trained using datasets older than these problems, we avoid any data contamination.
As these problems are published by Leetcode in the context of programming competitions, they are always original.
However, GitHub Copilot being an online closed-source tool, one cannot tell whether it underwent training with the problem set we employed. 
Our set was composed of 204 problems—labeled as 56 easy problems, 104 medium problems, and 44 hard problems, as classified by Leetcode upon the problem's publication.

To answer RQ1, we also performed our experiment a second time on the set of questions used by Döderlein  <cit.>, which is composed of 300 problems (95 easy, 105 medium, and 100 hard) from the most liked problems of Leetcode. 
We will refer to this dataset as the "old" dataset, and to our dataset of problems published during 2023 as the "new" dataset.
Code generation was performed in Python
for its ease of use and because of the prevalence of Python-written datasets for evaluating llm <cit.>.



Input prompts.
The instructions given by Leetcode for each programming problem contain (i) the description of the problem, (ii) examples of inputs and outputs, and (iii) constraints on the input data. 
To build the prompts inputted to the llm, we chose to only include the description of the problem.
This choice aimed to maintain the prompt's conciseness and ensure compliance with potential llm' length restrictions. 
Indeed, llm are constrained by a context size, imposing the maximum number of tokens they can process at once, encompassing both the prompt and the answer. 
In our study, the majority of llm impose a context window of 1024 tokens.
<ref> illustrates a descriptive prompt (enclosed within triple quotes) along with a solution generated using this prompt and the corresponding benchmark instructions. 
Leetcode occasionally includes an additional comment in the prompt, indicating the available methods of the elements passed to the solution (binary trees).
While adding examples in the prompt could potentially increase the likelihood of generating correct solutions <cit.>, we chose not to include them due to the complexity of representing Leetcode's data structures (such as arrays, graphs, linked lists, trees, etc.) in textual form.
On Leetcode's website, data structures are textually represented using an array notation format with brackets (“[1, 2, 3]"), which might be misleading for an llm working with data structures that differ from arrays or lists.

Canonical solutions.
Each problem was matched with a single valid solution, written by a human and fetched from various sources.
These solutions, referred to as “canonical solutions", are considered as baselines during the benchmarking process, although they may not represent the entirety of human-written solutions. 
They were used to assess the stability of the measuring process.
Most of the canonical solutions were fetched from the WalkCC repository of Leetcode solutions.[<https://github.com/walkccc/LeetCode>]
When one question did not come up with any solution in Python from this repository, we selected one solution proposed by the Leetcode community among the most upvoted ones (starting from the one with the most upvotes), which are also publicly available on the Leetcode website.
These modifications only included changes to the function name, variable names, and type hints except for one specific case, where a recursive solution provided by WalkCC was replaced by an iterative one from the Leetcode community because of the stack limit on our local setup.
We ended up with one Python-based canonical solution for each problem of our dataset.

Test cases.
Testing generated solutions is a twofold process, as both the correctness and scalability (performance) of each solution must be checked.
First, to ensure that generated solutions are correct, we provide such solutions to the Leetcode online judge system, which in turn validates them based on its test suites.
Second, we execute the valid solutions with input data to assess their performance.
To retrieve such input data, we crawled through the problems' instructions and extracted two to three examples of inputs and expected outputs provided by Leetcode.
However, such inputs were too small to exhibit significant performance differences, arrays containing only two to five elements.
To execute the generated solutions with larger input data, we took advantage of Leetcode's judge system that returns the inputs and expected output of the first failed test of a test suite.
We noticed that Leetcode tests if the submitted solution is inefficient by using a timeout system. 
Since all selected problems are algorithmic by nature, one simple way to put a heavier load on their implementations is to increase the size of the inputs.
For every problem, we thus submitted a modified version of a canonical solution that failed only when the size of the first parameter exceeded a certain threshold. 
We then set this threshold manually multiple times for every problem to extract three different inputs for each problem, resulting in more than 150 MB of fetched input data with most inputs having over 10^5 elements.



 §.§ llm Under Study

Our empirical study covers a total of 18 llm, specifically designed for coding purposes.
We selected the 18 popular code llm from Hugging Face,[<https://huggingface.co>] as well as GitHub Copilot, which is an online closed-source code assistant. The LLMs were selected based on the number of downloads and likes they exhibited.
We chose GitHub Copilot to offer a comparison between a commercial llm and open-source llm, but did not choose any other GPT models from OpenAI because of their cost.
<ref> summarizes all the llm considered in this study.
This includes variants of the same base llm—models with varying sizes (in billions of parameters) or different training data, such as variants of CodeGen, InCoder, and CodeT5.
llm belonging to the same family are models closely related in terms of training data and method. 
We first performed our experiment in March 2023 with a subset of the models presented here, with the "old" dataset. 
We then performed it a second time in September 2023 with all the models and the "new" dataset.







§ EXPERIMENT SETUP

This section describes our experiment setup to generate and validate the solutions produced by the llm. 
First, we describe how solutions were generated from each llm.
Then, we outline the three-step process used to filter invalid solutions. 
Finally, we explain how the run time of the generated solutions was measured.



 §.§ Code Generation

We generated 10 solutions for each problem from our dataset by varying the temperature of the llm used to generate them. 
Specifically, we considered 6 different temperatures (0.1, 0.2, 0.4, 0.6, 0.8, and 1.0).
As Copilot's temperature cannot be configured, we used its default temperature for generating all the problems.

Generating code with GitHub Copilot. 
Automatically generating with GitHub Copilot for a reproducible experiment proved to be a difficult task.
Firstly, the code suggestion feature of Copilot activates when typing in a text editor with the installed Copilot plugin. 
Additionally, GitHub Copilot produces code based on a context that encompasses the current file and the files previously accessed by the user, impacting the generated solutions.
Lastly, we noticed a caching mechanism on the GitHub Copilot server side, which resulted in very similar or identical solutions if we generated multiple solutions for a given problem in the same session.
To address these issues, we used a generation method similar to the one used by Döderlein  <cit.> by instrumenting the GitHub Copilot Neovim plugin[https://github.com/github/copilot.vim] and restarting the plugin between every generation to avoid the caching effect.
This method allowed us to automatically generate solutions in a quick and isolated fashion.
On top of that, GitHub Copilot provides 2 means of generation: inline generations (the suggested code is integrated with the editor) and panel generations (Copilot generates at most 10 completions and displays them on a panel next to the editor).
We chose to exclusively use inline generations, as panel generations yielded worse results in terms of functional correctness than inline generations.

Generating code with open-source models.
Regarding the open-source models, we generated solutions by deploying the models on servers provided by the Grid5000 platform <cit.>.
We used the Deepspeed library to make the generation process faster and fit larger models on our GPUs.
Concerning the sampling, we used the same methods as Chen  <cit.> and used nucleus-sampling <cit.> with top p=0.95. 
The maximum number of tokens to be generated was set to 600. 
This is because the llm we used have a limited context size of 1024 tokens (prompt included). 
Thus, to avoid exceeding the context size, we had to limit the number of tokens to generate.
We also verified it did not significantly impact the functional validity of the llm.

In total, we generated 2,040 solutions with Copilot and 12,240 with each of the eight other models, resulting in 210,120 generated solutions overall.



 §.§ Validation

Each generated solution was tested to ensure its functional correctness following a three-step process.
At every step, if a solution was found to be invalid, it was excluded from subsequent stages of the experiment.
The process was as follows:

i) Local validation. 
We filtered out code generations that included easy-to-spot errors, such as syntax errors or runtime errors, by using the small inputs we fetched earlier (see tasks - Test cases). 
While this step was not strictly necessary, it quickly reduced the number of solutions to be validated in the next step;

ii) Leetcode validation.
Next, we submitted the solutions to the Leetcode judge system using the Leetcode GraphQL API, where they underwent a rigorous test suite managed by Leetcode. 
We kept the solutions that passed all the test cases or exceeded Leetcode's allocated time limit. 
The latter was kept to ensure that correct solutions that were too slow remained included in the benchmark;

iii) Exclusion of timeouts and other errors.
Finally, we excluded the solutions that reported errors when executed with our benchmarking setup using the large inputs fetched beforehand.
We invalidated the code generations that took more than 10 seconds to run or raised an error. 
Most of the errors raised in this step were recursion errors caused by differences between Leetcode's Python interpreter and ours. 
Indeed, Leetcode's seemed to have a higher recursion limit than ours, which we set to 10,000 instead of the default 1,000 (we could not manage to set it any higher). 
Additional errors occurred because we developed our helper classes differently from Leetcode's implementation.
Although these classes are provided by Leetcode during the submission process, they are not publicly available.
Out of the 4,930 invalidated solutions that were caught in this step, 4,863 (98.6%) were due to timeouts, 20 (0.04%) to recursion errors, and 47 (0.1%) to other errors.
Following this validation process, the initial set of 210,120 generated solutions was pruned down to 7,481 (3.6%) valid solutions remaining across the 18 llm. 



 §.§ Measuring run time

We measured the performance as the run time of the generated solutions using pytest-benchmark,[https://github.com/ionelmc/pytest-benchmark] which runs pytest unit tests multiple times to obtain run times statistics.
The measurements were performed using parameters that ensured each solution ran at least 10 times and for at least 1 second in total.
We did not perform warm-up runs of the benchmarks, as we did not notice any significant difference in the measured time during preliminary testing. 
To facilitate the measurement protocol, the generated solutions were sorted into “runs", based on the specific llm and temperature that were used during the code generation.
Within each run, which was defined by a unique combination of model and temperature, the solutions were measured in sequence during a single program execution.
Furthermore, in every run, we added the canonical solutions we previously collected, which would run alongside the generated solutions.
This approach ensured that the same canonical solutions were executed in every run, allowing us to maintain measurement stability.
Specifically, we calculated the standard deviation of the canonical solution run times across all runs, thus providing a reliable measure of the variability of the measurement protocol.
We observed that over 96% (196 out of 204) of the canonical solutions had a standard deviation lower than 1/10^th of their average run time, which we deemed to be an acceptable level of variation.

The cluster we used to run the benchmark was the chiclet cluster of the Grid5000 testbed.[<http://grid5000.fr>]
It hosts 2 AMD EPYC 7301, with 16 cores per CPU and 128GB of memory.
When using the node, all the cores of both CPUs were reserved, but only one was used at a time to maximize the stability of the measurement protocol.



 §.§ Replication package

All the artifacts of this study, including our results, code, and datasets, are available in the following public repository: <https://zenodo.org/doi/10.5281/zenodo.7898304>.




§ DATA ANALYSIS

In this section, we describe the methods we adopted to analyze our results.
These methods fall into one of the two following categories: functional correctness and code performance.



 §.§ Functional Correctness

The functional correctness of an llm defines how much the llm outputs code conforming to the program contract (as specified by the input prompt).
To evaluate the functional correctness of our llm, we computed their pass@k metrics with k=1 and k=10, using the unbiased estimator proposed by Chen  <cit.>.
The pass@k unbiased estimator which, from k samples produced, considers the test as successful if one of these samples passes all the tests, is computed as follows (with n the total number of samples, c the number of correct samples and 𝔼 the expected value):



    pass@k := Problems𝔼[1- n-ck/nk]




As Chen  <cit.> suggest, we calculated the pass@k for each temperature when evaluating an llm's functional correctness and considered the best one as the pass@k for that llm.



 §.§ Code Performance

To measure the code performance, we considered three different metrics.
First, we used the memory usage reported by Leetcode. Then, we computed the median of the run times measured by pytest-benchmark for every generated solution.
Lastly, to compare the llm solutions to human-submitted solutions, we also used the rank reported by Leetcode when validating the solution. 
This rank is a number between 0 and 100 that indicates the share of submitted solutions that are slower than the current solution (if a solution has a rank of 90, it is faster than 90% of the submitted solutions on Leetcode). 

To assess the llm' performances, we conducted pairwise comparisons as follows: For each pair of llm, we identified problems where both models generated more than 5 valid solutions. 
For each identified problem, we conducted a Student t-test on the mean run time of the generations to determine if there was a significant difference. 
Then, for each pair A-B of llm, we computed the ratio of problems where A's code was significantly faster than B's and where B's code was significantly faster than A's.



§ RESULTS


In this section, we summarize the key observations from our experiment and answer our research questions. 
In <ref>, you can also find the functional validity results of the different llm on both of our Leetcode datasets.
Our results are also available in the form of a companion notebook in our replication package. 
The companion notebook offers more insight into the results and additional graphs.





 §.§ RQ1: Can Leetcode be used as a dataset and a benchmark platform for evaluating llm?




  §.§.§ Can Leetcode problems be adopted as a dataset for llm generation?





As on can observe in <ref>, the generated codes exhibit on a significant drop in functional correctness between the two datasets. 
The difference here is pretty staggering for every tested llm, reporting on a tenfold decrease in pass@k.
We believe this issue may stem from data contamination in the old dataset. 
Data contamination occurs when an llm is assessed on data that was included in the training dataset, introducing bias into the evaluation process.
In our case, a significant number of questions in the old dataset are widely known and have been extensively shared on GitHub. 
For instance, a search for the prompt of the "3sum" Leetcode problem on GitHub yields approximately 4.000 matches in public repositories.
These questions are also old enough to likely be included in the training datasets of the llm under study, as the majority of their training datasets have a cut-off date between 2021 and 2022. 
Due to this data contamination, llm tend to recite, reproducing verbatim source code when generating solutions.
This phenomenon is more pronounced when the prompt is highly specific and lacks contextual information, as seen in Leetcode prompts that closely match GitHub repositories.
The observed shift in functional validity between the two datasets could also arise from a genuine difference in the difficulty of the questions within each dataset. 
However, quantifying this last hypothesis proves to be challenging.





  §.§.§ Are Leetcode measurements reliable?


Run time.
As reported in <ref>, the coefficient of variation of the Leetcode measures (0.089) is slightly higher than the coefficient of variation of the local measures (0.035). 
This suggests that Leetcode's measuring setup is less suited to ensure accurate benchmarks.

We also study the correlation between our local measurements and Leetcode's, and we notice two issues: (1) the times measured locally and by Leetcode only slightly correlate on average (0.28). 
For some problems, the measures are highly correlated (> 0.8) while, for others, they are almost not (<0.2).
This is more apparent when we look at the scatter plot showing the measures of some problems in <ref>. 
In this problem, there are four clusters of generations with a different locally measured time, but the clusters are indiscernible in terms of Leetcode time. 
This could be due to two main reasons. 
Firstly, as previously discussed, the variance of the measures from Leetcode is higher and as such, there is much more noise in the measures, decreasing the precision. 
Secondly, the tests we employed may be more focused on performance testing than Leetcode's. 
Notably, our test suite comprises only three tests featuring significantly large inputs, potentially accounting for certain disparities in the results.

Although still usable, relying on the time reported by Leetcode introduces some limitations due to its higher variance. 
Consequently, Leetcode's measures cannot allow us to discern the differences in run time between different code implementations as precisely as locally measured time.





Memory usage.
While we did not measure the memory ourselves, we observed the variation of the memory usage measure provided by Leetcode. We notice that the memory usage for the same solutions decreases over time. There is indeed a slight correlation of -0.24 between the day of the year we tested our solution and the memory usage.
The fact that the memory usage measure evolves renders comparisons between LLMs harder. While we could theoretically offset the memory usage when we detect changes over time, it would require testing the canonical solutions alongside the generated one on Leetcode.

Leetcode rank.
The time ranking that Leetcode returns when we test a solution represents the share of submitted and valid solutions that are slower than ours. 
While this could be a great tool to rank llm among human-submitted solutions, we find that the ranking is heavily affected by our submissions and time. 
As you can see in <ref>, the overall rank of the llm we tested decreases over time. 
To verify this, we tested GitHub Copilot twice, once as the first llm, and a second time after testing all the llm.
The first test of Copilot has an overall rank of 77, and the second test ranks down to 54, despite it being tested with the same solutions.
This effect of the rank evolving becomes obvious when you consider that the rank is determined using all the previously accepted solutions, including ours.
This means that by testing thousands of our solutions on Leetcode, we are actively changing the ranks of future tests.



RQ1: The evaluation of llm using Leetcode questions as a dataset presents some challenges.
Although Leetcode's questions could serve as a valuable dataset akin to HumanEval, limitations arise due to the constraint that only the problems published after the llm's training dataset formation are usable for evaluating the llm in question.
This creates potential difficulties in reproducibility, particularly as new llm emerge, especially if they do not exclude Leetcode problems from their training datasets <cit.>. 
Additionally, while Leetcode's provided metrics, such as run time, memory usage, and rank may offer practicality in various scenarios, their usability and reliability are questioned when compared to more traditional measurement methods. 
The presence of these challenges emphasizes the need for careful consideration and scrutiny when adopting Leetcode to evaluate llm.








 §.§ RQ2: Are there notable differences in performances between llm?


The pairwise comparison depicted in <ref> reveals subtle distinctions in the performances of various llm.
Notably, some models, such as StarCoder and the CodeLlama model with 13B parameters specialized in Python, consistently exhibit slightly superior results compared to others. 
Despite these observed variations, the mean Cohen's d effect size measures a mere 0.024, a statistically insignificant magnitude. 
This suggests that the practical impact of these differences on the mean speed of code generation is remarkably small.
For instance, when comparing CodeLLama-13-instruct and CodeGen25-7B-mono, CodeLLama outperforms the latter in a statistically significant manner in 3 problems out of 8. However, it is crucial to note that the mean performance difference between these models is a mere 0.02 standard deviation.
It thus seems that improving an llm in terms of functional validity does not significantly impact the performance of the code it generates. 
This may be due to different factors, such as the fact that most llm share the same datasets or that they are trained to produce valid code and not fast code.
Improving the performance of an llm could be done by curating a training dataset of only efficient code and fine-tuning one of the foundational models we used, or by using reinforcement learning to "teach" the model to produce better code. 
Madaan produced an llm that could improve the performance of code <cit.>. 
This llm could be leveraged in a generation pipeline to directly improve the generated code. 



RQ2: Our analysis uncovers statistical differences in the performance of generated code among different llm.
However, the effect size, as measured by Cohen's d, is so negligible that it raises questions about the practical significance of these differences. 
Despite some models consistently outperforming others, the overall impact on the mean efficiency of llm-generated code appears to be minimal.










 §.§ RQ3: Is there an effect of the functional validity of the llm and its temperature on the generated code's performance?


Functional validity. 
When calculating for every problem the correlation between the success rate of the llm that generated the solution and the run time of the solution, we find that there is only a very slight negative correlation (-0.08) between the success rate and the performance.
There is close to no correlation (-0.11) observed between the success rate of the model and the variation in the performance of the generated code.

Temperature. 
There is no correlation (0.05) observed between the temperature of the generations and the performance of the generated code. 
This means that the temperature does not affect how fast the solutions are.
However, we observe that temperature is moderately correlated (0.41) with higher variations in performances. 
This means that higher temperatures tend to increase the variation in performance across generations. 
The complete distribution of the correlation for each of the 24 problems can be seen in <ref>. 
So, while increasing the temperature leads to a lower success rate <cit.>, it can help find a faster solution with an extended exploration of generations.
The fact that the temperature increases the variation in performances comforts the idea that higher temperatures lead to more diverse outcomes.



RQ3: Our analysis of llm indicates that the quality of generated code does not have a substantial impact on its performance. 
However, we observed that modifying temperature settings within an llm significantly affects the diversity of code performances produced. 
This implies that, while code validity may not be a decisive factor in performance, adjusting temperature settings can be a valuable strategy to enhance the variety of outcomes in code generation processes.








 §.§ RQ4: How fast are LLMs compared to humans ?

As previously stated, the Leetcode time ranking evolves, so we chose to compare the second model we tested on Leetcode with humans (Copilot being the first, it did not have enough generations overall because of its lack of a temperature setting). 
The results of this comparison are depicted in <ref>.
The comparison is done using the Leetcode ranking, with the assumption that most of the previous submissions were made by humans.

We observe in <ref> that the solutions generated from llm are faster than most previous submissions with a mean rank of 73%, and that it even generated some solutions that were faster than 95% of the previous submissions.



RQ4: It seems that the llm are faster than most of the human solutions on Leetcode, on average. 
If the llm we tested were in an actual competition, his valid solutions would be on average faster than 73% of the other solutions on Leetcode.





§ DISCUSSION

In this section, we discuss the results reported in the previous section, their implications, and the limitations of our study.



 §.§ Discussion of the results

On the Leetcode measures and usability.
The data contamination issue we unveiled poses a significant challenge in the evaluation of llm, as it prevents an accurate evaluation of their real performances. 
Because Leetcode's problems are not filtered from the training datasets, as research on llm continues, even the newer problems might contaminate future training datasets, thus rendering reproduction of our study harder <cit.>. 
This conclusion also holds for any study using Leetcode as an evaluation dataset, such as <cit.>. 
However, we believe that the methodologies employed and the conclusions drawn would likely hold validity with alternative sets of questions from Leetcode or other performance-oriented datasets. 
This suggests that future studies seeking to replicate our findings would primarily need to change the dataset employed for assessing llm.
Addressing the data contamination concern could involve leveraging pre-filtered evaluation datasets, such as HumanEval, already separated from the training processes, and repurposing them into performance evaluation datasets.




On functional correctness.
The ranking of the functional correctness of the llm is consistent with the previous evaluations of the llm on HumanEval <cit.>. 
However, it seems that InCoder performs worse than presented in its introductory paper <cit.>, which may be due to our experimental protocol—using it only for left-to-right generation instead of infilling like it was built for.  

We observe that Leetcode's problems seem harder for the llm to solve than HumanEval's problems. 
Indeed, StarCoder, the model that performed the best with a pass@1 of 0.09, had a pass@1 of 0.408 on HumanEval <cit.>. 
We believe this is due to multiple factors. 
First, our Leetcode prompts and expected solutions are longer than in HumanEval (the average length of solution in HumanEval is 180 characters vs 425 characters in our dataset), thus increasing the chance of a generation to fail. 
Indeed, having to generate more code can lead to a higher chance of making a mistake, as shown by a correlation of -0.30 between the solution's length and the success rate of the problem.
Second, Leetcode problems come from programming competitions and need a lot of thinking to be solved. 
The causal generation of the llm does not allow a "thinking" process to happen, which for harder problems causes a drop in functional validity. 
This could be solved by making the llm mimic human thinking with methods, such as Chain-of-Thought prompting <cit.>.


On the performance of generated code. 
To the best of our knowledge, our methodology for evaluating llm based on the performance of generated code is novel and could serve as a benchmarking approach for future studies involving new llm and datasets.
While we aimed for a singular performance score for llm, similar to pass@k, the varying rates at which llm generate valid solutions posed a challenge, leading us to employ pairwise comparisons.
An improvement to our method could involve using an optimal solution as a reference point and comparing the speed of each generated solution to the optimal solution's speed. 
Speed would thus be expressed as a factor relative to the optimal time, addressing the issue of problems having different time scales and potentially laying the groundwork for a more comprehensive "performance score."

The low success rate of llm on Leetcode posed a considerable challenge for their comparison. 
Among the 204 problems, only 24 had (i) valid solutions from at least 10 different models (out of the 18 assessed llm) and (ii) at least 10 valid solutions in total (see to the companion notebook). 
This low success rate complicated pairwise comparisons, as numerous models could not be compared due to an insufficient number of common problems with a substantial number of generated solutions.

Our discoveries offer valuable insights for developers in their choice of llm. 
For example, when developers are considering an LLM for tasks like code generation, such as with GitHub Copilot, the performance of the generated code may be an important concern.
With our findings, developers can be assured that there is no significant variance in the performance of code generated by different llm.
This means that if they aim to have fast code, 
there's no necessity to consistently opt for the largest model available; instead, a smaller one suffices.
We also hope that our findings will incentivize further research on building new llm that produce even more efficient code.





 §.§ Limits and Threats to Validity

Regarding functional correctness, although we did our best to generate code in the most optimal conditions, some changes to the input prompt (adding examples and constraints), or the configuration of the models may have changed the performance of the llm.
However, we believe that this should not significantly impact the validity of our experiment, as all the models were configured similarly.

GitHub Copilot being a closed-source tool, it might be retrained without the community's knowledge, which could potentially lead to a modification in the tool's performance in upcoming experiments.

The majority of our validation and benchmarking process relied on Leetcode's test suite and online judge system. 
Thus, it is possible that some big test cases we extracted for the benchmarking favored some types of implementations over others. 
We mitigated this issue by systematically fetching three test cases and by having a large dataset of problems to generate from.
During our experiments, we also tried to generate plausible inputs using random generators, but this method did not yield satisfying results because randomly generating data structures with specific shapes, or properties (generating valid regular expressions) is difficult to achieve. 

One gap in our study is that we do not consider memory usage at all when studying the llm. 
This is because our benchmarking setup did not allow for memory monitoring and doing otherwise would have cost us a lot of time. 
Moreover, we believe our results obtained with only the run time to be self-sufficient.

The way we compared an llm to a human using Leetcode rankings gives a good idea of where the llm stands in terms of performance. 
However, because the ranking evolves and we have no information about the population the llm is ranked against, the results here should be taken with a grain of salt.

It is also important to note that Leetcode as an evaluation dataset suffers from some issues. 
As we only evaluate the llm on algorithmic problems, the performances of the llm are hard to generalize across all programming fields. 
However, this is difficult to improve on for similar reasons to HumanEval: we only have a limited context size and have to make the llm generate in one go a completion to some code, which must be self-contained (meaning, all the information needed to generate the solution must be in the prompt). 
Also, in terms of performance, it is difficult to find another kind of self-contained code than algorithmic problems that have, such variations in performance. 
While studying the llm on SQL generation could be a good idea, it would not fit into our studies with generalist programming languages.

Regarding Leetcode as a platform, we are heavily dependent on it for validating the generated solutions and are limited by its daily rate limits of 1,000 submissions per account. 
For future studies, it would be great to consider alternatives to Leetcode, such as the project CodeNet <cit.>, which does not have as many restrictions as Leetcode.



§ RELATED WORKS

Previous research has investigated various aspects of llm for code-related tasks, including the security of their suggestions <cit.>, the prevalence of bugs in the generated code <cit.>, how developers interact with them  <cit.> or just the quality and correctness of the code they generate <cit.>. 
There have also been efforts to measure the efficiency of llm through the creation of benchmarks for comparing them, such as HumanEval<cit.>, MBPP <cit.>,  CoderEval <cit.>, APPS <cit.>, CodeXGLUE <cit.> or ReCode <cit.>. 
Xu  <cit.> also led a comparative evaluation of multiple llm for code including Codex and Codeparrot.

Leetcode, while being just a coding competition platform, is also used as a dataset to evaluate the capabilities of llm on programming tasks. 
Döderlein  <cit.> measured the performances of Copilot and Codex on Leetcode and the effects of changing the prompts. 
Nguyen and Nadi <cit.> studied Github Copilot's code suggestions on Leetcode problems and the complexities of its generated code. 
Vasconcelos  <cit.> studied the effects of highlighting the uncertainty of AI-powered code completions using Leetcode problems and Codex.

Various other methods have also been employed to investigate the impact of temperature on the generated code, apart from the approach we proposed: Chen  <cit.> evaluated the best temperature of Codex in terms of pass@k. 
Austin  <cit.> also studied the effects of the temperature on the performance of their model.
Christopoulou  <cit.> also studied the effects of the temperature and nucleus-sampling on their llm.
Döderlein  <cit.> highlight the importance of correctly tuning the temperature of a model when using it to generate code. 
The research led by Aghakhani  <cit.> shows that poisoned models suggest insecure code more often as the temperature increases.
Our results demonstrate that increasing the temperature also increases the chance of generating slow and inefficient code.

On the subject of performance, Madaan  <cit.> fine-tuned llm to make them improve the performance of code.
Multiple other techniques have been proposed for automatically improving the performance of code using llm <cit.>. 
Our contribution is the first—as far as we know—to investigate the differences in the performance of llm-generated code. 
Future llm that would be adapted with these performance-improving techniques could also be compared using our methodology.
Regarding recitation, few works have been done on this subject, but Yan  <cit.> proposed a method to detect cases of recitations in llm using inference fingerprinting. 
Jacovi  <cit.> explained why data contamination in llm was problematic and proposed methods to mitigate it.

To summarize, our paper evaluates the performance of code generated by various llm and investigates differences in the performance of the generated code across models on Leetcode problems, which, to the best of our knowledge, has not been done previously.



§ CONCLUSION

In this study, we presented a comprehensive analysis of the performance of code generated by various llm using a novel methodology that measures and compares the runtime speed of solutions to algorithmic problems. 
Our findings suggest that the performance of the generated code is largely similar across different models, regardless of their size or training data. 
Furthermore, increasing the temperature parameter during code generation leads to a greater variance in performance, though not necessarily to better or worse solutions on average.
We also critically evaluated the suitability of Leetcode as a dataset and benchmark platform for assessing llm. 
The results indicate that, while Leetcode's problems are suitable for performance evaluation, their measures should be used cautiously due to issues with stability and reliability. 
Additionally, we observed that the use of newer Leetcode problems is essential to avoid data contamination and ensure the validity of llm evaluations.


This work opens up several avenues for future research, including the development of performance-oriented training datasets and the fine-tuning of llm for performance improvement. 
As the field of AI-assisted programming continues to evolve, studies such as ours will play a critical role in understanding and enhancing the capabilities of llm. 






This work received support from the French government through the Agence Nationale de la Recherche (ANR) under the France 2030 program, including partial funding from the CARECloud (ANR-23-PECL-0003), DISTILLER (ANR-21-CE25-0022), and KOALA (ANR-19-CE25-0003-01) projects.
Experiments presented in this paper were carried out using the Grid'5000 testbed, supported by a scientific interest group hosted by Inria and including CNRS, RENATER and several Universities as well as other organizations.[See <https://www.grid5000.fr>]


ACM-Reference-Format





 Paper Title: GABInsight: Exploring Gender-Activity Binding Bias in Vision-Language Models.










2676 















A]Ali Abdollahi
[Equal contribution.]

A]Mahdi 	Ghaznavi

A,D]Mohammad Reza Karimi Nejad
 
A]Arash Mari Oriyad
A]Reza Abbasi
A]Ali Salesi
B,C]Melika Behjati
A]Mohammad Hossein Rohban


A]Mahdieh Soleymani Baghshah




[A]Sharif University of Technology
[B]ecole polytechnique fédérale de lausanne (EPFL)
[C]IDIAP research institute
[D]Shamsipour Technical and Vocational College



§ ABSTRACT 
Vision-language models (VLMs) are intensively used in many downstream tasks, including those requiring assessments of individuals appearing in the images. While VLMs perform well in simple single-person scenarios, in real-world applications, we often face complex situations in which there are persons of different genders doing different activities. We show that in such cases, VLMs are biased towards identifying the individual with the expected gender (according to ingrained gender stereotypes in the model or other forms of sample selection bias) as the performer of the activity. We refer to this bias in associating an activity with the gender of its actual performer in an image or text as the Gender-Activity Binding (GAB) bias and analyze how this bias is internalized in VLMs. To assess this bias, we have introduced the GAB dataset with approximately 5500 AI-generated images that represent a variety of activities, addressing the scarcity of real-world images for some scenarios. To have extensive quality control, the generated images are evaluated for their diversity, quality, and realism. We have tested 12 renowned pre-trained VLMs on this dataset in the context of text-to-image and image-to-text retrieval to measure the effect of this bias on their predictions. Additionally, we have carried out supplementary experiments to quantify the bias in VLMs’ text encoders and to evaluate VLMs’ capability to recognize activities. Our experiments indicate that VLMs experience an average performance decline of about 13.2% when confronted with gender-activity binding bias.













§ INTRODUCTION



Nowadays large multi-modal models have shown tremendous potential in various tasks, from information retrieval systems to image captioning, visual question answering, image generation, and visual reasoning.
As a branch of multi-modal models, Vision-Language Models (VLMs) <cit.>
that provide a shared cross-modal embedding space between text and image modalities, 
have been intensively investigated by researchers recently and are used in many real-world applications <cit.>. 








Despite their wide range of success in downstream tasks, VLMs are subject to many critical biases, which can consequently affect their performance, especially in sensitive applications. For example, <cit.> provides a comprehensive analysis of biases and their implications in the CLIP Models, revealing how this model is exposed to gender biases and how challenging it is to trust it in real-world applications.
As <cit.> and <cit.> state, the bias is introduced in visual-linguistic pre-training due to unfairness in the training data, and additionally, at inference time, the visual and linguistic contexts that is used for few-shot applications can also promote bias. 





One important type of bias, which is highly discussed in the literature, is the gender bias. Gender bias demonstrates the undesirable association of a factor with a gender according to the model. For instance, due to the high occurrence of data in which men are repairing devices, this activity is usually associated with men by deep models, which leads to failure when faced with women repairing devices.
 <cit.> suggests that different VLMs do not perform equally well on determining gender for an occupation, and do not assign equal retrieval likelihood to images of male and female professionals. Also, <cit.> and <cit.> show that CLIP <cit.> shows biased behavior in retrieval based on gender-neutral queries. Please note that, in line with common practice in the literature, our focus is on two genders, masculine and feminine. This approach streamlines the process of generating the dataset and analyzing the experiments.



In this research, we delve into the issue of gender-activity binding in retrieval tasks. We aim to retrieve the corresponding caption or image from two given captions or images, each depicting an activity performed by a different gender, based on a provided image or caption that shows one of the genders performing the activity. We refer to this bias in associating gender and activity as the Gender-Activity Binding (GAB) bias. This bias arises due to ingrained gender stereotypes in the model or other types of sample selection biases <cit.>. Our research indicates that VLMs do not display a substantial bias in retrieval tasks when both text and image modalities depict only one gender. This is because the identification of the performer’s gender can directly lead to the correct output, eliminating the need to bind the performed activity with the performer’s gender. Though, text encoders have shown a considerable bias towards the expected gender. However, The bias becomes more pronounced when the scene is more complex, i.e., when two individuals of different genders are present in the image. We observe a drop in retrieval accuracy in scenarios where the activity performer is the unexpected gender. 
For example, in the context of the “repairing” activity, the retrieval accuracy of VLMs in identifying the performer decreases when a woman is repairing a device and a man is also present in the scene. This is in comparison with scenarios where the performer is a man, or there is no man present in the scene to associate the activity with him.

The significance of this bias escalates when we consider its underlying implications. Not only does it perpetuate societal biases and fairness concerns related to gender stereotypes, but it can also lead to serious complications if incorporated into judgement and decision-making systems. The potential for such bias to inadvertently influence outcomes underscores the importance of addressing it.

















































We have created a dataset known as the Gender-Activity Binding (GAB) dataset as shown in Figure <ref>(a). This dataset comprises images depicting various activities performed by men or women in different contexts (with or without the presence of an individual of the opposite gender), each accompanied by a descriptive caption.


Given that our scenarios are uncommon due to biases and stereotypes, suitable real-world images are scarce for evaluation purposes. To address this issue, all images are generated using DALL-E 3[https://openai.com/dall-e3]. We employ extensive prompt enhancement techniques to ensure the diversity and quality of the generated images. We also evaluated the generated images on diversity, quality, and realism. We performed multiple quantitative assessments consisting of Fréchet Inception Distance (FID) for both quality and diversity assessment, Structural Similarity Index (SSIM) 
and Learned Perceptual Image Patch Similarity (LPIPS) for diversity assessment, and human feedback for a final qualitative assessment to ensure the final results accurately represent the intended activities and their related contexts.

The activities included in GAB dataset are sourced from various places to encapsulate societal gender stereotypes and other activities that are statistically biased towards a specific gender, as observed in the LAION-400M dataset <cit.>.






We benchmark the performance of 12 renowned multi-modal foundation models and assess their performance in terms of text-to-image retrieval, image-to-text retrieval, and recognizing activities on GAB dataset as shown in Figure <ref>(b). 
Specifically, in the image-to-text retrieval task, we observed that while an unexpected gender is doing a stereotypically biased activity, the average performance of VLMs declined by approximately 33.2% due to the presence of the expected gender in the scene. Moreover, most VLMs undergo an average accuracy reduction of approximately 13.2% when encountering gender-activity binding bias. On the other hand, in the text-to-image retrieval task, most VLMs achieve an accuracy of approximately 50%, indicating that their performance is nearly random, and they fail to recognize the performer of the activity based on the text.


















As far as we are aware, this research is the first to delve into the origins of the association between genders and activities in vision-language models by conducting a thorough analysis both image-to-text and text-to-image retrieval accuracy of VLMs. It encompasses not only straightforward scenarios involving single individuals, where VLMs demonstrate high accuracy for both genders <cit.>, but also more complex situations with individuals of different genders <cit.>. This comprehensive approach allows us to isolate the impact of the complexity of the scenario on performance decline <cit.> and more accurately measure the Gender-Activity Binding bias. Additionally, the study carries out supplementary experiments to investigate the bias in VLMs’ text encoders and their proficiency in recognizing activities.



To summarize, our contributions are as follows:

    
    
  * We have created a novel dataset, known as the gender-activity binding dataset. This dataset includes AI-generated images of various activities that are typically associated with a specific gender, and are excellent in terms of diversity, quality, and realness (Section <ref>).

    
    
    
    
    
    
    
    
    
    
  * We conduct an intensive performance benchmark of well-known vision-language models in retrieval tasks to assess their robustness against the Gender-Activity Binding bias. Our findings reveal that when two individuals of different genders are present, VLMs exhibit a bias towards binding the activity with the gender that is expected to perform it, resulting in a 13.2% drop in retrieval accuracy. However, in scenarios involving a single individual, these models demonstrate high image-to-text retrieval accuracy. We also demonstrate that VLMs lack the ability to bind gender and activity in text-to-image retrieval tasks (Section <ref>). 

    
  * We carry out additional experiments to investigate the binding bias in VLMs’ capability to comprehend activities and the bias present in their text encoders (Section <ref> and the Appendix). We delve into the insights that can be derived about how VLMs internalize this bias and its impact on the shared embedding space of VLMs (Section <ref>).

    
    







§ RELATED WORK







 §.§ Vision-Language Models

VLMs apply contrastive training to bridge the image and text embedding spaces. Using this shared embedding space, VLMs have exhibited a remarkable performance on a notable number of multi-modal downstream tasks such as image captioning, text-to-image and image-to-text retrieval, visual question answering, and text-to-image generation. As <cit.> and <cit.> mentioned, VLMs can be categorized into three main groups based on their architectures: I) Fusion VL-encoders such as FLAVA <cit.> and LXMERT <cit.>, II) Dual-stream encoders like CLIP <cit.>, GLIP <cit.>, and FILIP <cit.>, and III) VL Encoder-decoder including BLIP <cit.>.



 §.§ Gender-activity Binding Bias

Previous works discussed that VLMs have many categories of biases <cit.>, especially gender-related ones <cit.>. 

For example, <cit.> studied bias in VLMs as a task of measuring associations between entities and gender in visual-linguistic models using template-based masked language modeling. Moreover, <cit.> measured gender bias in VLMs by comparing the MASK token prediction probabilities of factual and counterfactual samples and concluded that VLMs inherit human stereotypes from the training data.

On the other hand, there exist a number of studies including a restricted assessment of how a VLM understands a particular activity <cit.>. For example, <cit.> has introduced a small dataset called Winoground containing objects, attributes, and activities to evaluate the ability of VLMs in compositional reasoning. Moreover, <cit.> designed VL-CheckList framework to study the capability of VLMs in understanding objects, attributes, and relations containing activities. However, to the best of our knowledge, there is no systematic study on the ability of VLMs in associating an activity with a gender.













§ GENDER-ACTIVITY BINDING BIAS






Despite the recent increase in large-scale data collection, it’s important to note that these datasets are not necessarily devoid of biases. These biases can manifest in large datasets derived from real-world distributions, primarily due to sample selection bias.

The issue causes VLMs to have varying levels of accuracy in recognizing the performer of an activity depending on the gender of the individual involved <cit.>. 
We refer to this bias in associating an activity with the gender of its actual performer in an image or text as Gender-Activity Binding Bias. 


Consider an image-to-text retrieval task, where we have two pieces of text C_1 and C_2, each describing an activity performed by a different gender. We also have an image I_1 depicting a person (either a man or a woman) performing an action. The task is to match the image with the correct piece of text (Figure <ref>-(a) left). Also, consider the text-to-image retrieval task, where we have two images I_1 and I_2, each showing an activity being performed by a specific gender, and we want to retrieve the correct image based on a given piece of text C_1 (Figure <ref>-(a) right). The Gender-Activity Binding bias manifests as a decline in the retrieval accuracy of VLMs when they incorrectly identify the gender of an individual performing activities typically associated with a different gender.


Let's denote the probability of a VLM correctly identifying the performer of an activity in a data i (image or text) as p, the activity as a, and the gender as g. Ideally, the probability p(g|a, i) should be equal for all g. 
However, because of the co-occurrence of gender g of the performer and the activity in the dataset, which is referred to as sample selection bias <cit.>, the gender of the performer becomes correlated with the activity. Thus, we often find that p(g_1|a, i)> p(g_2|a, i), where g_1 and g_2 represent different genders and g_1 shows the expected gender for the performer of a. It is also 
In simple terms, VLMs tend to incorrectly select the text or image because they are biased towards associating the activity with a specific gender. This is not always accurate, leading to errors in scenarios where the activity is performed by the less expected gender. 












The accuracy of models in image and text retrieval tasks is assessed in two scenarios: one where only a person of an unexpected gender (in terms of the bias) is depicted or described in the text or image, and another where both genders are present in the data, but the activity is performed by only one of them. Consequently, the impact of the bias can be observed as a decrease in accuracy in two situations: when the expected gender is absent or present, and when the activity is performed by the unexpected or expected gender. These cases together provide better insights into how these models handle gender-activity associations under different conditions. The first scenario allows us to understand how well the models can recognize and correctly associate an activity with an unexpected gender when the other gender is not present. This helps us gauge the models’ ability to break away from societal stereotypes and biases. The second scenario, on the other hand, tests the models’ ability to correctly identify the performer of an activity when both genders are present, which is a more complex and realistic situation. This indicates these biases affect the models’ performance in real-world scenarios. Together, these cases provide a comprehensive understanding of the models’ strengths and weaknesses in handling gender-activity binding, paving the way for targeted improvements to mitigate the gender-activity binding bias.






































































§ DATASET




This section details the creation process of the Gender-Activity Binding (GAB) Dataset, which serves to evaluate various VLMs against the gender-activity binding bias we have identified and VLMs ability to bind activities and performers. This dataset comprises AI-generated images that illustrate a variety of activities with potential subjects of both genders.





The GAB dataset divides images into four distinct groups: In two groups, both genders are present in the scene. In one of these groups, the expected gender is performing the activity while the other gender is just present in the image, and in the other group, the unexpected gender is performing the activity while the other gender is also in the scene.
In the next two groups, only one gender is present in the scene. In one of them, the expected gender is performing the action, and in the other one, the scenario is reversed, with the unexpected gender acting. 

For each activity, with respect to these grouping, we can consider four groups based on the performer of the activity (expected (E) or unexpected (U)) and the number of persons (with different genders) in the scene (1 or 2): E1, E2, U1, U2.






For each image in the dataset, we have templates that are replaced with activity or gender names, such as "man" and "woman," in accordance with the designed experiments. Details of the experiments and these replacements can be found in Section <ref>. The template for images containing two genders is: "a man/woman is doing activity and a woman/man is in the scene" and for images containing one gender, the template is: "a man/woman is doing activity". This specific template can also be used for images with two genders.





Each of the mentioned groups contains at least 26 images for each activity, resulting in a total of 5500 images.






This dataset structure facilitates the evaluation of gender-activity binding bias and the ability of VLMs to bind and recognize activities by definition of different retrieval tasks, as you can see in Section <ref>.

Pipeline of the dataset creation is shown in Figure <ref>(a)




 §.§ Selection of Activities




We proposed three methods to identify the biased activities, all utilizing Masked Language Modeling (MLM) as part of their approach to detecting biased activities. We utilized the 'RoBERTa-large' <cit.> model for its superior performance in masked token prediction, which was attributed to its objective function.

The sentences are formatted as "a mask is doing activity".[mask is the special mask token of the RoBERTa model] We then calculate the probabilities associated with "man" and "woman"
(as well as related masculine and feminine identifiers like "boy" and "girl"), from which we calculate the log ratios, as specified in Section <ref>.



  §.§.§ Stereotyped Activities from GPT Combined with MLM


Our first method involved using GPT-4 to request activities stereotypically associated with men or women in society. We provided over 100 activities from GPT-4 and formed sentences by replacing the activity token in the template of 'a mask is doing activity'. We then inputted these sentences into the RoBERTa model and calculated the following equation for the sentences.

    bias(sentence) = log(P(mask = man|sentence)/P(mask = woman|sentence))

We applied this methodology separately for activities stereotypically associated with each gender. For men, this resulted in a normal distribution[all the tests including log ratios formed normal distributions that we tested using Shapiro–Wilk test for normality] with a positive mean (μ > 0), from which we chose the sentences at the rightmost end. Similarly, for women, it also resulted in a normal distribution but with a negative mean (μ < 0), from which we then chose the sentences at the leftmost end.




  §.§.§ Everyday Activities from GPT Combined with MLM


In our second approach, we used GPT-4 to compile a list exceeding 1,000 sentences related to hobbies, jobs, and everyday activities. We then masked the subjects of the sentences just like in the first method and asked the RoBERTa model to fill the mask. We stored the log ratios of predicted probabilities. The log ratios formed a normal distribution, from which we selected sentences at each end.




  §.§.§ Activities from LAION-400M Dataset Combined with MLM


In the third method, we used the LAION-400M <cit.> dataset, which consists of 400 million image-caption pairs. We filtered out the NSFW pairs and sampled 20 million pairs randomly. We then extracted the verbs, subjects, and objects of all caption sentences using spaCy <cit.>, a natural language processing toolkit. We counted the number of masculine and feminine subjects for each verb-object pair and calculated their log ratios. This also formed a normal distribution.

We then used the Z-test to determine whether each verb-object pair had a significantly different ratio compared to the mean of all verb-object pairs. This test is appropriate due to the large dataset size of LAION-400M. By checking if differences are statistically significant, we identify biased activities. This method enhances the reliability of our activities by ensuring they are not a result of random variance or outliers. Then, we created sentences with the biased verb-object pairs as mentioned in Section <ref>, masked the subject, and submitted them to the RoBERTa model to fill the mask, conducting the same experiment as in the previous methods to select the biased activities.




















 §.§ Generation


The generation process consisted of two phases. The first involved generating a good prompt that helped the image generation system meet the criteria given below, derived from the base sentence. The second involved generating the image from the prompt. The generated images should accurately represent the original activity, look natural and high quality, and be diverse.


  §.§.§ Prompt Enhancement


We used a Large Language Model (LLM) to generate diverse prompts while also keeping the original activity intact (Details about the utilized LLM are provided in the Appendix.). We then created a list of predefined settings like the environment of the house, the skin color of the person, etc. Although we could have sampled N prompts from each setting to generate diverse images, we chose to create a list of different combinations of these settings and encode them using MiniLM <cit.> sentence transformer <cit.>. We then clustered them into K[K = 6 which resulted in sufficient diversity in our experiments] clusters and sampled N/K samples from each cluster to enforce more diversity semantically. We then generated a diverse prompt by instructing Llama2 to generate diverse prompts that maintain the original activity while describing the environment and the settings using the selected samples. Using these prompts, the generated images were significantly improved in quality and detail.



  §.§.§ Image Generation

We initially experimented with several diffusion models, as mentioned in the Appendix, to generate images. The generated images were realistic and diverse. However, they lacked representation of the intended activity, especially in the context of compositional generation. We then explored DALL-E3 <cit.> using OpenAI API. This approach resulted in significant improvement to the quality of compositional generation while also keeping the reality and the clarity of the image. We chose DALL-E3 as our final image generation solution.




 §.§ Image Filtering Methodology

Our image filtering methodology employs a combination of automated metrics and human evaluation to ensure the generated images are of high quality, relevant, and diverse. The process comprises three steps: quality assessment, diversity metrics, and human feedback, detailed as follows:



  §.§.§ Quality Assessment

The quality of generated images was assessed using the Fréchet Inception Distance (FID) <cit.> alongside criteria emphasizing the representation of human figures, as identified by the YOLOv8 <cit.> and MTCNN <cit.> models. The FID score aims to measure the similarity between the generated images and a set of reference images from the COCO <cit.> dataset, calculated as:


    FID(x, y) = ||μ_x - μ_y||^2 + Tr(Σ_x + Σ_y - 2(Σ_xΣ_y)^1/2),

where μ_x, μ_y are the feature-wise mean of the real and generated images, and Σ_x, Σ_y are the covariance matrices of the real and generated images, respectively. To achieve high-quality images, we initially filtered out images with high Fréchet Inception Distance (FID) scores. After this filtering process, we successfully reduced the mean FID score to 11.9. Furthermore, for an image to pass our quality assessment, individuals must constitute at least 15% of the detected objects and must have a detectable face, as confirmed by the MTCNN model. 
It’s worth mentioning that the FID scores for both expected (E1 and E2) and unexpected (U1 and U2) categories of images are 12.1 and 11.7 respectively, signifying an acceptable level of quality across all scenarios.



  §.§.§ Diversity Metrics



To ensure image diversity, we employed the Structural Similarity Index (SSIM) <cit.> and the Learned Perceptual Image Patch Similarity (LPIPS) <cit.> metrics. The SSIM index measures the similarity between two images; in our context, lower scores indicate greater diversity. It is defined as:


    SSIM(x, y) = (2μ_xμ_y + c_1)(2σ_xy + c_2)/(μ_x^2 + μ_y^2 + c_1)(σ_x^2 + σ_y^2 + c_2),

where μ_x, μ_y are the average intensities, σ_x^2, σ_y^2 are the variances, and σ_xy is the covariance. c_1 and c_2 are constants added to stabilize the division and avoid numerical instability with small denominators. Our target mean SSIM threshold across all image pairs is 0.046. It’s also worth mentioning that the SSIM for the expected (E1 and E2) and unexpected (U1 and U2) categories of images are 0.046 and 0.045, respectively.
This indicates a satisfactory level of diversity in both image categories.

LPIPS measures the perceptual similarity between two images, using deep network features to closely mimic human visual perception. To indicate greater diversity, we aim for higher LPIPS values, setting the mean target for all pairs at 0.66. Also, the LPIPS scores for the expected (E1 and E2) and unexpected (U1 and U2) categories of images are 0.66 and 0.65, respectively.





  §.§.§ Human Feedback

After quantitative evaluations, we performed a qualitative analysis with human feedback. This involved scrutinizing images for alignment with the desired activities and contexts. Images that did not align with the activities or did not depict the intended subjects were removed. Additionally, images with unrealistic or nonsensical elements, and those appearing abnormal to an external observer were excluded.

Following this step, a list of images that have passed both quantitative and qualitative evaluations is obtained. As a result, the images are diverse, high-quality, and realistic while also meeting our criteria for the generation phase.






§ EXPERIMENTS





























 We designed several experiments on our proposed GAB dataset to evaluate Vision-Language Models in text-to-image and image-to-text retrieval tasks as shown in Figure <ref>(b). Based on these experiments, we assessed the defined gender-activity binding bias and the ability of VLMs to recognize gender-biased activities. Finally, we evaluated the bias in the text encoders and image encoders of different VLMs separately.






 §.§ Setup





  §.§.§ Task Definitions

The experiments we have designed to assess the performance of VLMs on the aforementioned aspects are grounded in their zero-shot performance on both text-to-image and image-to-text retrieval tasks.


 






More formally, consider the space of images as ℐ and the space of captions as 𝒞.

 
Also consider a VLM with an image encoder E_𝒞:𝒞→ℝ^D and a text encoder E_ℐ:ℐ→ℝ^D, that map images and texts respectively to a shared embedding space, which is shown in the middle box in Figure <ref> (b). We can define matching score function s:ℐ×𝒞→ℝ^+, which measures the similarity between a caption and an image, as the cosine similarity between the embedding of an image I and the embedding of a caption C:


    s(I, C)=E_ℐ(I).E_𝒞(C)/‖ E_ℐ(I) ‖‖ E_𝒞(C) ‖.


Text-to-image retrieval involves finding the image that best matches a given text from a collection of images. Conversely, image-to-text retrieval is about finding the text that best describes a given image. We compute the similarity using the previously defined similarity measure in the upcoming experiments. For the text-to-image retrieval task, we have a caption C and make comparisons between two images: I, which shows an image corresponding to the caption C, and I^R which corresponds to C^R (obtained by reversing the gender of subjects in C). Similarly, for the image-to-text retrieval task, we have an image I and make comparisons between two captions C and C^R. 









  §.§.§ Selected Vision-Language Models



In our experiments, we benchmark 12 VLMs and present the results of various tests.



For comparing the effects of patch size and backbone size, we report the results for 4 CLIP models released by OpenAI <cit.>, each with different backbones and patch sizes.


Another model selected for evaluation is NegCLIP <cit.>, a version of the CLIP-ViT-B-32 model by OpenAI <cit.> which is fine-tuned on a modified subset of the Visual Genome Dataset <cit.> annotations. NegCLIP is fine-tuned to purportedly enhance the base model's performance on tasks requiring relational and activity understanding.


We also report results for several other notable VLMs that have been published in recent years. These include Eva01 <cit.> and Eva02 <cit.>, FLAVA <cit.>, ALIGN <cit.>, COCA <cit.> and AltCLIP <cit.>.










 §.§ Results

This section presents the outcomes of experiments designed to evaluate model performance across various tasks, including Image-To-Text and Text-to-Image retrieval, and activity recognition(Detailed results and descriptions of activity recognition experiments are provided in the Appendix.). Additionally, this section specifically addresses and reports on the bias observed in the text encoder.



  §.§.§ Image-to-Text Retrieval



We assess the performance of the models discussed in Section <ref> on the GAB dataset and calculate their accuracy in retrieving the correct caption as outlined in Section <ref>. We employ a caption template of the form: “a man/woman is doing activity and a woman/man is in the scene”. The results for activities that are stereotypically biased are presented in Figure <ref>, while the results for the other two groups of activities are provided in the Appendix. 













In this experiment, we assessed image-to-text retrieval across three distinct scenarios. The first scenario involves images where an individual of an unexpected gender is performing a typically biased activity with no other individuals present in the scene. The second scenario comprises images where both genders are present, and the activity is performed by the gender typically associated with it. The third scenario mirrors the second, but in this case, the activity is performed by the gender not typically associated with it.


Performance Drop Due to Presence of Expected Gender:
As depicted in Figure <ref>, there is a noticeable drop in the accuracy of the models when the scene includes two genders compared to scenarios where only the unexpected gender is present and performing the activity associated with bias. The chart reveals that most VLMs experience an average performance decline of approximately 33.2%. However, a few models demonstrate only a slight decrease in performance.


Performance Drop Due to Gender-Activity Binding Bias: 
In the scenarios where there are two individuals present at the scene, the majority of models display a substantial decline in retrieval accuracy when the action is performed by an individual of the unexpected gender. As indicated in Figure <ref>, the VLMs undergo an average accuracy reduction of approximately 13.2% when encountering gender activity binding bias. 

Note that, as illustrated in Figure <ref>, the models demonstrate satisfactory performance in the Image-to-Text retrieval task on both image scenarios of U1 and E1, indicating that the gender bias adversely affects the performance of the models only when there is more than one individual in the image. Additionally, the accuracy of the models remains consistent, regardless of whether the expected or unexpected gender is performing the activity. The models' average performance in both scenarios is approximately 80%.



  §.§.§ Text Encoder Bias

Given an activity a, consider e,u∈{man, woman} where e is the expected gender of the individual performing a and u is the unexpected one. To separately assess the bias of the text encoder, we determine the frequency at which the embedding of a gender-neutral sentence "a person is doing activity" is closer to the embedding of "a e is doing activity" than the embedding of "a u is doing activity". The outcomes are presented in Table <ref>. It can be observed that in all categories of the gathered activities, the majority of activities exhibit a bias towards the expected gender.














  §.§.§ Text-to-Image Retrieval




Figure <ref> illustrates the accuracy of VLMs in the task of text-to-image retrieval. For each activity, we form random pairs of images from E2 and U2[Pairing is in the form of one-to-one correspondence.]. Each model is then evaluated for its ability to retrieve the image that best matches captions formatted as "a man/woman is doing activity and a woman/man is in the scene". As can be seen in Figure <ref>, VLMs achieve an accuracy of approximately 50%, indicating that their performance is nearly random in this task.
If we replicate this experiment, altering only the captions to a gender-neutral phrase "a person is doing activity", and designate the image where the expected gender is performing the activity as the true label, the VLMs achieve an accuracy of approximately 50% again, as illustrated in Figure 4 in the Appendix.

In essence, the benchmarked models do not seem to comprehend the properties of the given images that would aid in retrieving the correct one given embedded information from the caption, i.e., they fail to recognize the performer of the activity based on the text.









































 











































§ DISCUSSION



 §.§ Image and Text Embedding in VLMs

To gain a deeper understanding of how texts and images are encoded into the shared embedding space, consider these points:

    
  * Given that the image retrieval accuracy is nearly 50% for both expected and unexpected caption groups (as shown in Figure <ref>), we can infer that the cosine similarity (as per Eq. <ref>) between both sets of captions and images (for expected and unexpected groups) is almost identical.
    
  * As the average text retrieval accuracy exceeds 60% for E2 and falls below 50% for U2, it suggests that text embeddings are somewhat closer to the image embeddings in E2.

A plausible interpretation of these observations is that VLMs encode images more closely to the encoding of the more expected text. However, the image encoders of VLMs embed images from both groups closely together, resulting in nearly equal cosine similarity with any caption embedding.



 §.§ Bias Mitigation

Multiple works have previously discussed approaches for mitigating bias from VLMs, such as orthogonal projection <cit.>, making unbiased datasets <cit.>, representation correction <cit.>, and prompt tuning <cit.>. Several of these approaches could be applied also for robustness to gender-activity binding bias. For more detail on these works, refer to the Appendix.




§ FUTURE WORKS

Study Other Social Biases.
In this work, we focused solely on gender bias; however, the experimental approaches described can be extended to other social biases, such as race and age. In future research, we plan to explore how VLMs perform across these dimensions using the experiments outlined in this study.

Study the Source of Bias in the Training Data.
Due to limited accessibility to the training sets, especially for models with unpublished datasets, this study could not investigate the source of bias in the training data directly. We plan to address this limitation in future work by examining the sources of bias in publicly available training datasets for VLMs and analyzing the origins of these biases within those datasets.






§ CONCLUSIONS





In this study, we have explored a significant yet overlooked bias in VLMs known as the gender-activity binding bias. We used a unique dataset named GAB, which includes about 5500 images. We found that the capacity of VLMs to link an activity with the gender of its performer notably decreases when another person of a different gender is in the scene. We discovered that while the bias is evident in image-to-text retrieval tasks and in the text encoder alone, the models perform randomly in text-to-image retrieval tasks. This suggests that the gender-activity binding bias is primarily absorbed by the text encoder rather than the image encoder in VLMs. Furthermore, we noted that while VLMs have difficulty with gender-activity binding, they do have some ability to recognize activities. We believe that the ability of VLMs to comprehend activities and perform compositional reasoning in complex scenes are other crucial factors contributing to the gender-activity binding bias that could be investigated in future research.
















































§ DATASET






 §.§ Detailed Report

The method of selecting activities is detailed above, ultimately resulting in three categories of biased activities. comprehensive report of each category provided in Table <ref> , Table <ref> , Table <ref>.











































































































 §.§ Experimented Models For Image Generation


Initially, we explored several diffusion<cit.> models including the original Stable Diffusion<cit.>, Stable Diffusion XL<cit.>, and its checkpoints like (Juggernaut XL[https://civitai.com/models/133005/juggernaut-xl] and DreamShaper XL[https://civitai.com/models/112902/dreamshaper-xl]). We tried Fooocus[https://github.com/lllyasviel/Fooocus], an image generation tool that includes its prompt rewriting system using GPT-2<cit.>. The generated images were realistic and diverse; however, they lacked representation of the intended activity, especially in the context of compositional generation. Upon using DALL-E 3 with OpenAI API, we observed a significant improvement in the quality of compositional generation while also keeping the reality and the clarity of the image. We chose DALL-E 3 as our final image generation solution.




 §.§ Used LLM For Prompt Enhancement


We evaluated various large language models (LLMs) for this task. Due to our hardware constraints, we ultimately selected the Llama2 model, which has 70 billion parameters, to generate prompts <cit.>.




§ BIAS MITIGATION

Multiple works have previously discussed approaches for mitigating bias from VLMs that could be applied also for robustness to gender-activity binding bias. 

In <cit.>, an orthogonal projection is applied to the end embeddings, making them orthogonal to the embeddings of spurious prompts, effectively mitigating bias introduced by these prompts. This alteration might also strip away useful nuances from the embeddings, potentially degrading the model’s overall performance on tasks where those nuances are beneficial and also this might not hold true in complex real-world scenarios where biases are not strictly linear.

In <cit.>, a method is introduced that utilizes additive residual image representations to correct biases in the original representations. However, training these residuals demands extensive and diverse data, making the process resource-intensive. Moreover, incorporating these residuals may alter other inherent characteristics of the original representations, potentially affecting the overall performance of the model.

In <cit.>, learnable prompts integrated into the text input are used to mitigate bias, optimized through the training of an adversarial classifier that evaluates similarity scores between outputs from two encoders. This method employs both adversarial and contrastive losses to preserve the quality of the joint representation while aiming to reduce bias. However, the effectiveness of this adversarial training depends on the availability of a large and diverse dataset; otherwise, there is a risk of overfitting to adversarial objectives. Additionally, since VLMs are trained with various objective functions, making adjustments to mitigate bias could potentially lead to trade-offs in other performance metrics.

Both <cit.> and <cit.> address the creation of fair datasets to mitigate bias in VLMs, though each method has its limitations. <cit.> uses AI to generate datasets for fine-tuning VLMs, which is computationally intensive and can introduce new biases due to the limitations of AI generation tools. On the other hand, <cit.> employs counterfactual data—images and texts modified to alter specific attributes. While this method targets specific biases directly, it is limited to those attributes it explicitly modifies and may not adequately address all potential biases.

In <cit.>, the authors address bias by calculating the mutual information between each dimension of the encoder outputs and the gender attribute, clipping dimensions with the highest mutual information to reduce their influence. While this approach effectively reduces gender bias in embeddings, it faces limitations similar to other inference-time interventions. Notably, it may not adapt well to a wide range of biases and could potentially degrade the accuracy of models in other downstream tasks due to the removal of critical information during the clipping process.







§ RESULTS






 §.§ Image-to-Text Retrieval


The performance of text retrieval for texts featuring two genders (one of them performing the activity) is demonstrated in Table <ref> and Table <ref>. This is akin to the experiment discussed in Section 5.2.1,
In this experiment, we assessed image-to-text retrieval across three distinct scenarios. The first scenario involves images where an individual of an unexpected gender is depicted performing a typically biased activity with no other individuals present in the scene. The second scenario comprises images where both genders are present, and the activity is performed by the gender typically associated with it. The third scenario mirrors the second, but in this case, the activity is performed by the gender not typically associated with it. In Table <ref> the experiment is reported for gender-biased activities from LAION-400M and in Table <ref> the experiment is reported for gender-biased everyday activities.








 §.§ Text-to-Image Retrieval


The performance of image retrieval for images featuring a single individual (the one performing the activity) is demonstrated in Figure <ref>. This is akin to the experiment discussed in Section 5.4 of the main text, where images in E1/U1 are categorized and the models are evaluated based on their ability to accurately retrieve the correct image using captions that follow the template “a man/woman is performing an activity”. The models exhibit an average accuracy of approximately 90%, indicating that they can effectively identify the image that correctly represents the gender of the individual performing the activity, provided there are no individuals of a different gender present in the scene.
Also, you can see the results of experiment mentioned in Section 5.4 of the main text in Figure <ref>.






 




 §.§ Activity Recognition


A crucial inquiry concerning the bias in binding gender with certain activities is whether this bias originates from a misunderstanding of the activity itself, and if this misunderstanding leads to a failure in binding the gender and the activity. To gauge the model’s understanding of activities, we collect several (at least 9, 17.25 on average) other different activities that are performed in the same environment as the reference activity for each activity in the GAB dataset using GPT-4 <cit.>. For each image in the GAB dataset, we create a caption from the reference and associate collected activities and arrange them based on their matching score to the image. We show the mean reciprocal rank (MRR), recall@1, and recall@3 for the true caption (reference activity) in Table <ref> for stereotypical activities. (results for other categories of activities is depicted in Table <ref> and Table <ref>).

The table indicates that the model can, to some extent, retrieve the true activity from a list of activities performed in a similar environment, demonstrating that VLMs can recognize the performed activity in image-to-text retrieval.




























 Paper Title: SimpleLLM4AD: An End-to-End Vision-Language Model with Graph Visual Question Answering for Autonomous Driving.



simplellm4ad-an-end-to-end-vision-language-model-with-graph-visual-question-answering-for-autonomous-driving


§ SIMPLELLM4AD: AN END-TO-END VISION-LANGUAGE MODEL WITH GRAPH
VISUAL QUESTION ANSWERING FOR AUTONOMOUS
DRIVING


Peiru Zheng, Yun Zhao, Zhan Gong, Hong Zhu, Shaohua Wu

IEIT Systems

abstract


 §.§ Abstract


Many fields could benefit from the rapid development of the large
language models (LLMs). The end-to-end autonomous driving (e2eAD) is one
of the typically fields facing new opportunities as the LLMs have
supported more and more modalities. Here, by utilizing vision-language
model (VLM), we proposed an e2eAD method called SimpleLLM4AD. In our
method, the e2eAD task are divided into four stages, which are
perception, prediction, planning, and behavior. Each stage consists of
several visual question answering (VQA) pairs and VQA pairs interconnect
with each other constructing a graph called Graph VQA (GVQA). By
reasoning each VQA pair in the GVQA through VLM stage by stage, our
method could achieve e2e driving with language. In our method, vision
transformers (ViT) models are employed to process nuScenes visual data,
while VLM are utilized to interpret and reason about the information
extracted from the visual inputs. In the perception stage, the system
identifies and classifies objects from the driving environment. The
prediction stage involves forecasting the potential movements of these
objects. The planning stage utilizes the gathered information to develop
a driving strategy, ensuring the safety and efficiency of the autonomous
vehicle. Finally, the behavior stage translates the planned actions into
executable commands for the vehicle. Our experiments demonstrate that
SimpleLLM4AD achieves competitive performance in complex driving
scenarios.

introduction


 §.§ 1. Introduction


Autonomous driving has garnered significant attention from both academia
and industry over the past decade.<cit.> The promise of
safer roads, reduced traffic congestion, and increased mobility for all
populations has driven rapid advancements in this field. Traditional
approaches to autonomous driving often rely on a modular pipeline
consisting of perception, prediction, planning, and control. However,
these methods can suffer from compounding errors across modules, leading
to suboptimal performance in complex and dynamic driving environments.

With the advent of large language models (LLMs),<cit.> a
new opportunity arises to redefine the approach to autonomous driving.
LLMs, particularly when integrated with vision-language models
(VLMs),<cit.> have shown remarkable capabilities in
understanding and generating human-like text based on visual inputs.
These capabilities can be harnessed to create more coherent and
integrated autonomous driving systems, capable of nuanced reasoning and
decision-making.

In this work, we introduce SimpleLLM4AD, an end-to-end autonomous
driving (e2eAD) method that leverages the power of VLMs. Our method
reimagines the traditional autonomous driving pipeline by structuring
the task into four interconnected stages: perception, prediction,
planning, and behavior. Each stage is framed as a series of visual
question answering (VQA) pairs, which are interlinked to form a Graph
VQA (GVQA). This graph-based structure allows the system to reason about
each VQA pair systematically, ensuring a coherent flow of information
and decision-making from perception to action.

The perception stage in SimpleLLM4AD utilizes ViT models to process raw
visual data, extracting meaningful features and identifying objects in
the driving environment. These visual insights are then translated into
a format that the language model can interpret, allowing for a more
sophisticated analysis of the scene. In the prediction stage, the system
forecasts the future states of the identified objects, considering their
potential movements and interactions. This predictive capability is
crucial for anticipating potential hazards and planning safe maneuvers.
The planning stage involves synthesizing the information gathered from
the previous stages to develop a driving strategy. This strategy is
designed to optimize safety and efficiency, taking into account the
dynamic nature of the driving environment. Finally, the behavior stage
converts the planned actions into executable commands for the vehicle,
ensuring precise control and responsiveness.

Our experiments demonstrate that SimpleLLM4AD not only achieves
competitive performance in driving benchmarks but also exhibits enhanced
robustness in complex scenarios. The integration of VLMs enables the
system to make context-aware decisions, significantly improving its
reliability and safety. Our key contributions are summarized as follows:

• We leverage the logical dependency of GVQA by utilizing the answers to
associated questions as contextual information for the current question.
This approach has been shown to significantly enhance the capabilities
of LLMs in terms of accuracy (ACC) and language score.

• We refine the prompts to further boost the performance of LLMs. We
optimize the simple question-and-answer (Q+A) format of the contexts to
simplify the contextual information, making it easier and more efficient
for LLMs to leverage previous knowledge.

• Finally, we introduce object detection branches into the LLM
optimization process, which include object localization, color
identification, and categorization. These additional branches provide
LLMs with a richer set of contextual cues, further enhancing their
performance.

related-work


 §.§ 2. Related work


large-language-models


  §.§.§ Large Language Models


LLMs have rapidly evolved in recent years, demonstrating unprecedented
capabilities in natural language understanding and generation. These
models, exemplified by OpenAI's GPT-3<cit.> and
GPT-4<cit.>, and other open-sourced transformer-based
architectures such as the LLaMA series,<cit.>
Vicuna,<cit.> Baichuan,<cit.>
Qwen,<cit.> Yuan 2.0,<cit.> Yuan
2.0-M32,<cit.> leverage vast amounts of data and
sophisticated training techniques to produce human-like text and
comprehend complex linguistic structures.

One of the significant breakthroughs with LLMs has been their ability to
perform a wide range of tasks with minimal fine-tuning. This versatility
arises from the models' pre-training on diverse corpora, enabling them
to capture extensive world knowledge and linguistic patterns. The
emergence of LLMs has led to remarkable advancements in various fields,
including natural language processing (NLP), question answering, machine
translation, and text summarization.

vision-language-models


  §.§.§ Vision-language Models


VLMs represent a significant advancement in the field of artificial
intelligence, aiming to bridge the gap between visual and textual data.
The architecture of VLMs typically involves two main components: a
visual encoder and a language decoder. The visual encoder processes the
image to extract features, which are then combined with text
representations generated by the language decoder. This integrated
approach enables the model to perform complex reasoning tasks that
require both visual and linguistic understanding. These models combine
the strengths of vision models, such as convolutional neural networks
(CNNs)<cit.> and ViT<cit.>, with the
capabilities of language models, allowing for more comprehensive
understanding and interaction with multimodal inputs.

Efforts in VLMs focused on tasks like image captioning and VQA. Image
captioning involves generating descriptive textual annotations for
images, while VQA requires the model to answer questions about the
content of an image. Nowadays, more and more VLMs have demonstrated
impressive capabilities in these areas by effectively integrating visual
and textual information. Flamingo<cit.> utilizes visual
and language inputs as prompts, demonstrating exceptional few-shot
performance in visual question answering. GPT-4,<cit.> the
LLaVA series<cit.> introduces visual instruction tuning
to enhance the instruction-following capabilities of VLMs. Concurrently,
models like KOSMOS-2<cit.>, and
Qwen-VL<cit.> have advanced VLMs by incorporating visual
grounding capabilities, facilitating tasks such as region description
and localization. Furthermore, PaLM-E<cit.> and
EmbodiedGPT<cit.> represent significant strides in
adapting VLMs for embodied applications, vastly expanding their
potential use cases. These advancements illustrate the rapid progress in
the development and application of VLMs, showcasing their ability to
handle increasingly complex multimodal tasks. As the integration of
vision and language continues to evolve, VLMs are poised to play a
pivotal role in various AI-driven domains, including autonomous driving,
healthcare, and interactive robotics.

language-grounded-driving


  §.§.§ Language-grounded
Driving


Language-grounded driving is an emerging research area that combines the
principles of NLP with AD technologies. This interdisciplinary approach
leverages the understanding and generation of natural language to inform
and enhance the decision-making processes of autonomous vehicles. By
grounding driving actions in language, these systems can achieve greater
interpretability, flexibility, and user interaction capabilities.

Recent advancements in LLMs and VLMs have propelled the field of
language-grounded driving. Models like GPTDriver,<cit.>
LLM-Driver,<cit.> and LMDrive<cit.> offer
new possibilities for integrating language grounding into autonomous
driving. These models enable more nuanced and context-aware
interpretations of driving environments and instructions.

Language-grounded driving represents a promising frontier in autonomous
vehicle technology. By integrating natural language understanding with
driving systems, researchers are developing more interactive,
interpretable, and adaptable autonomous vehicles. Our SimpleLLM4AD
method builds upon these advancements, utilizing language grounding to
enhance each stage of the e2eAD pipeline, from perception and prediction
to planning and behavior. This integration not only improves the
vehicle's decision-making capabilities but also facilitates better
human-vehicle interaction and scenario-based training.

method


 §.§ 3. Method


overall-architecture.


  §.§.§ 3.1. Overall architecture.


The pipeline of our method is outlined in Figure 1. The overall
architecture comprises two main modules: a vision encoder that processes
images, and an LLM decoder that handles questions.

Vision Encoder. We choose InternViT-6B as the vision encoder.
InternViT-6B, a vision transformer with 6 billion parameters, was first
introduced by Chen et al.<cit.> It was pre-trained using
web-scale image-text data from various sources to align with large
language models. The Query Model serves as a bridge between the vision
encoder and the LLM decoder, aligning the vision and text modalities.
This vision-text alignment component is initialized with a
multilingual-enhanced LLaMA.<cit.>

LLM Decoder. We employ Vicuna-13B as the LLM decoder.
Vicuna-13B is an open-source LLM that was fine-tuned from LLaMA using
user-shared conversations collected from ShareGPT.31 Although different
questions share the same LLM decoder model, we have devised a GVQA
strategy to enhance the language model's capabilities and crafted
tailored prompts based on different problem types.

In our method, each key frame in the nuScenes dataset undergoes a series
of QA pairs. Initially, the six images of the nuScenes key frame are
encoded into feature maps using InternViT-6B. Next, in the Query module,
the image features interact with 96 learnable queries and the question
text from the QA pair. The Query module transforms the image tokens
generated by InternViT-6B into representations that are aligned with the
LLMs. Finally, the output from the Query module is fed into Vicuna-13B
to generate an answer. Notably, the answer generated in the previous
step is combined with the subsequent question to form a context-rich
question. By iterating this process, the pipeline progresses round by
round to accomplish the end-to-end autonomous driving (e2eAD) tasks step
by step.

This modular approach guarantees that each processing stage—from
visual encoding to vision-text alignment and ultimately language
generation—is optimized for its distinct function while maintaining
seamless integration throughout the entire pipeline. This architecture
not only elevates the system's capability to handle intricate visual and
linguistic inputs but also ensures a coherent flow of information,
enabling more precise and context-aware decision-making in autonomous
driving scenarios.


    < g r a p h i c s >


Figure 1. Overall architecture.

gvqa-logical-dependency


  §.§.§ 3.2. GVQA logical
dependency


The SimpleLLM4AD method encompasses a sequence of four stages, each
intricately linked by the logical dependencies of the QA pairs they
contain. As depicted in Fig. 2, the logical dependency of GVQA is
graphically represented, illustrating the interconnected nature of the
QA pairs throughout the process. The diagram consists of two primary
elements: nodes (N) and edges (E). Nodes (N) symbolize individual QA
pairs, and the edges (E) that connect them denote the logical
dependencies that exist between them. Specifically, the answer from a
preceding node (NP) serves as contextual information for
the subsequent node (NS).

In the context of AD, understanding this logical dependency is crucial
for the system's decision-making process. For instance, during the
Perception stage, the system must identify key objects in the current
scene that will be considered for future reasoning and driving
decisions. This initial identification sets the stage for the Prediction
stage, where the system assesses the motion status of objects and
predicts their potential future states.

The Planning stage then utilizes the information gathered during
Perception and Prediction to formulate a safe and effective course of
action for the ego vehicle. This involves considering the potential
actions that could be taken in response to the identified objects and
predicting the outcomes of those actions, including the likelihood of
collisions and the safety of the proposed maneuvers.

Figure 2 elucidates the logical flow from one stage to the next, with
each node representing a critical decision point or informational
milestone. For example, (c1,CAM_FRONT,714.3,503.6) represents the
identification and initial assessment of an object captured by the front
camera. The subsequent nodes then build upon this information, asking
questions about the object's movement status and how it might interact
with other objects in the scene, such as node
(c3,CAM_FRONT,1300.8,531.7), which could represent another vehicle or
significant obstacle.

The edges connecting these nodes indicate the flow of logic and the
progression of the system's thought process. For instance, the question
of whether object `c1' will be in the moving direction of object `c3' is
predicated on the answers derived from the previous nodes. Similarly,
the decision-making process regarding the ego vehicle's actions is
contingent upon the predictions and assessments made in the preceding
stages.

By mapping out these logical dependencies, the SimpleLLM4AD method
ensures a coherent and systematic approach to autonomous driving
decision-making. This structured methodology not only aids in the
development of more sophisticated AI systems but also enhances the
transparency and reliability of the decisions made by these systems.


    < g r a p h i c s >


Figure 2. GVQA logical dependency.

experiments


 §.§ 4. Experiments


datasets-and-metrics


  §.§.§ 4.1. Datasets and Metrics


dataset


  
Dataset

The datasets used for fine-tuning and evaluation is
DriveLM-nuScenes.<cit.> DriveLM-nuScenes is a
comprehensive dataset specifically designed for developing and
benchmarking autonomous driving models. It consists of a training set of
4072 frames and a validation set of 799 frames, providing a robust
foundation for training and testing. DriveLM-nuScenes incorporates both
scene-level descriptions and frame-level QA pairs, divided into three
categories: perception, prediction, and planning. This structure ensures
a comprehensive understanding of the driving scenarios by addressing
various aspects of the driving process.

Perception: This category involves questions related to the thorough
examination of the entire frame. While some questions are manually
annotated, prompts are designed to generate questions based on the
observational facets of objects within the scene, leveraging ground
truth from nuScenes and OpenLane-V2 datasets.

Prediction: This category includes inquiries regarding the future states
of key objects and the ego vehicle in the current frame, focusing on the
reasoning process behind these predictions. Due to the complexity of
these predictions, the answers are manually annotated.

Planning: This involves questions related to the subsequent actions of
the ego vehicle within the current frame. Similar to prediction,
planning questions are challenging and require manual annotation for the
reasoning process.

Each key object within the QA pairs is encoded as c tags in the format
c, CAM, x, y, where c is the identifier, CAM
indicates the camera, and x, y are the coordinates of the object's 2D
bounding box in the respective camera's coordinate system. The dataset
also includes a dictionary in each key frame, recording basic
information about key objects such as bounding box size, category,
moving state, and visual description.

metrics


  
Metrics

To evaluate the performance of our models, we use a set of
well-established metrics tailored to different tasks within the
autonomous driving domain. The metrics are divided into three main
categories: P1-3 VQA Metrics, Behavior Task Metrics, and Motion Task
Metrics.

P1-3 VQA Metrics: These metrics assess the performance of
perception, prediction, and planning question-answering (QA) tasks.
Common Visual Question Answering (VQA) metrics are employed, alongside a
new GPT score for a more nuanced semantic evaluation.


BLEU: Measures n-gram overlap between generated text and
reference text, focusing on precision but often insensitive to semantic
nuances.

ROUGE_L: Focuses on recall by calculating the longest common
subsequence between model outputs and reference answers.

METEOR: Considers precision, recall, stemming, synonymy, and
word order, offering a detailed alignment between model outputs and
references.

CIDEr: Utilizes a combination of BLEU and vector space models,
calculating semantic consistency through cosine similarity of n-gram
TF-IDF vectors.

SPICE: Parses text into a syntactic dependency tree and maps it
to a scene graph, computing F-scores to evaluate the alignment between
predicted and reference scene graphs.

GPT Score: Provided by ChatGPT, this metric assesses prediction
quality through a more rational score based on robust reasoning
capabilities, assigning a score between 0 and 100.


Behavior Task Metrics: These metrics evaluate behavior
predictions, focusing on classification accuracy of behavior, behavior
speed, and behavior steer.


Classification Accuracy: Evaluates the accuracy of behavior
predictions by comparing the predicted behaviors of the ego vehicle to
the ground truth. The ground truth trajectory points are categorized
into speed and steering behaviors, and the accuracy is calculated based
on the agreement between predicted and actual behaviors.


These metrics ensure a comprehensive evaluation of the model's
performance across various aspects of autonomous driving, from
perception and prediction to planning and behavior prediction.

implementation-details


  §.§.§ 4.2. Implementation
Details


In our method, the SimpleLLM4AD model is finetuned with DriveLM-nuScenes
dataset. We use the pretrained weights of InternViT-6B and keep it
frozen. QLLaMA and 96 queries are trainable during finetuing. The
pretrained Vicuna-13B could be totally frozen or tuned by
parameter-efficient fine-tuning (PEFT)<cit.> methods such
as LoRA<cit.>. The image resolution is set to 224 × 224.
We finetuned the model on NVIDIA GPU with a learning rate of 1e-4 and a
global batch size of 16.

test-results-on-drivelm-nuscenes


  §.§.§ 4.3. Test Results on
DriveLM-nuScenes


All models in Table 1 are fine-tuned on the training split of the
DriveLM-nuScenes dataset and tested on the test set of the same dataset.
The DriveLM baseline,<cit.> which fine-tunes
LLaMA-Adapter-V2<cit.>, achieves a final score of 32.4,
primarily due to its poor performance on the accuracy (acc.) metric. Cho
et al. fine-tune LLaVA-1.5<cit.> and
Cube-LLM,<cit.> resulting in a significant improvement
compared to the baseline.

Our method, SimpleLLM4AD, demonstrates a marked enhancement in both the
accuracy and language scores, leading to a final score of 52.7. The
partial result presentation is illustrated in Figure 3. This substantial
improvement highlights the effectiveness of our approach. The detailed
methodology and analysis of this result will be expounded in the
following section.


    < g r a p h i c s >


Figure 3. Result Presentation.

Table 1. DriveLM-nuScenes Benchmark. (Test set)

[]@llllllll@
Method     Acc.     ChatGPT     Match    
BLEU1     ROUGEL    
CIDEr     Final ScoreDriveLM baseline<cit.>     0.0     65.1     28.3     5.0     8.4    
9.9     32.4LLaVA-1.5<cit.>     38.5     53.5     26.1     15.8     14.3     30.3    
36.1Cube-LLM<cit.>     38.5     89.4     39.0    
16.3     20.4     31.3     50.1SimpleLLM4AD     66.5     57.4     35.3     75.7    
73.4     15.3     52.7

ablation


  §.§.§ 4.4. Ablation


During our exploration, we conducted training and inference of
SimpleLLM4AD under various settings. The differences among these
solutions primarily involve the treatment of prompts and the methods for
detecting key objects. The baseline performance shown in Table 1 was
obtained by using LLaMA-Adapter-V2 for evaluation on a validation set
that we created. This validation set was extracted from the original
training set by selecting one scene out of every six.

Table 2. DriveLM-nuScenes results of different settings. (Validation set
split by us)

[]@llllll@
Method     Acc.     Match     BLEU1    
ROUGEL    
CIDErBaseline     60.1     1.16     57.6     65.9     3.3Version A     62.7     2.94     66.8     68.4     14.1Version B     63.4     2.94     72.2     72.4     17.0Version C     67.7     3.43     74.8     71.9     15.0Version D     67.5     3.55     80.5     75.1    
33.2Version E     -     24.6     -     -    
-

Notice: In Table 2, the "match" only measures the box coordinate
matching; unlike in Table 1, where the "match" includes both box
coordinate matching and the ChatGPT score.

chain-of-thought


  
Chain of Thought

Chain-of-Thought (CoT) is an approach for prompting, in which one
includes the intermediate steps of reasoning within the prompt
(intermediate “thoughts”), besidesthe task
input/output.<cit.> CoT was shown to significantly improve
the capability of LLMs to solve problems without resorting to any model
updates.

In SimpleLLM4AD, we employ CoT by use the answer of NP
as contextual information for NS, and NS
would always be the next QA pair of NP in the dataset.
Comparing to the DriveLM baseline, Version A that utilizing CoT show a
remarkable improvement in acc. And language score.

In the DriveLM-nuScenes dataset, `What are the important objects in the
current scene?' question would always be the first question in each
frame, which we call it N0. In Version B, we apply
N0 + NP as context for each
NS.

graph-of-thought


  
Graph of Thought

While CoT only support unidirectional single branch of thought when
prompting LLMs step by step, graph of thought (GoT) enable current task
use any previous QA pairs as context and would allow any subsequent
questions refer to the results of current task. After trying different
logical dependency graphs, the prominent one is shown in Fig. 2. By
using GoT to arrange context like this, Version C get a prominent
improvement.

prompt-optimization


  
Prompt Optimization

Refined prompt would contribute to the performance of LLMs. In the
solution of DriveLM baseline, the context that pass to the subsequent
question would be previous Q + A, making the context redundant and hard
for LLMs to understand. Therefore, we optimize the simple Q + A format
of the contexts to simplify the context, making it easier and more
efficient for LLMs to use the previous information.

The original answer of N0 is:

`There is a red car to the front of the ego vehicle, a white SUV to the
front of the ego vehicle, a white sedan to the front of the ego vehicle,
a black sedan to the front of the ego vehicle, and a red light to the
front of the ego vehicle. The IDs of these objects are
c1,CAM_FRONT,714.3,503.6,
c2,CAM_FRONT,993.3,503.3,
c3,CAM_FRONT,1300.8,531.7,
c4,CAM_FRONT,892.5,507.5, and
c5,CAM_FRONT,712.6,361.8.'

As you can see, the color/class information and the coordinate
information of the same object are separately described in two sentence,
which could make LLMs confused. Therefore, when using it as the context
for subsequent question, we format the answer as:

`c1,CAM_FRONT,714.3,503.6 is a red car to the
front of the ego vehicle.'

We only give the information of the object mentioned in the current
question instead of all objects detected in N0.

Besides, we modify other QA pairs into declarative sentence when used as
a context. For example, QA pair like `Would
c1,CAM_FRONT,714.3,503.6 be in the moving
direction of the ego vehicle? No.' would be modified to
`c1,CAM_FRONT,714.3,503.6 would not be in the
moving direction of the ego vehicle.' when used as a context.

By reformatting the QA pairs, brief and informative context provide
improvement of the language score in Version D.

detection-of-the-key-objects


  
Detection of the Key
Objects

Format_instruction is "Input six images in turn. The first of six
images is CAM_FRONT, which is in the front of
the ego vehicle. The second of six images is
CAM_FRONT_LEFT, which is in the front left of
the ego vehicle. The third of six images is
CAM_FRONT_RIGHT, which is in the front right
of the ego vehicle. The fourth of six images is
CAM_BACK, which is in the back of the ego
vehicle. The fifth of six images is
CAM_BACK_LEFT, which is in the back left of
the ego vehicle. The sixth of six images is
CAM_BACK_RIGHT, which is in the back right
left of the ego vehicle. number, number is
object box center coordinate in the image (1600*900)."

To enhance the performance, we have intergrated Format_instruction to
guide LLM. To accurately detect and classify objects, we leverage
existing, well-established detection networks, such as dinov2, which
provide us with robust target detection output. In addition, we have
trained a specialized detection classification network that goes beyond
mere object detection. This network is fine-tuned to identify key
attributes of the detected objects, including their color, precise
position, and even orientation. This approach allows us to generate
detailed descriptions such as "These is a red car to the front of the
ego vehicle, and the box center is [714.3,503.6]."

By incorporating this rich information about the location, color, and
class of objects within the ego vehicle's environment, we have achieved
a significant improvement in the match score of Version E. Especially,
the positioning accuracy of the box center has significantly improved to
24.6%.

conclusion


 §.§ 5. Conclusion


In this paper, we present SimpleLLM4AD, a multi-modal language model
that achieves competitive performance in complex driving scenarios. The
integration of VLM allows for more nuanced and context-aware
decision-making, significantly improving the robustness and reliability
of the autonomous driving system. This approach also highlights the
potential for LLMs to enhance multimodal AI applications, paving the way
for further advancements in the field of e2eAD. In the future, we will
continue to optimize box positioning as well as the alignment of
multi-frame vision-language large models and other related directions.


unsrt

<cit.>
<cit.>
<cit.>
<cit.>
<cit.>
<cit.>
<cit.>
<cit.>
<cit.>
<cit.>
<cit.>
<cit.>
<cit.>
<cit.>
<cit.>
<cit.>
<cit.>
<cit.>
<cit.>
<cit.>
<cit.>
<cit.>
<cit.>
<cit.>
<cit.>
<cit.>
<cit.>
<cit.>
<cit.>
<cit.>
<cit.>
<cit.>
<cit.>
<cit.>
<cit.>
<cit.>
<cit.>







 Paper Title: Robust Box Prompt based SAM for Medical Image Segmentation.





 

Y. Huang et al.


1National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, School of Biomedical Engineering, Medical School, Shenzhen University, China

nidong@szu.edu.cn 

2Medical Ultrasound Image Computing (MUSIC) Lab, Shenzhen University, China

3Shenzhen RayShape Medical Technology Co., Ltd, China

4Centre for Computational Imaging and Simulation Technologies in Biomedicine (CISTIB), University of Leeds, UK

5First Affiliated Hospital, Southern University of Science and Technology, Shenzhen People’s Hospital, China

Robust Box Prompt based SAM for 
 Medical Image Segmentation
    Yuhao Huang1,2, Xin Yang1,2 Han Zhou1,2 Yan Cao3 
Haoran Dou4 Fajin Dong5 Dong Ni1,2()
    July 2024
==========================================================================================



§ ABSTRACT 
The Segment Anything Model (SAM) can achieve satisfactory segmentation performance under high-quality box prompts.
However, SAM's robustness is compromised by the decline in box quality, limiting its practicality in clinical reality.
In this study, we propose a novel Robust Box prompt based SAM (RoBox-SAM) to ensure SAM's segmentation performance under prompts with different qualities.
Our contribution is three-fold.
First, we propose a prompt refinement module to implicitly perceive the potential targets, and output the offsets to directly transform the low-quality box prompt into a high-quality one.
We then provide an online iterative strategy for further prompt refinement.
Second, we introduce a prompt enhancement module to automatically generate point prompts to assist the box-promptable segmentation effectively.
Last, we build a self-information extractor to encode the prior information from the input image.
These features can optimize the image embeddings and attention calculation, thus, the robustness of SAM can be further enhanced.
Extensive experiments on the large medical segmentation dataset including 99,299 images, 5 modalities, and 25 organs/targets validated the efficacy of our proposed RoBox-SAM.






§ INTRODUCTION


The Segment Anything Model (SAM) has marked a significant breakthrough in image segmentation, due to its powerful zero-shot performance <cit.>.
SAM was trained on a large annotated dataset with 11M images and 1B masks based on the Vision in Transformer (ViT).
It supports different types of prompts, including point, box, text, and mask. 
Recent studies have widely explored the potential of applying SAM to the medical imaging domain <cit.>, which demonstrates that SAM can outperform the task-specific state-of-the-art conventional deep models after fine-tunning with additional medical images <cit.>.

Most previous studies equipped the SAM with high-quality prompts, e.g., the center point or tight box of the target.
However, this assumption typically cannot be held in clinical practice, where numerous low-quality prompts commonly occur.
As shown in Fig. <ref>, SAM prompted with high-quality boxes can achieve good performance, whereas it will easily generate poor segmentation after randomly shifting the promoting boxes. 
Hence, enhancing the robustness of SAM against imprecise prompts is crucial for its reliability.





Segmentation ability improvement of SAM.
Fine-tuning SAM using medical datasets to enhance medical image segmentation efficacy is an important field of research. 
Ma et al. ﬁxed the prompt encoder, and tuned the parameters in both image encoder and mask decoder during training <cit.>.
Wu et al. then introduced the adaptor modules with few trainable parameters to capture medical knowledge <cit.>.
Xie et al. reconstructed the mask decoder within SAM to develop a few-shot fine-tuning strategy <cit.>.
Moreover, Semantic-SAM <cit.> and HQ-SAM<cit.> were proposed to improve the segmentation quality. 
However, these studies aimed to improve the segmentation performance of SAM in different scenarios. 
They assumed to directly use the high-quality tight box as prompt, lacking the exploration of the low-quality prompt.


Robustness of SAM. 
Several studies have evaluated SAM's robustness under various corruptions and adversarial attacks <cit.>. 
However, their robustness tests were conducted at the image level rather than the prompt level. 
In <cit.>, the authors randomly shifted the tight box to mimic the practical operation.
They concluded that the SAM is sensitive to prompt randomness, especially box prompts.
The higher the random shifts, the more serious the performance degradation.
Although they have noticed the weak robustness of SAM under different prompt shifts, they did not introduce solutions to solve this problem.

One study observed that when subjected to low-quality prompts, SAM's mask decoder encounters an attention drift issue, leading to the cross-attention module activating inaccurate regions <cit.>.
To address this issue, stable SAM (SSAM) leveraged a deformable sampling plugin to shift attention to the target region.
However, SSAM only predicted the indirect feature offset to enhance the robustness.
Besides, it was developed and validated on natural images characterized by clear target boundaries, distinguishable foregrounds and backgrounds, and color encoding. 
These characteristics significantly differ from those of medical images, which may present unique challenges to the SSAM.


In this study, we introduce a novel robust box prompt based SAM named RoBox-SAM, aimed at making SAM robust to various prompt qualities.
Our contribution is three-fold.
First, we introduce an intuitive prompt refinement module (PRM) designed for potential target perception and directly predict box prompt offsets to refine low-quality prompts.
We also design an online refinement strategy to optimize the box prompt quality iteratively, and further achieve robustness improvement.
Second, we propose the prompt enhancement module (PEM), and leverage the movement information between the noisy and optimized prompts to output the potential point prompts automatically.
Finally, we extract intrinsic features from the input image via a self-information extractor (SIE), to enrich the image embeddings and refine the cross-attention calculation within the decoder. 
Validated on the large medical segmentation dataset, we demonstrate that RoBox-SAM is robust to box prompts of varying qualities, and general for different medical modalities and targets, achieving satisfactory results.





§ METHODOLOGY


Fig. <ref> shows the schematic view of our proposed Robox-SAM.

In stage 1, to eliminate the effects of low-quality prompts, the proposed PRM first predicts the offsets to transfer the input box prompt into a high-quality one.

In stage 2, we adopt the prompt correction knowledge to output potential points via the proposed PEM.

In the last stage, we leverage the prior knowledge encoded by SIE to optimize the image embeddings, and update the attention calculation in the mask decoder.
This can further optimize the feature activations under suboptimal box prompts, thus, improving the robustness of SAM.



Original SAM.
SAM mainly consists of three parts: image encoder, prompt encoder, and mask decoder <cit.>.
The image encoder is used to extract the image embedding, while the prompts are encoded using different methods (e.g., point and box: positional encoding <cit.>, text: CLIP <cit.>). 
Before being fed to the mask decoder, the image embedding should add the positional encoding, while the prompt tokens should concatenate with two output tokens, i.e., IoU token and mask token, to form the embedding (e) and tokens (t), respectively.
The mask decoder mainly contains two Transformer layers, and each layer contains four modules: 1) self-attention of tokens, 2) token-to-image cross-attention, 3) Multi-Layer Perception (MLP) on tokens, and 4) image-to-token cross-attention.
For example, the token-to-image cross-attention can be formulated as C_Att(t,e)=softmax(Q(t),K(e)^T/√(d_k))· V(e), where d(k) represents the dimension of the keys.
Finally, the mask token is separated from tokens and operated with the final up-scaled image embedding to predict multiple outputs.
Besides, the IoU token predicts the IoU score for each mask.



PRM for Target Perception and Box Prompt Correction.
Inaccurate box prompts may cause the SAM's mask decoder to activate imprecise regions, and the cross-attention between image features and tokens may further accumulate the errors, resulting in unstable segmentation <cit.>.

We propose the PRM to learn and perceive the target location, and predict the offsets to update the input prompts.
Given the image and prompt features (f_img, f_p), cross-attention can be performed to obtain f^*=f_img+C_Att(f_img,f_p).
Then, f^* will be inputted to the light-weighted PRM, outputting the offsets Δ(o)={Δ x_1, Δ x_2, Δ y_1, Δ y_2}.
The input box prompt (p_c={x_1,x_2,y_1,y_2}) can be updated to p_c^*=p_c+Δ(o).
The loss function can be written as L_o=L_1(p_c^*, p_gt), where p_gt represents the coordinate of GT tight box.
This supervision can drive the development of SAM's target perception ability, outputting reliable offsets to make the input box prompt move and match the potential target.
Moreover, during testing, the box refinement process can be interactive. Specifically, the output refined box in the current step can be considered as the input of the next step.
The iteration will terminate when ∑Δ(o)=0 or iterating over k=5 steps.
This interactive refinement strategy can further improve the model's accuracy and robustness.


PEM for Automatic Point Prompt Generation.
We only consider the box as the initial input prompt. 
However, we believe that the point prompts can assist the box prompts in some cases, especially negative points for restraining over-segmentation.
The above box correction process is intuitive and effective in revealing the potential region for point prompt generation.
Refer to stage 2 (right-down corner) in Fig. <ref>, given the original (green, A) and refined (red, B) boxes, and the annotated ground truth (GT) mask (orange, C), the non-GT region (i.e., (A∪ B)-C) indicated by yellow arrows can be treated as the potential negative area.
This region can be flattened into a one-dimensional vector and n uniformly sampled points can be taken as GTs (p_gt).
The CNN-based PEM (ℳ) can learn the potential points by inputting image-prompt attention features before and after refinement (p_pre=ℳ(f_r^*-f^*)).
Specifically, it generates n=5 points and matches them with sampled GT points one by one to find the combination with the smallest distance errors to calculate the L2 loss: L_p=||(p_pre,p_gt)||_2


Self-information Mining for Feature and Attention Optimization.
Utilizing the prior knowledge of the input image can enhance the accuracy and robustness of the segmentation model.
In the self-information pool, we provide several components that contain useful shape and edge information.
As shown in the green block of Fig. <ref> (stage 3), yellow arrows in the canny-based sketch <cit.> and SLIC-based superpixel <cit.> represent the location and boundary of the target.
We also adopt the fast Fourier Transform to transfer the image to the frequency domain <cit.>, including amplitude and phase parts.
Specifically, the low-frequency amplitudes are masked to highlight the details and edges of the image.
This information was encoded by the proposed SIE, and the output dimension was mapped to the same as the image embedding.
Then, the image embedding can be re-calculated by f'=f^*+f_s to fuse and emphasize the self-information.
Finally, f' will be input into the cross-attention module for attention re-calculation and optimization.
Thus, though the refined box prompts may still include noise, the inherent self-information can help remove it via attention update, and assist in improvement in terms of both accuracy and robustness.







§ EXPERIMENTAL RESULTS


Materials and Implementations. 
We validated the RoBox-SAM on a large medical segmentation dataset, including 99,299 images/slices, 5 modalities, and 25 types of targets.
This dataset contains 14 public and 5 in-house datasets (marked with ^*).
Specifically, for CT datasets <cit.>, the targets include chest (heart, lung, trachea) and abdomen (aorta, bladder, gall bladder, kidney, liver, pancreas, spleen, stomach).
MRI datasets contain the brain <cit.>, brain tumor <cit.>, knee (femur and tibia) <cit.>, and prostate <cit.>.
Ultrasound datasets have nine objects, including breast nodule^*, eyes^*, femur^*, heart <cit.> (left atrium, left ventricle endocardium, left ventricle epicardium), kidney pelvis^*, nuchal translucency^*, and thyroid nodule <cit.>.
We also considered two other targets from two modalities, i.e., Colonoscopy: polyp <cit.> and Fundus: OpticDisc <cit.>.
Approved by the local IRB, the in-house ultrasound datasets were collected and annotated by an experienced sonographer under strict quality control.
CT/MR volumes were sliced into 2D images similar to <cit.>.
The whole dataset was split into 6:1:3 for fine-tuning, validation, and testing, respectively.


We implemented RoBox-SAM in Pytorch, using an NVIDIA A40 GPU.
The batch size, learning rate, and total epoch are set to 16, 1e-4, and 20, respectively.
SAM's backbone was ViT-H.
GT box was generated by the adjacency box of the annotated mask.
Input box prompts were randomly shifted by 0-30% based on the GT box to evenly mimic the human operations with varying experiences/annotation preferences, with the condition that their IoU>0.5.
We followed the SAM to use cross-entropy and dice losses to optimize the model via AdamW optimizer.
We further added L_o and L_p to assist the model learning.
The proposed PRM, PEM, and SIE are two-layer CNNs with linear projection.
They encode the high-dimensional features and output offsets, points, or low-dimensional features to satisfy specific requirements.
As shown in Fig. <ref>, during fine-tuning, the original components in SAM were frozen, whereas other parts were learnable.
During testing, all the parameters were frozen, and the mask decoder will output three predictions with corresponding predicted IoU scores.
The mask with the highest IoU prediction was selected as the final output.






Quantitative and Qualitative Analysis.
We performed N=5 times random testing, and reported the average performance to obtain stable and reliable results.
Evaluation metrics in terms of accuracy (DICE, %) and robustness to prompt (PR, %: ∑_i=1^N(DICE(M_i, M_u))/N, similar to <cit.>), were used to validate the segmentation performance.
M_i represents the mask in i-th testing, while M_u is the union mask region of all N masks.
The higher the PR score, the more consistent the segmentation results are across box prompts of different qualities.


Table <ref> compares the RoBox-SAM with 7 methods, including SAM <cit.> (different model sizes: ViT-B/L/H), MedSAM <cit.>, MedSAM_boxaug, MedSAM_offset, and HQ-SAM <cit.>. 
All the MedSAM-based methods and HQ-SAM are based on ViT-H and fine-tuned using our medical datasets.
MedSAM_boxaug adds the random box augmentation strategy during fine-tuning, and MedSAM_offset trains an additional detector to predict the offsets for prompt correction before inputting to SAM. 
Based on the first five experiments, we concluded that enlarging the model size, fine-tuning using medical images, and augmenting with random boxes can optimize model robustness in most cases.

However, comparing MedSAM with MedSAM_offset, we found that adding naive offset prediction cannot steadily improve robustness, with 3/5 modalities having performance drops.
RoBox-SAM achieves the best performance among all the competitors (p<0.05), including the strong HQ-SAM, showing its efficacy and superiority in model robustness improvement.
Besides, RoBox-SAM is economical, has low training learnable computational overhead (<1M params), and takes only ∼0.6s per testing image.






We also performed ablation studies to evaluate the contribution of each proposed module in Table <ref>.
Specifically, we treated MedSAM as baseline, and progressively added the proposed modules to observe the performance changes.
It can be seen that PRM contributes ∼6% and ∼8% improvements to DICE and PR scores, respectively.
Adding PEM, the metrics can be increased further.
This illustrates that the generated point prompts can effectively assist box prompts in achieving better segmentation. 
By aggregating the prior self-information (RoBox-SAM), the model's performance can achieve consistent improvement across the five modalities.
It is highlighted that the performance improvement brought by each module is statistically significant with p<0.05.





In Table <ref>, we further decreased the quality of box prompts by progressively increasing the random shift max to 30%.
This can verify the model's performance under extreme conditions.
The lines GT mean that the tight boxes of objects are taken as input box prompts.
In this situation, although the PRM may not output large offsets, point prompt enhancement and self-information mining can still improve the segmentation.
Thus, RoBox-SAM shows a slight improvement compared with MedSAM under high-quality tight box prompts.
As the box prompt quality continued to decline, the accuracy and stability of MedSAM severely degraded.
For example, CT targets dropped about 50% in DICE values (89.06%→49.98%).
The performance of RoBox-SAM was less affected by low-quality box prompts (e.g., CT: 5.26%↓ in DICE).
This shows that RoBox-SAM can cope well with extreme box prompts and output more stable results than MedSAM.
Moreover, we provided the results of RoBox-SAM (Iter), and found that the interaction strategy can help boost the DICE and PR scores, especially for extremely low-quality box prompts (>1% significant improvement).

Fig. <ref> and Fig. <ref> are qualitative results, with DICE scores shown in the right-down corners of the images.
Fig. <ref> visualizes the segmentation results of compared methods and our proposed RoBox-SAM.
It can be observed that the original SAM shows poor performance under low-quality prompts.
RoBox-SAM performs better than other methods, approaching the upper-bound (RoBox-SAM_GT) and GT.
Fig. <ref> (a)-(i) shows the box refinement and performance comparison (in each sub-figure, from left to right shows GT, MedSAM, and RoBox-SAM results).
Given low-quality box prompts, MedSAM produces poor prediction, while RoBox-SAM can refine the box and output satisfactory results.
Fig. <ref> (j)-(k) shows the interactive prompt refinement process.
RoBox-SAM performs worse in the first-round refinement, but achieves good correction finally due to the iterative refinement strategy.
Fig. <ref> (l)-(m) reveals the robustness of RoBox-SAM. In each case, given three different random prompts (orange boxes), RoBox-SAM can refine them to the green ones, and output stable results. 





§ CONCLUSION


In this paper, we propose RoBox-SAM for improving SAM's robustness to box prompts of different qualities.
We first directly refine the input box with poor quality via the proposed PRM to obtain a high-quality box prompt.
Then, we automatically generate potential point prompts through REM to assist the box-promptable segmentation.
Last, we mine the self-information in the image to further ensure robustness via attention optimization.
Extensive experiments on the large medical dataset validate the efficacy of RoBox-SAM.
In the future, we will extend the framework to more modalities and 3D segmentation tasks.




  §.§.§ 

This work was supported by the grant from National Natural Science Foundation of China (12326619, 62101343, 62171290), Science and Technology Planning Project of Guangdong Province (2023A0505020002), and Shenzhen-Hong Kong Joint Research Program (SGDX20201103095613036).



  §.§.§ 

The authors have no competing interests to declare that are relevant to the content of this article.


splncs04




 Paper Title: Social Learning through Interactions with Other Agents: A Survey.


Confining modeling of quark propagator
    X.L. Shang
    August 5, 2024
======================================



§ ABSTRACT 
    Social learning plays an important role in the development of human intelligence. As children, we imitate our parents' speech patterns until we are able to produce sounds; we learn from them praising us and scolding us; and as adults, we learn by working with others. In this work, we survey the degree to which this paradigm – social learning – has been mirrored in machine learning. In particular, since learning socially requires interacting with others, we are interested in how embodied agents can and have utilised these techniques. This is especially in light of the degree to which recent advances in natural language processing (NLP) enable us to perform new forms of social learning. We look at how behavioural cloning and next-token prediction mirror human imitation, how learning from human feedback mirrors human education, and how we can go further to enable fully communicative agents that learn from each other.
We find that while individual social learning techniques have been used successfully, there has been little unifying work showing how to bring them together into socially embodied agents.




§ INTRODUCTION

Social learning is “learning that is influenced by the observation of another individual or their products” <cit.>. The “embodiment hypothesis” in cognitive science suggests that intelligence emerges as a result of embodiment within a social environment, that “starting as a baby grounded in a physical, social, and linguistic world is crucial to the development of the flexible and inventive intelligence that characterizes humankind” <cit.>. We can define this embodiment as being when “the environment is acting upon the individual and the individual is acting upon the environment” <cit.>. We explore how social embodiment – learning that is influenced by the actions and products of other agents in the learning environment – has played a role in the development of AI, and how it could influence the development of embodied agents in the real world.

Despite often not being recognised as such, a large proportion of AI research can be considered or framed as employing aspects of social learning. In particular, models trained on human-produced data often implicitly rely on learning to imitate humans. Take for example language model pretraining, in which the model is trained to directly reproduce human text. The form of this social learning can have important implications, for example through the reproduction of human biases in textual data <cit.> and facial recognition <cit.>. As such, the prevalence and impact of AI systems trained in this way warrants a more thorough investigation of the mechanics of social learning.

From another perspective, social learning techniques have largely seen success on supervised offline learning tasks with fixed datasets. Real-world deployment often requires accounting for more complicated, two-way interactions between agent and environment. This has traditionally been done through individualistic reinforcement learning methods, which have issues like sample inefficiency, unspecified reward functions, and poor explainability <cit.>. In contrast, humans – who develop in a socially embodied setting – are able to quickly adapt to changes in the environment. Since communication between agents has been made easier by the development of LLMs, it is an apt time to research the development of agents using social learning.





 §.§ Social Learning in Humans


Social learning plays an instrumental role in the way humans learn <cit.>. We do not learn individualistically by simply observing the world – instead, we take advantage of the fact that other humans have learnt before us and have built up cultural knowledge.
Indeed, Homo Sapiens' exploitation of cultural knowledge is exactly what has allowed us to build complex societies and dominate the globe. Rather than having bigger brains (and thereby having a cognitive advantage), our ability to “accumulate information across generations and develop well-adapted tools, beliefs, and practices that no individual could invent on their own” is responsible for our success <cit.>.

Similarly, the transmission and generation of this cultural knowledge is of paramount importance in human development at both an individual and societal level. Social learning enables this cultural transmission and development through our interactions with others. This suggests to us that the promising route to achieving more advanced agents lies not just in building bigger artificial brains (scaling) but also in building artificial brains socially, that can learn from and teach other agents, and take advantage of and develop this rich cultural knowledge base <cit.>. 

According to Tomasello's influential theory of Social Learning, it can be divided into Imitative Learning, Instructive Learning, and Collaborative Learning <cit.>.
The organisation of this survey follows this division, as depicted in Figure <ref>.



  
Imitative Learning.
In Tomasello's theory of Social Learning, children learn by copying adults in a number of ways. A distinction is drawn between emulation and true imitation. In the case of the former, the child attempts to reproduce the actions and outcomes of the adult demonstrator; in the latter, the child additionally models the intent behind the action and learns from that. Such true imitation seems to require at the least a nascent theory of mind, insofar as it requires modelling the intents of others.




  
Instructive Learning.
Instructive learning refers to the process by which an adult or expert learner tries to teach a non-expert learner. Other than explicit instruction, this process can include feedback, explanations, demonstrations and construction of a curriculum. Collectively, the process by which these mechanisms are employed to aid a learner is referred to as scaffolding <cit.>.



  
Collaborative Learning.
Collaborative learning is the setting in which neither or none of the collaborators are experts. In these cases, the learners must work together to progress in the learning task. 
From a developmental perspective, this emerges during early education – in contrast to imitation learning which is observed almost from birth <cit.>. However, it plays an important role in childhood development, especially in the development of language outside of their asymmetric interactions with adult teachers. For example, Vygotsky and Cole vygotsky1978mind emphasise the role of childhood play in enabling peers to practice social roles within their capabilities.



 §.§ Social Learning in AI


At first glance, some forms of social learning techniques have been widely used in AI at large <cit.>.
Diving deeper, we seek to make sense of these from a unified perspective, and examine whether there are more aspects of social learning that can be applied to AI.

In the simplest case, training data is generated or curated by humans. Furthermore, many models are trained to generate more of this human-supplied data – this often consitutes a form of imitation learning. For example, a generative image model may be trained to imitate the art or photography of humans. While such imitative models are powerful, they are also inherently limited to recreating the level of the source of their learning <cit.>. Several instructive techniques like feedback <cit.> and curricula <cit.> are also widely used in various domains. Finally, collaborative learning regimes have been explored in the Multi Agent Reinforcement Learning (MARL) literature <cit.>. All these techniques and their relations are depicted in Figure <ref>.



 §.§ LLM Agents


The development of LLMs poses a striking opportunity to enable embodied agents to learn socially. Indeed, early experiments with creating embodied agents almost entirely from LLMs have shown that such agents are able to achieve remarkable zero-shot performance on embodied tasks <cit.>. This has been boosted by the development of Vision Language Models (VLMs) and other multimodal models that have been used to create agents that can naturally leverage human demonstration data and cultural knowledge <cit.>. In addition, since they are language models, such agents possess an inherent ability to communicate with each other, without the need to develop such methods from scratch. This has accordingly enabled the creation of cooperative groups of agents <cit.>.



 §.§ Objective of This Survey


There has not been a recent survey on social learning covering the developments arising from large language models (LLMs). Meanwhile, a growing number of works have called for more exploration of social learning approaches <cit.>.
In this survey, we aim to provide a review of recent works that use social learning techniques with a view to building agents that can learn just through interacting with others. Given the relative maturity of imitation techniques for training generative models, in this survey we put more focus on the potential for training agents using instructive and  collaborative forms of social learning, as this has been most directly enabled by the emergence of LLM Agents.

As depicted in Figure <ref>, we first consider existing approaches to imitation learning, both in the wider machine learning literature, as well as in their application to building embodied agents. Relatedly, we briefly consider some of the work that has explored how to enable this imitation learning in Section <ref>. In Section <ref>, we then extensively explore how feedback has been used for aligning LLMs, and how richer forms of feedback might be leveraged. Many of these lessons are also applicable to Explanations and Instructions as briefly covered in Section <ref>. We then explore the degree to which curricula are useful for these embodied model given some negative results in Section <ref>. We cover how MARL has traditionally been used for collaborative learning in Section <ref> Finally, we cover the recent development of collaborative LLM-agents in Section <ref>.

More generally, by carefully considering the gaps between human social learning paradigms and current techniques used in training embodied agents we hope to inspire research that more fully utilises the advantages offered by social and cultural learning.



§ IMITATIVE LEARNING




 §.§ LLM Pretraining

Large language models are largely trained on the task of next token prediction – essentially predicting the next word in a sentence. This is arguably a form of imitative learning in so far as the model is fed observations of the productions of a human agent (the sequence of tokens), and learns to autoregressively predict that sequence of tokens. As indicated in Section <ref>, such models have achieved remarkable feats, to the point of matching humans on a wide number of tasks.

Nevertheless this acquisition process is highly data hungry – GPT-3 was trained on over 200 billion tokens, whereas the average 12 year old would have only seen roughly 100 million <cit.>. One potential explanation for this is that the techniques used for training LLMs do not consist in true imitation in the sense described by Tomasello et al. tomasello1993cultural and rather perform “emulation”. However, Kosinski kosinski2023theory argues that LLMs already possess theory of mind, although this position has been criticised <cit.>.

 More pointedly, during the training regime, a large language model is devoid of the socially-embodied context that typically accompanies a child learner, such as being able to build up a model of the social environment that may be useful for prediction. In this context, language models have been shown to recreate human biases, reproduce a static view in the face of changing social mores, and lack accountability <cit.>.

We have already seen how childhood development relies on other learning regimes <cit.>, therefore what remains to be seen is the development of models that can actually leverage this socially embodied setting to further learn and adapt through their collaboration with other agents. Indeed, this has begun to be explored through building social simulation environments <cit.>. Hopefully this will enable us to move beyond LLMs just acting as “tools of imitation” <cit.> and mitigate some of the social issues mentioned above.



 §.§ Imitation Learning

The field of Imitation learning itself (and indeed its dual – instruction by demonstrations) has been a popular reward-free method of training embodied agents. We do not aim to provide a comprehensive overview of the field, as covered by e.g. <cit.>. Key examples of techniques in the field include:

    
  * Behavioural Cloning – in behavioural cloning, direct access to the underlying policy of the teacher agent is assumed, and used as a training signal.
    
  * Inverse Reinforcement Learning (IRL) – in IRL, the reward function is generated from the teacher's policy and then used to train the student model by existing reinforcement learning algorithms.
    
  * Observational Imitation Learning (OIL) – Rather than assuming direct access to the teacher's policy, in observational imitation learning, the learner agent only has access to external observations of the teacher agent acting in the learning environment.

These imitation learning techniques been widely used in the development of embodied agents; For instance RT2 <cit.>, an embodied foundation model for robotics, was created by collating and training on a dataset made from a mix of human demonstrations and question answering web data. This model was then pretrained through OIL (in the form of next token prediction). Similarly, AlphaStar <cit.> was also pretrained from imitating games of professional players, and Ramrakhya et al. ramrakhya2022habitat utilise an embodied object-search strategy learnt from human demonstrations and robot trajectories.



  
Section Summary:
Imitative Learning techniques are widely used, including in RL and NLP. There is nevertheless valuable work to be done exploring ways to increase the sample efficiency of these techniques, and whether socially embodied settings may play a role in this.




§ INSTRUCTIVE LEARNING



 §.§ Demonstration


Demonstrations as an instructive learning paradigm consist largely in the provision of examples that a student agent can imitate. Importantly this means that the teacher may impart additional intent onto the demonstration through the choice and mode of presentation of those examples. For example a human teacher may draw attention to particular aspects of a demonstrated process by narrating it. Indeed human demonstrations are typically accompanied by explanations and narrations. Furthermore Ramrakhya et al. ramrakhya2022habitat verify that using imitation learning techniques on 70k human demonstrations outperforms using 240k agent trajectories, indicating that the quality of demonstrations can have a large impact. We have seen relatively little research exploring these elements of demonstration learning. Sumers et al. sumers2023show conduct a study on the provision of demonstrations to humans and find that if the demonstrations are not accompanied by explanations, they are less effective than linguistic interactions between teacher-student pairs for conveying concepts.

Bhoopchand et al. bhoopchand2023learning do explore some of the mechanisms at play with demonstrations, especially in the light of how they can be performed to maximise cultural transmission. They explore a navigation task with demonstrations performed by humans, that are then dropped out. Naive imitation (or rather emulation) of the experts enables the agent to perform well when the expert is there to follow, but once the expert is removed, these agents fail to generalise. The authors are able to create an agent that can imitate the plan of the teacher through equipping the agent with 1) memory, 2) an attentional bias towards the expert, 3) the expert “dropping out” during training, and 4) randomisation of the domain to ensure that the student agent is learning to utilise the expert and not just memorise the paths. This “attentional bias” may be related to the notion of intent as an aspect of “true imitation” <cit.> and further accords well with Tomasello's tomasello2009cultural theory of joint attention.


 §.§ Feedback


Feedback is just information given by the teacher on the performance of the learner. Consider the example of a student learning to perform addition. If they perform an incorrect addition like 18+17=25 we could provide a simple label evaluation (incorrect), a preferred answer (18+17=35), or language feedback like “you forgot to carry over the tens term” – the particular form of feedback that is appropriate will depend on the situation.

Labels are of course omnipresent in machine learning, as supervised learning forms a major paradigm. Learning from preferences has also been developed especially in the field of RL, where Reinforcement Learning from Human Feedback (RLHF) has now been adapted for finetuning language models <cit.>. While we mostly consider linguistic feedback, human feedback is often multimodal in nature, and may include gestures, tone, expressions and other physical actions. We encourage readers to refer to <cit.> for progress in these dimensions; we expect these to become increasingly feasible and important as more works appear exploring multimodal models.



  §.§.§ Preference-Based Feedback


Human preference-based feedback on generative AI output has proven invaluable for aligning the outputs of foundation models with human expectations through the use of `instruction tuning' <cit.>. This finetuning technique was designed for the purposes of alignment – the process of optimising a model according to human values and needs (e.g. following instructions). Typically this employs a three step process: 1) collecting demonstration data and performing supervised finetuning; 2) training a reward model on comparisons between model outputs; and 3) optimising the model as a policy against that reward model using an RL algorithm (PPO). The success of this approach has played an instrumental role in the recent rise of LLMs by providing a scalable and effective way to finetune models and control specific attributes.

Despite this, there have been a number of issues due to the cost of collecting human feedback data, and difficulty in optimising using PPO. One interesting development of late is Direct Preference Optimisation (DPO) <cit.> – which skips training the reward model and uses the LLM as an implicit reward model. This is essentially a KL-constrained contrastive loss that trains the model directly on preference datasets. Hejna et al. hejna2024contrastive point out that this method is not applicable to embodied trajectories and introduce a generalised version called “contrastive preference optimisation”, wherein a dis-preffered trajectory is contrasted to a preferred one.

These developments should enable us to train embodied agents built with multimodal language models (e.g. <cit.>) using preference optimisation and bring this aspect of feedback into embodied agents.



  §.§.§ Natural Language-Based Feedback


While it is relatively simple to use comparisons or scalar feedback scores, fully leveraging the richness of language feedback has proven more of a challenge.

Sumers et al. sumers2021learning investigate learning under the regime of “unconstrained linguistic feedback”. They categorise the feedback given as evaluative – scalar feedback similar to that described above, imperative – giving a preferred action e.g. “you should have carried over the tens term”, or descriptive – providing relevant information to the task. Sumers' approach to leveraging these rich rewards is to provide models that try to model the sentiment (a simple sentiment analysis model and a more complicated model handcrafted from Gricean pragmatics), as well as an inference network trained to predict the rewards given by the teacher. In their experiments, they firstly validate the importance of the more complex models that can leverage the richer language feedback, secondly they show that human teaching typically involves a mix of all three categories of feedback in human experiments.

More recently, several works have exploited the recent advances in NLP to use LLMs to directly update the reward functions used for training embodied models in accordance with the feedback. For example in Eureka <cit.> the authors train a robotic hand to execute complicated gestural object manipulation (spinning a pen around) by updating a reward function from observations. Similarly, Yu et al. yu2023language use LLMs to define reward parameters that are then optimised over for a variety of robot embodiments including a quadruped and arm based robot. They find that this approach is generalisable and achieves 90% success rates across a wide variety of tasks. The import of such approaches is that they enable the utilisation of LLM's high-level cultural knowledge about actions to be translated into low-level actions.

A promising alternative solution to this is to use the feedback to condition future actions of the learning agent, and rely on existing learning mechanisms to learn from these actions. Scheurer et al. scheurer2022learning use this paradigm to learn from natural language feedback by generating multiple responses conditioned on feedback. The response that is semantically most similar to the feedback is then treated as the “correct” response and used to finetune the model. Similarly, Chen et al. chen2024learning explore “Imitation learning from Language Feedback” in which they let the learner refine its answers based on human feedback, and then finetune on these refinements.

Zha et al. zha2023distilling further explore the mechanisms of memorising and retrieving feedback for conditioning embodied agent actions on the fly. They create a system for distilling corrections into retrievable knowledge that can be used by the LLM-agent in future generations. They use the feedback to firstly perform replanning, and then to create a correction that can be placed in a knowledge base (rephrased to be “contextually complete”) and retrieved in future situations where relevant. In particular, they use both visual and textual representations for the correction embedding so that it can be retrieved when a situation is similar in either dimension.



 §.§ Instruction and Explanation


Instruction following itself has been widely used as a task in the alignment of models. In particular, the RLHF framework described in Section <ref> was popularised in NLP for instruction following <cit.>. This alignment process (learning to follow instructions) has enabled foundation models to learn to perform complex tasks from instructions. A large body of works explore methods of instruction in the form of prompting techniques like chain of thought <cit.>, that have been shown to improve reasoning abilities. For example, Mu et al. mu2024embodiedgpt finetune their embodied model on complicated instructions that require the use of chain of thought planning over egocentric videos.

Instruction following has also seen use as a paradigm in the popular benchmarks TEACh <cit.> and ALFRED <cit.>. TEACh consists in human-human dialogue sessions where one human is playing the role of a teacher or commander; the other acts as a student and must follow the instructions of the teacher, while being able to ask questions for clarification. ALFRED is a relatively simple task and involves following singular grounded instructions like “walk to the coffee maker on the right”. The latter instruction can be seen as an example of the vision-language navigation task, which is currently one of the more popular paradigms in embodied AI, given that it was recently enabled by increased performance of LLMs. These benchmarks however do not focus on actually learning from the interactions with the instructors per se, nor indeed are the instructions targeted at the learning agent.

Saha et al. saha2024can explore more deeply modelling interactions between students and teachers using theory of mind. One of their research questions is the degree to which personalisations of explanations aid students. While they do find a positive effect, its influence is relatively small. While personalisation of explanations have not been found to be beneficial thus far, in an embodied RL setting it was found that the act of giving explanations itself was helpful in aiding the teacher learning <cit.>. This is an example of the so called protégé effect – when we try to explain or instruct others we test and expand our own understanding of a concept by reconceptualising it.

While it would be desirable to heavily use targeted instructions and explanations by asking for help when needed, the provision of help, especially in the case of humans in the loop, is expensive. Indeed per the theory of scaffolding, we expect that as the learner gains proficiency, the learning experience can be less structured, and intervention can be reduced over time. As such one body of works considers the problem of efficiency by learning when to ask for help. For example Ren et al. ren2023robots train an agent to ask for help when there is a high level of uncertainty in the task. The agent generates a set of possible actions, and if the likelihood of the likeliest of them being correct (as measured by the agent) is below a certain threshold, the robot then asks for further instruction.



 §.§ Curricula


Curriculum Learning is a machine learning approach that parallels the scaffolding of human learning at a more abstract level. The classic paradigm introduced by Bengio et al. bengio2009curriculum is to order the training data by complexity for some notion of complexity, and progressively increase the level of complexity as the learning progresses. Per Figure <ref>, curriculum learning consists in the teacher agent altering the learning environment rather than directly interacting with the learner agent.

The idea behind this is that by structuring the data in this way, the initial learning task becomes more feasible, and after basic concepts are acquired, these can be used as a basis for learning more complex concepts. While this seems simple, there are two key problems. First of all defining complexity for a given task is difficult – the curriculum may have to be tailored for individual learners, and require insight into the current capabilities of the learner. Additionally it may be unclear when the learner has sufficiently mastered a concept to enable the complexity to increase.

Accordingly a body of works exists that try to automate this process. This approach is typically called automatic curriculum learning, and is a popular paradigm in the reinforcement learning community (ACL). For instance, Matiisen et al. matiisen2019teacher explore a “Teacher Student Curriculum Learning” paradigm in which a teacher agent monitors the student's training progress and is used to determine the tasks suitable for the student. The mechanism used is the student's progress on a task – the teacher increases sample weight for tasks that are seeing progress – the weight is then reduced when the task has been mastered since the rate of progress is measurably decreased. The authors treat the problem as a bandit learning problem and explore a number of sampling algorithms, finding that such an automated curriculum can help for maze solving in minecraft, and simple addition tasks using LSTMs.

Other works such as Ramrakhya et al. ramrakhyal2023curriculum and Morad et al. morad2021embodied use estimations of task success probabilities to order the experience while training embodied navigation systems. In particular, they aim to find tasks of `intermediate difficulty'. Interestingly, in the former they note that the addition of easy tasks to the training mix without curriculum learning per se has a similarly beneficial effect, as the learning agent simply spends less time in easier episodes since it can achieve them quickly.

Despite their success in RL domains,curriculum-based approaches have not seen as much success in NLP or computer vision. Recently Warstadt et al. warstadt-etal-2023-findings ran a competition to train LLMs using a human-developmentally plausible dataset and found that although curriculums were the most popular approach explored by entrants, it was relatively ineffective for improving sample efficiency. Indeed the key negative result of their competition report was exactly that none of the common curriculum learning approaches offered substantial benefits to the developmental task of language learning. Given the differing success of curriculum learning approaches it is likely that it's benefit is largely dependent on how structured the learning problem is.

Indeed embodied language models may be a more suitable domain for the application of curriculum learning methods. Wang et al. wang2024voyager leverage the LLM's knowledge of task properties to derive new suitable tasks. Rather than relying on handcrafted curricula or designing a separate automatic curriculum, they simply prompt the LLM to come up with tasks itself, conditioning the new tasks on the state of the agent, previously learnt skills, the previous task completion history, and the long term goals of the agent. They find that this automatic curriculum is both critical for the agent's success, and furthermore beats out manually defined expert curricula. This process of giving the agent meta-learning abilities is promising but likely requires more principled analysis; in any case it speaks to the potential of using LLM-Agents to instantiate social learning paradigms in a unified way.



  
Section Summary: More principled work remains to be done on demonstration learning, such as investigating whether ToM can enable student agents to imitate more efficiently. Feedback has been widely used in the form of preference learning. Leveraging general language feedback has seen increasing use but requires more research. Approaches like conditioning on feedback likely have similar applications to leveraging explanations and instructions. Finally, while curriculum learning has been widely used in RL, it is not as promising for training LLMs – this may not however be the case for LLM-Agents.



§ COLLABORATIVE LEARNING



 §.§ Multi-Agent Reinforcement Learning (MARL)


Historically the majority of research on collaborative learning has occurred in the context of multi-agent reinforcement learning. For instance AlphaStar <cit.> was trained to grandmaster human level in the game StarCraft II through playing with other players. The system was initialised using imitation learning, and then further finetuned by an RL algorithm to perform well against a mixture of opponents. The solution proposed by the authors is an evolution of previous so called “self-play” strategies. This consists in pitting different versions of an agent against itself in strategy games. The key insight here was to firstly add a greater mix of adversarial agents (previous versions of the learning agent) in addition to agents that were specifically designed to exploit the flaws of the agent in question.

Recently, an AI agent (Cicero) was trained for the game Diplomacy at a human level, ranking in the top 10% of participants in an online league <cit.>. In Diplomacy, players take the role of diplomats controlling countries, and have to negotiate with other players, make promises, and deceive them. By leveraging pretrained LLMs to conduct the negotiation and model other users' intents, a strategic reasoning model could then be used to plan actions and intents and was then trained with a combination of RL methods including self-play and behavioural cloning.

In the more cooperative setting, a number of recent works have focused on the “learn to communicate” paradigm <cit.>. In these paradigms, the sets of agents may have to learn both what and when to communicate, and even some explorations of who to communicate to via attention mechanisms <cit.>. Many of these works have started from a point of zero linguistic competence and require the agents to learn from scratch. Accordingly a future trend will likely include the incorporation of existing language models <cit.>, similarly to the approach taken in the work of FAIR et al. meta2022human described above.



 §.§ Collaborative LLM-Agents


A significant recent trend in NLP has been exploring the use of communicative LLM-agents. Firstly, Park et al. park2023generative explored using such generative agents as models of human behaviour – in particular, they simulated a town of agents that perform daily activities and interact in natural language, enabled by a game environment that tracks the memories and positions of each agent. 

These behaviours, including emergent abilities at party planning, were rated as being largely believable as human by crowdworkers.

Later, Qian et al. qian2023communicative explored the creation of software using teams of LLMs mimicking the `waterfall' model of software development. The authors broke down the development process into a sequence of discussions between LLM agents acting in roles (e.g. CEO, programmer etc.). In each discussion the LLMs would act in turn as instructors and assistants, making decisions through concensus. In contrast to simply generating the codebase for a software development project directly using an LLM designed for coding, they managed to achieve an ∼88% success rate on achieving the software development task.

Chen et al. chen2024agentverse then went further to provide a more general framework for communicative LLM based agents. The process proceeds in rounds made up of 1) expert recruitment – building up a group of roles to perform, 2) decision making – deliberating over what to do 3) action execution, and 4) evaluation in which the outcome is compared to the goal and feedback is given.

In the embodied setting, Zhang et al. zhang2024building have explored building cooperative embodied LLM agents. This reportedly requires more complicated models and theory of mind - in particular their agents consist in modules handling belief, communication, reasoning, and planning. The belief module is intended to give the agents theory of mind by tracking the knowledge of the scene, other agents, the self, and the task. On benchmarks, the authors found that the LLM agents were able to cooperate well on various benchmarks, and that the theory of mind was necessary for this success.

Such cooperative LLM agents are already able to outperform state-of-the-art MARL agents on several tasks, such as navigation <cit.>, without the extensive number of training episodes typically required for the MARL agents. Importantly however, in all these collaborative settings, no learning per se is going on – the agents are able to achieve complicated tasks through collaboration but only “learn” by updating the memory or context used in their internal decision making processes (since they are frozen language models).


More generally, we expect that collaborative settings are more useful for generating experience trajectories that would be unachievable for singular agents which can then be leveraged for learning using ordinary learning mechanisms, rather than being a source for learning mechanisms per se, in a similar way to Scheurer et al. scheurer2022learning exploring conditioning on feedback as a way of learning from natural language feedback rather than providing a separate feedback learning mechanism per se.



  
Section Summary: Collaborative Learning has been explored mostly through Multi Agent Reinforcement Learning. These have included interesting studies on optimising the population of collaborators and emergent communication. In the case of LLM-Agents, there have been remarkable achievements through applying prompting, including application of ToM, but thus far limited work directly updating the weights of the models constituting these agents, limiting their adaptability.




§ CONCLUSIONS AND FUTURE DIRECTIONS

Although social learning is not typically conceived as a key paradigm or organising principle for machine learning, we have shown that it permeates a large proportion of the machine learning literature.


Moreover, we have seen how performant LLMs have enabled the application of more advanced social learning techniques using the preferred mode of human instruction – natural language. For example, in Section <ref> we showed that this has enabled the construction of models that can directly respond to language feedback, and in Section <ref> how they had been used to build communicative agents purely from language models that were able to outperform existing MARL approaches.



  
Comparison and Combination of Social Learning Paradigms.
As shown in Table <ref>, the majority of approaches explored in this paper use multiple social learning techniques. However, only four use all three of the main categories and are largely not embodied. As we have argued, equipping agents with linguistic capabilities has enabled agents to leverage communication and instructive learning paradigms out of the box. Given this, we expect to see more work exploring the combination of these approaches.



  
Learning Permanently through Social Conditioning.
The promising LLM-Agent approaches have thus far largely relied on the knowledge acquired by the LLM through imitation learning. There has been little work performing further gradient updates once these models are socially embodied, and the “learning” has largely constituted updates to the agent's memory modules (as in <cit.>). Accordingly, identifying how models can learn more permanently in these settings remains a research priority. We see a promising direction in utilising low-level learning methods on experiences conditioned on the social interactions that entail “higher-level” learning mechanisms like feedback and instruction. This has seen some application in the non-embodied setting <cit.>, but we expect this to be replicated for embodied settings in the near future as we move beyond simply using LLM-agents without finetuning.


  
Multimodality.
Our survey has been primarily concerned with linguistic approaches, however the recent development of multimodal models will likely enable richer forms of social interaction (and thereby learning). We have already seen this in the case of demonstration learning from vision and language <cit.>, but given the relative immaturity of video-foundation models it may be some time before we are able to fully leverage non-linguistic signals like gestures and visual demonstrations for teaching embodied agents. As discussed in Section <ref>, existing techniques for learning from feedback could be applied to learn from these signals.


  
Theory of Mind and Intent.
As can be seen in Table <ref>, there have been a number of approaches exploring the use of Theory of Mind as a prerequisite for richer interactions between teachers, students, and peers. Indeed Theory of Mind cuts across social learning. Modelling the intent of a teacher enables a learner agent to fully leverage the learning signal, as we discuss in Section <ref>. Modelling teacher agents also allows learners to evaluate the teacher's potential as sources of learning <cit.>. While ToM modules are a relatively common aspect of agents, especially collaborative agents, we have seen little work exploring these techniques so far. As such, we expect to see more work investigating the effects of these modules, especially given the interest in ToM initiated by Kosinski kosinski2023theory.



  
Application Domains.
Social learning techniques are broadly applicable in the field at large. Nevertheless, they may play an especially important role in domains where social skills themselves are more relevant. For instance, in the case of socially-assistive robotics, the ability to adapt to individual needs and build a relationship over time is especially important <cit.>. Similarly, the ability to use models to teach humans skills would be a significant benefit of developing instructive social learning techniques. For example, this has explored in the case of social skill training <cit.>. In the meantime, we expect the majority of progress to occur in using social-learning techniques for training agents in video games, due to the low cost of training, ample amounts of training data, and ease of interacting with humans. Games are already a test-bed for many approaches covered above <cit.>, but we can expect social learning to enable richer forms of collaborative play with AI agents, as explored for example in the work of Gong et al. gong2023mindagent.




§ ACKNOWLEDGMENTS

We would like to thank Chloe He for her invaluable suggestions. This work was supported by an A*STAR CRF award to C.T., as well as an A*STAR SINGA scholarship to D.H.


named




 Paper Title: Dual-Constrained Dynamical Neural ODEs for Ambiguity-aware Continuous Emotion Prediction.



On the Perturbed States for Transformed Input-robust Reinforcement Learning
    Tung M. Luu^∗, Haeyong Kang^∗, Tri Ton, Thanh Nguyen, and Chang D. Yoo
    August 5, 2024
===========================================================================




§ ABSTRACT 
    There has been a significant focus on modelling emotion ambiguity in recent years, with advancements made in representing emotions as distributions to capture ambiguity. However, there has been comparatively less effort devoted to the consideration of temporal dependencies in emotion distributions which encodes ambiguity in perceived emotions that evolve smoothly over time. Recognizing the benefits of using constrained dynamical neural ordinary differential equations (CD-NODE) to model time series as dynamic processes, we propose an ambiguity-aware dual-constrained Neural ODE approach to model the dynamics of emotion distributions on arousal and valence. In our approach, we utilize ODEs parameterised by neural networks to estimate the distribution parameters, and we integrate additional constraints to restrict the range of the system outputs to ensure the validity of predicted distributions. We evaluated our proposed system on the publicly available RECOLA dataset and observed very promising performance across a range of evaluation metrics.








§ INTRODUCTION

Speech emotion recognition plays an important role in building the next generation of human-machine interactions <cit.>. Dimensional representation, describing emotions with continuous values on affect dimensions (e.g., arousal and valence), allows for broader descriptions of the complexity and richness of emotion, making it extensively utilised in the speech emotion recognition research community <cit.>. 
Typically, the continuous emotion labels (also denoted as ratings) are collected from a group of human raters by asking for their perceptual evaluations of the same speech or video recordings. This is depicted in the top plot of Figure <ref>, where dashed lines represent the various arousal ratings collected from 6 raters. The differences in perception, referred to as inter-rater ambiguity <cit.>, reflect the subtlety of natural emotions, and this ambiguity should be taken into consideration when developing emotion recognition systems.






Until recently, researchers in affective computing have often treated this ambiguity as noise and used the average ratings from multiple raters as a `gold standard' <cit.>. However, there is now growing recognition that inter-rater ambiguity in emotion labels conveying information about the emotion state and such ambiguity is being taken into account by treating arousal/valence labels as distributions  <cit.>. 
This paradigm shift has spurred the development of ambiguity-aware emotion prediction systems that predict emotion states as distributions on the arousal/valence dimensions, rather than conventional mean predictions <cit.>.


Additionally, emotions are dynamic processes that evolve continuously over time <cit.>. This dynamic nature applies not only to the emotional state itself but also to the associated ambiguity <cit.>. Figure <ref> illustrates this idea of emotion dynamics with arousal ratings (coloured dashed lines on the top plot) from each rater gradually changing over time. The associated emotion ambiguity (grey shaded area) also follows a smooth evolving process leading to the smoothly varying distributions (bottom plot) that encode the underlying ambiguity. Nevertheless, among the limited works that model emotion ambiguity, only a small subset of these systems have attempted to incorporate the temporal dependencies amongst the distributions.








In this paper, we present a speech-based ambiguity-aware emotion prediction system that explicitly incorporates the temporal dependencies of emotion distributions. Leveraging the advantages of the recently proposed constrained dynamical neural ordinary differential equation (CD-NODE) for modelling time series with the ability to impose constraints on the smoothness of dynamic process, and its effectiveness in modelling time-varying emotion ratings <cit.>, we develop a novel ambiguity-aware dual constrained CD-NODE system (denoted as CD-NODE_γ).
This system predicts ambiguity-aware emotion states with time-varying distributions along the arousal/valence dimensions, and automatically learns how these emotion distributions evolve over time. Moreover, CD-NODE_γ integrates smoothness and range constraints to manage the rate of change and the absolute scope of emotion distributions, respectively, further ensuring the validity of the distributions.


















§ RELATED WORK





Recent years have witnessed an increasing amount of effort on modelling and predicting the ambiguity associated with emotion labels, often treated as probability distributions <cit.>. However, the majority of existing research primarily addresses the ambiguity in predictions while neglecting the temporal dependencies, treating distributions at each time step as independent. Among the few studies addressing temporal dependencies in emotion distributions, initial methods modelled emotion labels using Gaussian Mixture Models (GMMs) without considering temporal dynamics <cit.>. Subsequent improvements incorporated Kalman filters to account for the temporal evolution of GMM parameters <cit.>. However, this approach assumes that the temporal dynamics follow a linear dynamical system. Zhang et al. proposed using Bidirectional LSTM to estimate the means and standard deviations of Gaussian-distributed labels <cit.>, but the poor prediction in standard deviations suggests the unsuitability of the predefined distribution types. While LSTM learns temporal dependencies in the parameters, it lacks transparency and flexibility in integrating constraints or prior knowledge on emotion dynamics. More recent efforts, such as a system utilizing a Sequential Monte Carlo approach, explored predicting time-varying emotion distributions with a non-linear dynamical system, yet focused solely on the dynamics of distribution means <cit.>. These approaches either operate in separate stages rather than end-to-end, apply simplistic linear constraints or make specific and inappropriate assumptions about distribution dynamics, or lack the flexibility in incorporating constraints. On the other hand, the CD-NODE system captures the temporal dynamics of emotions without imposing predefined assumptions on their evolution, facilitating end-to-end modelling and allowing for incorporating constraints, which has shown superior performance in continuous emotion prediction <cit.>. However, it only predicts mean arousal/valence ratings and neglects the associated ambiguity.























§ OVERVIEW OF CD-NODE






The recently published CD-NODE was developed to model time series with additional constraints on the dynamics of the predicted quantity. 
It models the continuous emotion labels (i.e., arousal/valence ratings) y(t) = {y_t_0, y_t_1,⋯, y_t_N} as a non-linear dynamical process using ODE with governing function f parameterized by neural networks θ, and the dynamics in y(t) is dependent on both the input speech features x(t) = {x_t_0, x_t_1,⋯, x_t_N} and previous y. It is defined as: 



    dy(t)/dt = ϕ(f(x(t),y(t); θ)))

The function ϕ(·) is an additional constraint to limit the rate of change of y(t), which controls the  smoothness of y(t) as:



    ϕ(·) = α * tanh(1/α)

where α is a hyperparameter that can be tuned based on the characteristics of y(t). A smaller α results in a smoother trajectory of y(t) and vice versa. The governing function f can be parameterised by any network structure that approximates the nonlinear function in ODE. The final trajectory of y(t) is obtained by solving an ODE initial value problem as shown in Equation (<ref>). 



    y(t) = ODESolve(f,θ,y_t_0, t_0, ⋯, t_N, x(t))







§ PROPOSED CD-NODE_Γ



While the original CD-NODE based system only predicts the mean arousal/valence rating, this paper proposes an ambiguity-aware dual-constrained Neural ODE, referred to as, CD-NODE_γ, that predicts ambiguous arousal/valence states as time-varying Beta distributions as suggested in <cit.> that Beta distributions are highly suitable for modelling emotion ambiguity. Furthermore, in addition to the explicit smoothness constraint which was a key feature of the CD-NODE, we introduce a second constraint γ, which restricts the system output to fall within a given range, which in turn helps ensure the validity of the predicted distributions. 





 §.§ Model Structure

Assuming the arousal/valence at time t_n is represented by a beta distribution that is parameterised by ψ_t_n, the aim of the emotion prediction system is to predict ψ(t) = {ψ_t_0, ψ_t_1,⋯, ψ_t_N} given a sequennce of input speech features x(t) = {x_t_0, x_t_1,⋯, x_t_N}. The most likely emotion intensity and the associated ambiguity can then be inferred from the predicted beta distribution, ψ_t_n, at each time step.

The Beta distribution is typically parameterised using two parameters, 
denoted as ψ_t_n = {ψ^1_t_n,ψ^2_t_n}. This then leads to a multi-task learning framework to model {ψ^1_t_n,ψ^2_t_n}:




    dψ^i(t)/dt = ϕ_i(f_i(𝐱(t),ψ^i(t); θ^i)), i ∈ [1,2]

where i ∈ [1,2] corresponds to the two parameters,  f_i are the two governing functions modelled as neural networks (blue blocks in Figure <ref>) with the aim of learning to predict dψ^i(t)/dt. They are each constrained with their own smoothness constraint function ϕ_i (green blocks in Figure <ref>).

Noting that the parameters of any beta distributions will have constraints to ensure a resulting distribution is valid (for e.g., when using the mode and concentration parameterisation of a beta distribution, the concentration must be higher than 2 to ensure a bell shaped distribution), we impose an additional constraint function γ(·) to limit the range of ψ^1_t_n and ψ^2_t_n.

Thus, the final outputs are obtained as per equation (<ref>).
 
    ψ^i(t) = γ_i (ODESolve(f_i,θ^i,ψ^i_t_0, t_0, ⋯, t_N, x(t)))


Finally, the loss function for the proposed multi-task learning framework is taken as: 



    L = λ_1 L_1 + λ_2 L_2


    L_i= 1 - ρ_c^i(ψ^i(t), ψ^i(t)), i ∈ [1,2]


where λ_i are scaling factors; and L_i are loss functions for ψ^i(t), dependent on the concordance correlation coefficient (CCC) ρ_c^i between the predicted ψ^i(t) and the ground truth ψ^i(t) <cit.>. The model aims to minimize L_i, i.e., maximizing the CCC.




 §.§ Constraint Function: γ


The initial stage of our approach involves identifying the most suitable parameter pairs for the Beta distribution in our proposed CD-NONE_γ. This is a crucial step in ensuring that our model can effectively learn the distributional properties of emotions. The most commonly used parameter pairs include: shape parameters {a, b}, and {mean(μ), variance(σ)} and {mode(w),concentration(k)} which indicate the central tendency (μ or w) and spread (σ^2 or k) of the distribution. We observed that shape parameters {a, b} are less interpretable compared to {μ, σ} and {w,k}. Furthermore, when comparing {μ, σ} to {w,k}, we observe that both μ and σ are both bounded to the interval [0,1], while only w is bound to a finite interval with the only bound on k being k > 2. The lack of an upper bound to k leads to dramatic changes in its dynamics which makes a CD-NODE_γ that predicts {w,k} harder to train. Therefore, we chose {μ, σ} to represent the Beta distributions. Consequently, we design the constraint functions γ_i as in equation (<ref>) for both μ and σ as:

 


    
    γ_μ = pS(·), γ_σ = qS(·)

    

    0 < p ≤ 1, 0 < q^2 ≤ min(p^2 1 - p/1+p, (1-p)^2 p/2-p)



where, S(·) denotes a sigmoid function and is used to constrain the output to [0,1], and p and q are adjustable constants that must satisfy the relationship as per equation (<ref>) in order to ensure the Beta distribution is bell shaped <cit.>.



    











§ EXPERIMENTAL SETTINGS




 §.§ Dataset

The RECOLA dataset is one of the most commonly used multimodal corpus which consists of 9.5 hours spontaneous dyadic conversation recordings in French <cit.>. There are 9 five-minute utterances each in the training and development sets, identical to the data partition in the AVEC challenge 2015 <cit.>. Since the challenge test sets are not publicly available, the system is trained and the hyperparameters are optimized using the training set and tested on the development set as per standard practice <cit.>. The dataset is annotated by 6 human raters with continuous arousal and valence labels within range [-1,1], with sampling rate of 40ms. 





 §.§ Beta distribution parameter estimation

To obtain the ground truth beta distribution, we follow the same approach used in <cit.> to estimate the Beta distribution parameters. The original ratings are first mapped to range x ∈ [0,1] via a linear transformation y = 0.4975x + 0.5 in order to perform the Beta fit. To compute the Beta parameter at each frame, the ratings from neighbouring F frames are concatenated as y = [y_n-F:y_n+F], y_n refers to the labels from 6 raters at frame n to avoid overfitting. F is 6 and 1 for arousal and valence respectively as suggested in <cit.>. Finally, the Beta parameters ψ_t_n are obtained by Maximum A Posterior (MAP) as:



    ψ_t_n = _ψ∏_m P(y_n,m|ψ)P(ψ)


where y_n,m denotes the rating from the m^th rater at frame n; p(y_n,m|ψ) refers to the posterior probability of y_n,m; and P(ψ) refers to the prior probability which is estimated with kernel density estimation, the same as in <cit.>.





 §.§ System configurations



Features. The Bag-of-audio-words(BoAW) feature representations are extracted with 100 clusters from 20-dimensional MFCCs using OpenXbow <cit.>. Detailed explanation refers to <cit.>. 4 second delay compensation is applied to both arousal and valence to compensate the annotation delay <cit.>. 

Network parameters. 
A fully connected (FC) layer with 64 neurons and tanh activation function is first employed to transfer the BoAW features. The following f_1 (i.e., f_μ) and f_2(i.e., f_σ) for modelling dynamics of μ and σ are composed of three FC layers respectively, with tanh activation function applied for the first two layers to enforce the nonlinearity, and the smoothing constraint applied to the last FC layer (c.f. Equation (<ref>)), with α_u=0.5 <cit.> and α_σ=10 which is optimized within {0.5, 1, 2, 4, 6, 8, 10, 15} for both arousal and valence. For the proposed constraints in Equations (<ref>), we chose p = 0.75 for both arousal and valence 
as 95% of the ratings are within the range [0,0.75], and q = 0.15 satisfying the requirement in Equation (<ref>). The initial values ψ^i_t_0 in Equation (<ref>) are the Beta distribution parameters inferred from the labels of the first frame during the training phase, and set to zeros during the test phase given emotions at the beginning of any utterances tend to be zeros.


We used the adjoint method with Runge-Kutta 5 ODE solver <cit.>. The absolute and relative error tolerances of the ODE solver were 
10^-13 and 10^-7 based on preliminary empirical analyses <cit.>. The number of model parameters is approximately 30k[Code: https://github.com/JingyaoWU66/CD-NODE-gamma].

Baseline. 
 Two state-of-art systems are compared: the Sequential Monte Carlo (SMC) approach <cit.> and the LSTM based system <cit.>. The LSTM system follows the same settings as in <cit.> which directly predicts Beta distribution parameters. In our experiments, the sigmoid activation function is applied at the output layer to limit the output range for a fair comparison. 
The SMC approach predicts distributions that are not constrained to Beta distributions and we employ the same settings as in <cit.>.

Training. For all systems, Adam optimizer was used, and the initial learning rate is optimized to 0.01 and 0.001 for f_μ and f_σ, with a decaying ratio of 0.9. 60 and 30 epochs were tested for CD-NODE and LSTM baseline. We fixed the seed numbers to ensure reproducibility when testing different models.

The training utterances were chunked into 4-second windows (i.e., 100 frames) in each batch for training efficiency, while it was tested with the entire utterance to match the practical scenarios. The scaling factors in the loss function in Equation (<ref>) were optimized within the range [0,20] with step size 1 and λ_1 = 1 and λ_2 = 10 were selected. The predictions were post-processed with a moving average filter with 12 frames( 0.5s) window size. The training time is approximately 72h on Apple M2 chip with 8 cores on 16GB CPU.

Evaluation Metric. It is suggested in <cit.> that at low ambiguity regions, a well-predicted distribution should be centred at the ground truth mean; whereas at high ambiguity regions, the mean is not representative of the underlying emotions, and thus, the spread of the distributions is more important. We evaluate the first aspect by comparing root mean squared error (RMSE) between the predicted mean and ground truth mean at different ambiguity regions <cit.>; and the second aspect using Concordance Correlation Coefficient (CCC) between the predicted standard deviation (SD) and ground truth SD. 










    ρ_c = 2 ρσ_y σ_ŷ/σ_y^2 + σ_ŷ^2 + (μ_y - μ_ŷ)^2



where σ _y and σ _ŷ refers to the standard deviation for the ground truth and the predicted quantities, μ _y and μ _ŷ are the corresponding mean of the two variables. ρ is the Pearson's correlation coefficient between the two variables.








§ RESULTS




 §.§ Comparisons with state-of-the-art




The comparison to the state-of-the-art systems is reported in Table <ref>, in terms of CCC between the predicted Beta distribution (i.e., mean μ and standard deviation σ) and the ground truth Beta distributions (refer to section <ref>) correspondingly. In addition to <cit.> that uses LSTM to predict w and k of beta distribution (referred to as LSTM-w-k), we further validate the system performance for predicting μ and σ for a fair comparison, named as LSTM-μ-σ. The proposed system is also compared to the existing SMC model which adopts non-parametric distribution <cit.>. It is observed that the proposed model CD-NODE_γ 
yields the highest CCC in terms of mean predictions μ for both arousal and valence, with significance p-value < 0.001. Regarding the SD prediction, CD-NODE_γ shows comparable performance to the best system LSTM-μ-σ for arousal, and underperforms the best system SMC for valence. This could be because the dynamics of SD change more rapidly compared to the mean, which may require a more advanced ODE solver to obtain a more accurate solution.




Figure <ref> presents the RMSE values between the predicted and ground truth means across different ambiguity regions, defined based on deciles of the ground truth standard deviation. As expected, the RMSE generally increases at higher standard deviation deciles for both arousal and valence, indicating that mean prediction errors increase with rising ambiguity (low inter-rater agreement). Conversely, the systems perform better at low ambiguity regions (high inter-rater agreement), which supports our hypothesis. Notably, our proposed system, CD-NODE_γ, consistently achieves the lowest RMSE compared to all baseline systems for both arousal and valence. 
These results demonstrates the superior performance of CD-NODEγ in modelling time-varying emotion distributions, as it accurately predicts the emotion state at low ambiguity regions while still reasonably predicting the level of ambiguity at both low and high ambiguity regions. Although CD-NODE_γ and LSTM-μ-σ achieved similar performance on valence, it may be attributed to the inherent difficulty in predicting valence using speech data.









 §.§ Impact of range constraint




We further validate the effectiveness of the constraint functions in the CD-NODE_γ, by comparing the system performance without constraint (D-NODE), with only the smoothness constraint (CD-NODE), and with both the smoothness and range constraints (CD-NODE_γ). The performance in terms of CCC in Table <ref> shows that CD-NODE_γ achieves the highest CCC and D-NODE without constraint shows the lowest CCC, indicating the advantages of adding constraints. More importantly, CD-NODE_γ yields 3.6% and 6.5% relative improvements over CD-NODE for mean and SD of arousal,  and 3.7% and 26.6% relative improvements for mean and SD of valence. This further confirms the effectiveness of the proposed range constraint γ.

RMSE is also reported in Figure <ref> for the three systems. All three systems show an increasing trend of RMSE with increasing SD decibels, aligning with the assumption that prediction error is larger where inter-rater ambiguity is high. Notably, CD-NODE_γ consistently shows a smaller RMSE compared to the other two systems, e.g., 6.3% and 3.3% relative decreases at lowest ambiguity decile on arousal compared to D-NODE and CD-NODE respectively, further validating the benefits of proposed constraints. Overall, the results suggest that CD-NODE_γ with the range constraint can capture the time-varying dynamics of emotion distributions more effectively, which might be because the constraint function eliminates the noisy predictions outside the possible range of the predicted quantity.


    











§ CONCLUSION


In this paper, a multi-task learning framework is used to develop an ambiguity-aware dual-constrained dynamic neural ODE (CD-NODE_γ) system that models ambiguous emotional states as time-varying Beta distributions. Our proposed method takes into account the temporal dynamics of distributions and incorporates a range constraint that restricts the outputs to better ensure the validity of predicted emotion distribution, in addition to a smoothness constraint. Experimental results, when compared to various baselines, show that the proposed CD-NODE_γ model outperforms them in modelling emotional states and produces accurate predictions in low ambiguity regions and reliable predictions of high ambiguity regions. Moreover, the outcomes acquired with various constraint configurations indicate that the inclusion of a range constraint enhances the CD-NODE's ability to learn the distribution dynamics. The results highlight the benefits of utilising CD-NODE_γ to model temporal dynamics of distributions with explicit constraints, which opens up new avenues for emotion ambiguity modelling.












IEEEtran



 Paper Title: Barlow Twins Deep Neural Network for Advanced 1D Drug-Target Interaction Prediction.



Information flow-enhanced precision in collisional quantum thermometry   
    Mauro Paternostro
    August 5, 2024
=========================================================================



§ ABSTRACT 
Accurate prediction of DTI is critical for advancing drug discovery.
By reducing time and cost, ML and DL can accelerate this discovery process.
Our approach utilises the powerful Barlow Twins architecture for feature-extraction while considering the structure of the target protein, achieving state-of-the-art predictive performance against multiple established benchmarks.
The use of GBM as the underlying predictor ensures fast and efficient predictions without the need for large computational resources.
In addition, we further benchmarked new baselines against existing methods.
Together, these innovations improve the efficiency and effectiveness of DTI predictions, providing robust tools for accelerating drug development and deepening the understanding of molecular interactions.






§ INTRODUCTION


Studying DTI is crucial for understanding the biochemical mechanisms that govern how molecules interact with proteins.<cit.>
Key challenges in drug discovery are the identification of proteins that can be used as targets for the treatment of diseases.<cit.>
To achieve the desired therapeutic effects, the discovery of molecules that interact with and activate or inhibit target proteins is essential.<cit.>

Recent advances in computational methods have transformed the drug discovery landscape, providing robust tools for cost-effective exploration of the chemical space.
These in silico approaches facilitate the prediction and analysis of DTI, aiding in the identification of potential drug candidates and their corresponding protein targets.<cit.>
The use of computational techniques allows researchers to gain a comprehensive understanding of the molecular mechanisms underlying DTI, thereby accelerating the drug discovery process and minimising reliance on traditional, resource-intensive experimental methods.<cit.>
Different methods have been used to understand how drugs interact with target proteins. 
These methods are grouped into three main categories: structure-agnostic, structure-based and complex-based.

Structure-agnostic approaches use 1D inputs like molecule SMILES and protein amino acid sequences, or 2D inputs like graphs and predicted contact maps.<cit.>
These methods are cost-effective and and sufficiently accurate compared to experimental or in silico structure prediction,<cit.> as they are independent of the protein's structure when predicting effects. 

Structure-based approaches require 3D protein structures and 1D or 2D molecular inputs.

3D structures are usually derived from experimental data, although computational predictions are increasingly employed.<cit.>
These methods have great potential but can be unreliable. 
They depend on accurate 3D protein structures and may be limited in their ability to generalise beyond experimentally observed DTI.<cit.>
Due to the complexity of the experimental setup, 3D protein structures can be difficult to obtain. 

In addition, models often overlook the fact that proteins are not rigid structures, but are generally in motion, e.g., ligand binding induces a conformational change.<cit.>

Finally, complex-based approaches require protein-ligand co-complex structures, which additionally require 3D information, as well as protein interaction information about the ligand.<cit.>
For this reason, complex-based approaches can provide a more detailed insight into the interactions, but they are by far the most difficult to obtain data for.

Considering these different approaches, we designed as a fully data-driven, sequence-based approach that relies on SMILES and amino acid sequences as the most accessible data, avoiding costly and time-consuming experimental data such as crystal structures.
Additionally, we use a specialised bilingual PLM to embed the 1D amino acid sequence, which uses a 3D-alignment method that results in a structure-sequence representation.<cit.>
This approach makes input data structure-agnostic, yet benefits from structure-sequence PLM embeddings.
Unlike most other methods, we have developed a system that uses a hybrid best of both worlds ML and DL approach to improve DTI prediction performance in low data regimes where training data is limited.<cit.>
We have found that DL architectures such as Barlow Twins<cit.> are excellent at learning features<cit.> that can then be used for GBM training to achieve state-of-the-art performance, as the size of datasets is usually too small to reliably train a DL model that will perform competitively.





§ RESULTS AND DISCUSSION




  
design
We propose a novel method for predicting DTI using SMILES notations, primary amino acid sequences and annotated interaction properties.
relies on a three key components, visualised in <ref>:


    
  * Firstly, the input needs to be vectorised. This is achieved by converting SMILES to an ECFP.
    The amino acid sequences are processed by a PLM that uses both modalities, combining 1D protein sequences and 3D protein structure.<cit.>
    
  * Secondly, we teach the SSL based Barlow Twins model interaction of molecule and protein without considering labels.<cit.>
    The objective function implements invariance of both representation of one interaction while ensuring non-redundancy of the features.<cit.>
    
  * Finally, takes a combination of embeddings generated by the encoders from the Barlow Twins DL model and uses them as features to train a GBM based on the interaction annotations.<cit.>
    This approach exploits two key strengths: it uses DL to refine representations, and it leverages the power of ML in scenarios with limited data. 
    This is particularly relevant for DTI datasets, where only around 50000 annotated pairs are publicly available.<cit.>





  
Benchmark selection
We select a comprehensive set of literature-based benchmarks to evaluate the performance of against several leading methods.
The benchmarks considered in this study are derived from several key sources. 
These sources include biomedical networks,<cit.> the US patent database,<cit.> and data detailing the interactions of 72 kinase inhibitors with 442 kinases, representing over 80 of the human catalytic protein kinome.<cit.> 
These datasets provide DTI as pairs of molecules and amino acid sequences, each coupled to an interaction annotation.

To ensure a fair comparison, is retrained across all benchmarks. 
Finally, we evaluate performance in a binary classification setting:


    
  * We compare with a total of seven established models: the model by , MolTrans,<cit.> DLM-DTI,<cit.> ConPLex,<cit.> DrugBAN,<cit.> PSICHIC,<cit.> and STAMP-DTI.<cit.>
    Overall, various structure-agnostic, structure-based and complex-based methods have demonstrated state-of-the-art performance in benchmarks.
    
  * This comparison is performed on a total of four datasets with twelve literature-proposed splits: 4 × BioSNAP,<cit.> 4 × BindingDB,<cit.> 1 × DAVIS<cit.> and 3 × Human.<cit.>
    Our aim is to investigate the behaviour of different methods in diverse splitting scenarios. 
    These splits help us to assess how well models generalise under challenging evaluation conditions, for example where either the drug or the target has not been seen before, thus providing insight into their real-world applicability.
    
  * In addition, we investigate the addition of a more rigorous model baseline. The GBM XGBoost is known to be one of the best models, e.g. in QSAR tasks, often outperforming DL-based approaches.<cit.>




  
shows state-of-the-art performance in predicting DTI
We evaluate the performance of in binary classification on four datasets using different splitting procedures. 
Overall, significantly outperforms all other models in <ref>.
Looking at BioSNAP, we improve >5 over the leading method DLM-DTI in terms of PR_AUC.
Furthermore, as shown in <ref> again outperforms the PSICHIC method with a >5 PR_AUC improvement independent of the split.



When switching to BindingDB, significantly outperforms DLM-DTI in terms of PR_AUC with a >12 improvement (<ref>).
Investigating the BindingDB splits shows that outperforms all existing methods when looking at unseen ligands, almost matches the performance of DrugBAN in the random setting and becomes third best in the unseen protein split (<ref>).
Overall, is performs best in two out of four splits in this benchmark.

once again outperforms all of the established approaches when looking at the DAVIS benchmark, with a >18 improvement over the leading ConPLex model in terms of PR_AUC (<ref>).

Lastly, we evaluate the performance on the Human benchmark. 
shows the best performance when looking at the unseen protein split as well as the random split (<Ref>).
PSICHIC comes first in the unseen ligand setting, when looking at ROC_AUC, while DrugBAN is best in PR_AUC.
In summary, outperforms all other models in two out of three splits.



To look at why outperforms  other methods in various benchmarks, we looked into the architecture and its components in an ablation study.





  
Unravelling the performance contributions of the architecture
To investigate the impact of each element of the architecture, we remove them one at a time.
We do this across all baselines and splits with the following ablations:


    
  * We remove the hyperparameter optimisation step of the classifier.  
    
  * From the first removal, we replace the Barlow Twins architecture entirely and instead concatenate ECFP and PLM embeddings for training. We keep the hyperparameter optimisation procedure as in .
    
  * Finally, we remove the hyperparameter optimisation procedure from the previous ablation, analogous to the first modification. 


We observe a significant decline in performance, as illustrated in <ref>a and <ref> for the initial ablation, emphasising the crucial role of hyperparameter optimisation for achieving optimal model performance. 

The second ablation also indicates a significant reduction in performance. 
This is likely attributed to the DL architecture based on the SSL Barlow Twins model, which effectively learns embeddings to describe DTI. 
The Barlow Twins objective promotes orthogonality between drug and target modalities while ensuring the non-redundancy of both, thus preventing informational collapse. 
As a result, this leads to an overall state-of-the-art predictive performance.

The final ablation shows a further decline in performance, consistent with the results of the initial ablation experiment. 

In summary, the sustained reduction in performance of our ablation experiments demonstrates that each component of our pipeline is needed to maximise performance. 
This architecture integrates the best of both worlds: DL and GBM to enhance predictive performance.
Compared to other pure ML- or DL-based approaches, we can demonstrate a performance boost.
In particular, the use of the state-of-the-art PLM<cit.> could offer an advantage over other methods.
Other PLM variants are ProtTrans T5<cit.> in ConPLex<cit.> and ProtBERT proposed by  also used in DLM-DTI.<cit.>
The structural awareness of added by the inclusion of 3D-alignment in ProstT5<cit.> hints towards better generalisation capabilities.



    Choosing baseline models

Selecting an appropriate baseline model is critical to effectively comparing different ML and DL techniques. Robust baselines allow meaningful comparisons to be made and highlight the improvements made by new methods. 
Without appropriate baselines, it becomes difficult to determine whether new approaches are truly advancing the field.

Current leading DTI models predominantly use DL methods and are often evaluated against simple baseline models such as logistic regression, Ridge or DNN classifiers.<cit.> 
To improve the benchmarking process, we propose to add GBM as a baseline for DTI benchmarking purposes, as shown in the final ablation configuration. 
GBM such as XGBoost have demonstrated broad adaptability, e.g. in QSAR modelling, offering strong predictive performance and fast training times, particularly in scenarios with limited data availability, such as DTI prediction.

We compare the overall model performance across all datasets in <ref>b and <ref>.
Here, the performance of XGBoost trained on ECFP and PLM embeddings is highlighted as it shows competitive performance across all methods and datasets. 



§ CONCLUSIONS


Our proposed method, , integrates sequence information with the Barlow Twins SSL architecture and GBM models, representing a powerful fusion of ML and DL techniques.

This approach demonstrates state-of-the-art DTI prediction capabilities, validated across multiple benchmarks and data splits. 
Notably, our method outperforms existing literature benchmarks in ten out of twelve datasets evaluated.

To elucidate the efficacy of , we conducted an ablation study to investigate the contribution of its core components and their impact on performance. 
In addition, we re-evaluated the choice of baselines in numerous publications and advocate the inclusion of GBM baselines.

Given its exceptional performance, we are confident that can significantly accelerate the drug discovery process and offer significant time and cost savings through the use of virtual screening campaigns.



§ METHODS




 §.§ Datasets


To evaluate the performance of , three established benchmarks are used.
They all provide fixed splits for training, evaluation and testing.
In some publications the training and evaluation is merged to improve predictive performance.
To endure comparability, this was not done in this work.
All metrics listed from other publications are also listed where only the training set is used.

In addition,  first proposed splits for large DTI datasets, BioSNAP,<cit.> BindingDB<cit.> and DAVIS.<cit.>

The addition of a variety of splits with an additional benchmark Human<cit.> (based on DAVIS) are proposed by , we evaluate these separately.<cit.>

For all datasets, to reduce bias and improve model performance, the SMILES are cleaned using the Python ChEMBL curation pipeline.<cit.>
All duplicate and erroneous molecule and protein information that could not be parsed is removed.
Training is performed on the predefined training splits.



 §.§ Representations




  
Molecular information
The SMILES are converted into ECFP using RDKit.<cit.>
We used them with 1024 and a radius of 2.



  
Amino acid sequence information
The amino acid sequences are converted into vectors, by using the PLM ProstT5.<cit.>



 §.§ Barlow Twins model configuration


The proposed method is based on the Barlow Twins<cit.> network architecture, which employs one encoder for each modality and a unified projector.
The encoders and projector are MLP based.
The loss function is adapted from the original Barlow Twins publication and enforces cross-correlation between the projections of the modalities.<cit.>

The architecture is coded in Python using PyTorch.<cit.>



  
Pre-training Barlow Twins 
Here we pre-train the Barlow Twins architecture on our joint DTI dataset, based on BioSNAP, BindingDB, DAVIS and DrugBank,<cit.> removing duplicates and without labels to teach DTI. 
Early stopping is implemented to avoid overfitting, which is carried out using a 15 validation split.



    Hyperparameter optimisation


Manual hyperparameter optimisation is performed, shown in <ref>.





  
Feature-extractor
When performing feature-extraction, we use the pre-trained model.
For training and prediction, we extract the embeddings after the encoders for each modality and concatenate them.
Finally, a GBM, XGBoost<cit.> Python implementation, is trained on the embeddings in combination with the labels for each training sets respectively.



    Hyperparameter optimisation

If a benchmark provides a dedicated validation set, this was used for Optuna<cit.> hyperparameter optimisation.
The optimisation was carried out for 100 trials with the parameters shown in <ref>.






 §.§ Baseline model configuration


As a baseline, we have selected a GBM.
Similar to our feature-extraction implementation, for all features we concatenate both ECFP and PLM embeddings.
Finally, a GBM, XGBoost Python implementation, is trained on the ECFP and PLM embedding concatenation in combination with the labels for each training set, respectively.


The authors thank Merck KGaA Darmstadt for their generous support with the Merck Future Insight Prize 2020. 
This project is also cofunded by the European Union (ERC, breakingBAC, 101096911).
M.G.S. thanks Joshua Hesse and Aleksandra Daniluk for their valuable input and helpful feedback.













§ ADDITIONAL RESULTS AND DISCUSSION




  
Statistical testing
We focus on PR_AUC as our metric because it is an established performance indicator in unbalanced scenarios. 
Secondly, it shows a more pronounced separation between different methods, as most methods show very high values of ROC_AUC. 

We apply the two-sided Welch's t-test,<cit.> with Benjamini-Hochberg<cit.> multiple test correction.
This is done for all methods for which the required performance information exists in the published literature.

In <ref>, our primary focus is on the overall change in performance.
We therefore make comparisons across all datasets collectively rather than individually. 
Detailed individual comparisons are provided in <ref>.















