Paper Title: TASI Lectures on Physics for Machine Learning.







Quantum Non-Demolition Measurements and Leggett-Garg inequality
    Stefano Gherardini
    August 5, 2024
===============================================================





        0.75
        § ABSTRACT 
            These notes are based on lectures I gave at TASI 2024 on Physics for Machine Learning. The focus is on neural network theory, organized according to network expressivity, statistics, and dynamics. I present classic results such as the universal approximation theorem and neural network / Gaussian process correspondence, and also more recent results such as the neural tangent kernel, feature learning with the maximal update parameterization, and Kolmogorov-Arnold networks. The exposition on neural network theory emphasizes a field theoretic perspective familiar to theoretical physicists. I elaborate on connections between the two, including a neural network approach to field theory.
        
        









§ INTRODUCTION




Computer science (CS) is still in its infancy.  “What!?” you incredulously exclaim, thinking back to the earliest computer scientists you can think of, perhaps Lovelace or Babbage or Turing. Since its early days, CS has steadily progressed, from cracking the Enigma code in the 40's, to personal computers in the 70's and 80's, the internet in the 90's, powerful computers-in-your-pocket in the 2000's, and finally to some version of artificial intelligence in the last decade; see <cit.> for a thorough account. It shows no signs of stopping, and one might wager that future historians will not speak of decades of progress in CS, but centuries. Computer science is still in its infancy.

It is 2024, what have we seen recently? A plethora of “experimental" breakthroughs in machine learning (ML) that constitute some of the most exciting developments in CS to date. Reinforcement learning algorithms have become world-class at Go <cit.> and Chess <cit.>, merely by playing against themselves. Diffusion models <cit.> can generate images of people that don't exist, and yet you might never know. Large language models (e.g. GPT-3 <cit.> literature, or popular accounts <cit.>) can code better than you — or me, at least —  and they can even write compelling bedtime stories for your children. These forms of artificial intelligence have already started a trillion dollar industry, and some have speculated that we've already seen sparks <cit.>  of AGI, or human-level intelligence.

We've also started to see significant advances in the sciences. For instance, AlphaFold <cit.> has achieved state-of-the-art performance for predicting the structure of folded proteins, and neural networks (NNs) give state-of-the-art models for complex objects, from quantum many-body ground states <cit.> to metrics <cit.> on the Calabi-Yau manifolds of string theory. Lack-of-rigor is a natural concern, but in certain cases ML techniques can be made rigorous or interpretable at a level that would satisfy a formal theoretical physicist or pure mathematician; see, e.g., <cit.> and references therein for examples in string theory, algebraic geometry, differential geometry, and low-dimensional topology, including uses of ML theory for science.
Of course, this is just scratching the surface, see, e.g., <cit.> for a smattering of results across physics and the physical sciences.

With this in mind, we have some explaining to do. There are a large number of impressive ML results in both pop culture and the sciences, but experiments have significantly outpaced theory over the last decade. An analogy from physics might be useful, given the original audience. The current situation in ML might be reminiscent of the 1960's in particle physics: some essentials of the theory were known, e.g. quantum electrodynamics (QED), but a deluge of newly discovered hadrons left theorists scratching their heads and demanded an organizing principle to explain the new phenomena. This was one of the motivating factors for the development of string theory, but the right theory in the end is the quark model, as described by quantum chromodynamics (QCD). It's tempting to speculate that recent results in ML are waiting for someone to discover a foundational theory, akin to QCD for the strong interactions. As theoretical physicists, we hope for this sort of mechanistic understanding. However, it might be that ML and intelligence are so complex that sociology is a better analogy, and it would be a fool's errand to expect such a precise theory. My personal guess is that detailed theory will pay off in certain areas of ML — perhaps related to sparsity in NNs, architecture design, or the topics of these lectures — but not in all. Nevertheless, given the experimental progress, one must try.


These lectures were given at TASI 2024 <cit.> to Ph.D. students in theoretical high energy physics. I was given the title “Physics4ML", which means that I'm trying to keep ML-for-Physics to a minimum. I take a physics perspective, but acknowledge that this is an enormous field to which many communities have contributed.

The central idea of the lectures is that understanding NNs is essential to understanding ML, a story I built on three pillars: expressivity, statistics, and dynamics of NNs, one for each of the lectures I gave. The content builds on many ideas explained to me over the last five years by friends, acknowledged below, who pioneered many of the developments, as well as a few of my own results. Throughout, I take a decidedly field-theoretic lens to suit the audience, and also because I think it's a useful way to understand NNs. 










 §.§ The Setup.

Understanding ML at the very least means understanding neural networks. A neural network is a function

    ϕ_θ:^d →

with parameters θ. We've chosen outputs in  because, channeling Coleman, scalars already exhibit the essentials. We'll use the lingo

    Input:      x ∈^d 
    Output:     ϕ_θ(x) ∈
    Network:     ϕ_θ∈(^d, ) 
    Data:     ,

where the data  depends on the problem, but involves at least a subset of ^d, potentially paired 
with labels y∈.

With this minimal background, let's ask our central question:
What does a NN predict?
For any fixed value of θ, the answer is clear: ϕ_θ(x). However, the answer is complicated by issues of both dynamics and statistics. 

First, dynamics. In ML, parameters are updated to solve problems and we really have trajectories in 

    Parameter Space:     θ(t) ∈^| θ |
    Output Space:     ϕ_θ(t)(x) ∈
    Function Space:     ϕ_θ(t)∈(^d, ).

governed by some learning dynamics determined by the optimization algorithm and the nature of the learning problem. For instance, in supervised learning we have data

    = {(x_α, y_α) ∈^d ×}_α=1^||,

and a loss function 

    [ϕ_θ] = ∑_α=1^||ℓ(ϕ_θ(x_α), y_α),

where ℓ is a loss function such as ℓ_MSE = (ϕ_θ(x_α)-y_α)^2. One may optimize θ by gradient descent 

    dθ_i/dt = -∇_θ_i[ϕ_θ],

or other algorithms, e.g., classics like stochastic gradient descent (SGD) <cit.> or Adam <cit.>, or a more recent technique such as Energy Conserving Descent <cit.>. Throughout, t is training time of the learning algorithm unless otherwise noted.

Second, statistics. When a NN is initialized on your computer, the parameters θ are initialized as draws

    θ∼ P(θ)

from a distribution P(θ), where  means “drawn from" in this context.
Different draws of θ will give different functions ϕ_θ, and a priori we have no reason to prefer one over another. The prediction ϕ_θ(x) therefore can't be fundamental! Instead, what is fundamental is the average prediction and second moment or variance:

    [ϕ_θ(x)]     = ∫ dθ P(θ)  ϕ_θ(x) 
    [ϕ_θ(x) ϕ_θ(y)]     = ∫ dθ  P(θ)   ϕ_θ(x) ϕ_θ(y),

as well as the higher moments. Expectations are across different initializations. Since we're physicists, we henceforth replace [·]=· and we remember this is a statistical expectation value. It's useful to put this in our language:

    G^(1)(x)     = ϕ_θ(x)
    
        G^(2)(x, y)     = ϕ_θ(x) ϕ_θ(y),

the mean prediction and second moment are just the one-point and two-point correlation functions of the statistical ensemble of neural networks. Apparently ML has something to do with field theory. 

Putting the dynamics and statistics together, we have an ensemble of initial θ-values, each of which is the starting point of a trajectory θ(t), and therefore we have an ensemble of trajectories. We choose to think of θ(t) drawn as 

    θ(t) ∼ P(θ(t)),

a density on parameters that depends on the training time and yields time-dependent correlators 

    G_t^(1)(x)     = ϕ_θ(x)_t 
    
        G_t^(2)(x, y)     = ϕ_θ(x) ϕ_θ(y)_t,

where the subscript t indicates time-dependence and the expectation is with respect to P(θ(t)). Of course, assuming that learning is helping, we wish to take t→∞ and are interested in
G_∞^(1)(x) = mean prediction of ∞-number of NNs as t→∞.
Remarkably, we will see that in a certain supervised setting there is an exact analytic solution for this quantity.

There is one more pillar beyond dynamics and statistics that I need to introduce: expressivity. Let's make the point by seeing a failure mode. Consider neural networks of the following functional form, or architecture:

    ϕ_θ(x) = θ· x.

The one-point and two-point functions of these are analytically solvable, but going that far defeats the purpose: this is a linear model, and learning schemes involving it are linear regression, which will fail on general problems. We say that model is not expressive enough to account for the data. Conversely, we wish instead to choose expressive architectures that can model the data, which essentially requires that the architecture is complex enough to approximate anything. Of course, such architectures must be non-linear.

We now have the three pillars on which we'll build our understanding of neural networks, and associated questions:

    
  * Expressivity. How powerful is the NN?
    
  * Statistics. What is the NN ensemble?
    
  * Dynamics. How does it evolve?

We will approach the topics in this order, building up theory that takes us through the last few years through a physicist's lens. 

A physicist's lens means a few things. First, it means a physicist's tools, including:


  * Field Theory as alluded to above.

  * Landscape Dynamics from loss functions on θ-space.

  * Symmetries of individual NNs and their ensembles.
 
A physicist's lens also means we will try to find the appropriate balance of rigor and intuition. We'd like mechanisms and toy models, but we won't try to prove everything at a level that would satisfy a mathematician. This befits the current situation in ML, where there are an enormous number of empirical results that need an O(1) theoretical understanding, not rigorous proof.

Henceforth, we drop the subscript θ in ϕ_θ(x), and the reader should recall that the network depends on parameters.






§ EXPRESSIVITY OF NEURAL NETWORKS



Neural networks are big functions composed out of many simpler functions according to the choice of architecture. The compositions give more flexibility, prompting
How powerful is a NN?
Ultimately this is a question for mathematics, as it's a question about functions.  Less colloquially, what we mean by the power of a NN is its ability to approximate any function.



 §.§ Universal Approximation Theorem


The Universal Approximation Theorem (UAT) is the first result in this direction. It states that a neural network with a single hidden layer can approximate any continuous function on a compact domain to arbitrary accuracy. More precisely, the origin version of the theorem due to Cybenko is

Let f: ^d → be a continuous function on a compact set K⊂^d. Then for any ϵ>0 there exists a neural network with a single hidden layer of the form

    ϕ(x) = ∑_i=1^N ∑_j=1^d w_i^(1)σ(w_ij^(0)  x_j + b^(0)_i)+b^(1),

θ = {w_ij^(0), w_i^(1), b^(0)_i, b^(1)},
where σ:→ is a non-polynomial non-linear activation function, such that

    sup_x∈ K |f(x) - ϕ(x)| < ϵ.


The architecture in Eq. (<ref>) is known by many names, including the perceptron, a fully-connected network, or a feedforward network. The parameter N is known as the width. It may be generalized to include a depth dimension L encoding the number of compositions of affine transformations and non-linearities. Such an architecture is known as a multi-layer perceptron (MLP) or a deep feedforward network.

The UAT is a powerful result, but it has some limitations. First, though the error ϵ does get better with N, it doesn't say how many neurons are needed. Second, it doesn't say how to train the network: though there is a point θ^* in parameter space that is a good approximation to any f, existence doesn't imply that we can find it, which is a question of learning dynamics.

To ask the obvious,
Why does this UAT work? 
In Cybenko's original work, he focused on the case that σ was the sigmoid function 

    σ(x) = 1/1+e^-x,

which allows us to get a picture of what's happening. The sigmoids appear in the network function of the form 

    σ(w_i^(0)· x + b_i^(0))

which approximates a shifted step function as w^(0)→∞. A linear combination can turn it into a bump with approximately compact support that gets scaled by w^(1). These bumps can then be put together to approximate any function; see, e.g., Fig. <ref>.

Cybenko's version of the UAT was just the first, and there are many generalizations, including to deeper networks, to other activation functions, and to other domains.  See <cit.> for the original paper and <cit.> for a generalization to the case of multiple hidden layers.





    


 §.§ Kolmogorov-Arnold Theorem


The UAT essentially says that any function can be approximated by a neural network, i.e. a complicated function comprised out of many building blocks. It is natural to wonder whether there are other theorems to this effect, and whether they inspire other neural network constructions.

One such result is due to Kolmogorov <cit.> and Arnold <cit.>, who showed that any multivariate continuous function can be represented (exactly, not approximated) as a sum of continuous one-dimensional functions:


Let f:[0,1]^n→ be an arbitrary multivariate continuous function. Then it has the representation 

    f(x_1,…,x_n) = ∑_q=0^2nΦ_q(∑_p=1^n ϕ_q,p(x_p))

with continuous one-dimensional functions Φ_q and ϕ_q,p.


This result is perhaps less intuitive than the UAT. If every multivariate function is to be represented by univariate functions plus addition, what then of f(x,y)=xy? This is just a pedagogical example since

    f(x,y) = xy = e^log x + log y,

but of course the general theorem is very non-trivial.

A notable feature of (<ref>) is that it is a sum of functions of functions, which one might take to inspire a neural network definition. However, one aspect that is a little unusual is that ϕ_q,p depends on both p and q, and therefore lives on the connection between x_p and x^(1)_q:= ∑_p=1^n ϕ_q,p(x_p). If this is a neural network, it's one with activations ϕ_q,p on the edges, rather than on the nodes. Furthermore, for distinct values of p,q, the functions ϕ_q,p are independent in general.

In fact, this idea led to the recent proposal of Kolmogorov-Arnold networks (KAN) <cit.>, a new architecture in which activation functions are on the edges, as motivated by KART, and are learned. Much like how MLPs may be formed out of layers that repeat the structure in the single-layer UAT, KANs may be formed into layers to repeat the structure of KART. See Figure <ref> for a detailed comparison with MLPs, including a depiction of the architectures and how they are formed out of layers motivated by their respective mathematical theorems. The KAN implementation provided in <cit.> includes visualizations of the learned activation functions, which can lead to interpretable can architectures that can be mapped directly onto symbolic formulae.




§ STATISTICS OF NEURAL NETWORKS



Let's try to understand neural networks at initialization. 
 For this, firing up your computer once is not enough, since one initialization give you one draw θ∼ P(θ) and therefore one random neural network. Instead, we want to understand the statistics of the networks:
 What characterizes the stats of the NN ensemble?
 One aspect of this is encoded in the moments, or n-pt functions,

    G^(n)(x_1,…,x_n) = ϕ(x_1)…ϕ(x_n),

which may be obtained from a partition function as 

    Z[J]    = e^∫ d^dx J(x) ϕ(x)
    
    G^(n)(x_1,…,x_n)    = (δ/δ J(x_1)…δ/δ J(x_n) Z[J])|_J=0,

where J(x) is a source.
This expectation · is intentionally not specified here to allow for flexibility. For instance, using the expectation in the introduction we have

    Z[J]     = ∫ dθ P(θ) e^∫ d^dx  J(x) ϕ(x),

reminding the reader that the NN ϕ(x) depends on θ. The partition function integrates over the density of network parameters. But as physicists we're much more familiar with function space densities according to 

    Z[J]     = ∫ϕ e^-S[ϕ] e^∫ J(x) ϕ(x),

the Feynman path integral that determines the correlators from an action S[ϕ] that defines a density on functions.

Since starting a neural network requires specifying the data (ϕ,P(θ)), the parameter space partition function (<ref>) and associated parameter space calculation of correlators is always available to us. Given that mathematical data, one might ask
What is the action S[ϕ] associated to (ϕ,P(θ))?
When this question can be answered, it opens a second way of studying or understanding the theory. The parameter-space and function-space descriptions should be thought of as a duality.



 §.§ NNGP Correspondence


Having raised the question of the action S[ϕ] associated to the network data (ϕ,P(θ)), we can turn to a classic result of Neal <cit.>.

For simplicity, we again consider a single-layer fully connected network of width N, with the so-called biases turned off for simplicity:

    ϕ(x) = 1/√(N)∑_i=1^N ∑_j=1^d w_i^(1)σ(w_ij^(0) x_j),

where the set of network parameters is θ = {w_ij^(0), w_i^(1)} independently and identically distributed (i.i.d.). 

    w_ij^(0)∼ P(w^(0))      w_i^(1)∼ P(w^(1)).
 
Under this assumption, we see
ØThe network is a sum of N i.i.d. functions.
This is a function version of the Central Limit Theorem, generalizing the review in Appendix <ref>, and gives us the Neural Network / Gaussian Process (NNGP) correspondence, 
NNGP Correspondencein the N →∞ limit, ϕ is drawn from a Gaussian Process (GP), 

    lim_N→∞   ϕ(x) ∼(μ(x), K(x,y)),

with mean and covariance (or kernel) μ(x) and K(x,y).
By the CLT, exp(-S[ϕ]) is Gaussian and therefore
S[ϕ] is quadratic in networks. Now this really feels like physics, since the infinite neural network is drawn from a Gaussian density on functions, which defines a generalized free field theory.

We will address generality of the NNGP correspondence momentarily, but let's first get a feel for how to do computations.
To facilitate then, we take P(w^(1)) to have zero mean and finite variance, 

    w^(1)=0     w^(1)w^(1) = μ_2,
 which causes the one-point function to vanish
G^(1)(x) = 0. Following Williams <cit.>, we compute the two-point function in parameter space (with Einstein summation)

    G^(2)(x,y) 
                = 1/Nw_i^(1)σ(w_ij^(0) x_j)  w_k^(1)σ(w_kl^(0) y_l)
       1/N w_i^(1) w_k^(1)σ(w_ij^(0) x_j) σ(w_kl^(0) y_l)
        = μ_2/Nσ(w_ij^(0) x_j) σ(w_il^(0) y_l),

where the last equality follows from the ones being i.i.d., w_i^(1)w_k^(1) = μ_2 δ_ik. The sum over i gives us N copies of the same function, leaving us with 

    G^(2)(x,y) = μ_2  σ(w_ij^(0) x_j) σ(w_il^(0) y_l),

where we emphasize there is now 
no summation on i. This is an exact-in-N two-point function that now requires only on the computation of the quantity in bra-kets. One may try to evaluate it exactly by doing the integral over w^(0). If it can't be done, Monte Carlo estimates may be obtained from M samples of w^(0)∼ P(w^(0)) as

    G^(2)(x,y) ≃μ_2/M∑_samples^M σ(w_ij^(0) x_j) σ(w_il^(0) y_l).

In typical NN settings, parameter densities are easy to sample for convenience, allowing for easy computation of the estimate. If the density is more complicated, one may always resort to Markov chains, e.g. as in lattice field theory.

With this computation in hand, we have the defining data of this NNGP,

    lim_N→∞   ϕ(x) ∼(0, G^(2)(x,y)).

The associated action is 

    S[ϕ] = ∫ d^dx d^dy ϕ(x)  G^(2)(x,y)^-1 ϕ(y),

where 

    ∫ d^dy   G^(2)(x,y)^-1 G^(2)(y,z) = δ^(d)(x-z).

defines the inverse two-point function. In fact, this allows us to determine the action of any NNGP with μ(x)=G^(1)(x)=0, by computing the G^(2) in parameter space and inverting it.



So certain large neural networks are function draws from generalized free field theories. But at this point you might be asking yourself
How general is the NNGP correspondence?
Neal's result — that infinite-width single-layer feedforward NNs are drawn from GP — stood on its own for many years, perhaps (I am guessing) due to focus on non-NN ML techniques in the 90's and early 2000's during a so-called AI Winter. As NNs succeeded on many tasks in the 2010's after AlexNet <cit.>, however, many asked whether architecture X has a hyperparameter N such that the network is drawn from a Gaussian Process as N→∞. Before listing such X's, let's rhetorically ask
Didn't Neal's result essentially follow from summing N i.i.d. random functions? Maybe NNs do this all the time?
In fact, that is the case. Architectures admitting an NNGP limit include

    
  * Deep Fully Connected Networks, N=width,
    
  * Convolutional Neural Networks, N=channels,
    
  * Attention Networks, N=heads,

and many more. See, e.g., <cit.> and references therein.



 §.§ Non-Gaussian Processes


If the GP limit exists due to the CLT, then violating any of the assumptions of the CLT should introduce non-Gaussianities, which are interactions in field theory. From Appendix <ref>, we see that the CLT is violated by finite-N corrections and breaking statistical independence. See <cit.> for a systematic treatment of independence breaking, and derivation of NN actions from correlators.

We wish to see the N-dependence of the connected 4-pt function. William's technique for computing G^(2) extends to any correlator. To avoid a proliferation of indices, we will compute it using the notation

    ϕ(x) = ∑_i w_i φ_i(x)

where w_i is distributed as w^(1) was in the single layer case, and φ_i(x) are i.i.d. neurons of any architecture. The four-point function is 

    G^(4)   = ϕ(x)ϕ(y)ϕ(z)ϕ(w)
       = ∑_i,j,k,lw_i w_j w_k w_lφ_i(x)φ_j(y)φ_k(z)φ_l(w)
       = ∑_i w_i^4φ_i(x)φ_i(y)φ_i(z)φ_i(w)
               + ∑_i≠ jw_i^2w_j^2φ_i(x)φ_i(y)φ_j(z)φ_j(w) + perms.

One can see that you have to be careful with indices. The connected 4-pt function is <cit.>

    G^(4)_c(x_,y,z,w) = G^(4)(x,y,z,w) - (G^(2)(x,y)G^(2)(z,w) + perms),

and watching indices carefully we obtain 

    G^(4)_c(x_,y,z,w) = 1/N(μ_4  φ_i(x)φ_i(y)φ_i(z)φ_i(w)
     - μ_2^2  (φ_i(x)φ_i(y)φ_i(z)φ_i(w) + perms)),

with no Einstein summation on i.
We see that the connected 4-pt function is non-zero at finite-N, signalling interactions. We will see that in some examples G^(4)_c can be computed exactly.




 §.§ Symmetries


We have gotten some control over the statistics of the ensemble of neural networks. It natural 
at this point to ask 
Is there structure in the ensemble?
By this I mean properties that the ensemble realizes that an individual network might not see. We will return to this question broadly in Section <ref>, but for now we will focus on one type of structure: symmetries.

To allow for symmetries at both input and output, in this section we consider networks

    ϕ: ^d →^D.
 
with D-dimensional output. Sometimes the indices will be implicit.

A classic example is equivariant neural networks <cit.>.
We say that ϕ is G-equivariant with respect to a group G if 

    ρ_D(g) ϕ(x) = ϕ(ρ_d(g) x)     ∀ g ∈ G,

where 

    ρ_d ∈Mat(^d)    ρ_D ∈Mat(^D)

are matrix representations of G on ^D and ^d, respectively. The network is invariant if ρ_D = 1, the trivial representation.
Equivariance is a powerful constraint on the network that may be implemented in a variety of ways. For problems that have appropriate symmetries, building them into the network can improve the speed and performance of learning <cit.>, e.g., at the level of scaling laws <cit.>. For instance, in Phiala Shanahan's lectures you'll learn about SU(N)-equivariant neural networks from her work <cit.>, which are natural in lattice field theory due to invariance of the action under gauge transformations.

But this is the statistics section, so 
we're interested in symmetries that arise in ensembles of neural networks, which leave the statistical ensemble invariant. In field theory, we call them global symmetries. Let the network transform under a group action as 

    ϕ↦ϕ_g,      g ∈ G.

We say that ensemble of networks has a global symmetry group G if the partition function is invariant,

    Z_g[J] = Z[J],     ∀ g ∈ G.

At the level of expectations, this is 

    e^∫ d^d x J(x) ϕ_g(x)  =e^∫ d^d x J(x) ϕ(x)    ∀ g ∈ G,

where one can put indices on ϕ and J as required by D. By a network redefinition on the LHS, this may be cast as having a symmetry if · is invariant. 
In the usual path integral this is the statement of invariant action S[ϕ] and measure ϕ. In parameter space, the redefinition may be instituted <cit.> by absorbing g into a redefinition of parameters as θ↦θ_g, with symmetry arising when 

    ∫ dθ_g P(θ_g) e^∫ d^d x J(x)ϕ_θ(x) = ∫ dθ P(θ) e^∫ d^d x J(x)ϕ_θ(x),

i.e. the parameter density and measure must be invariant. We will give a simple example realizing this mechanism in a moment.

It is most natural to transform the input or output of the network. Our mechanism allows for symmetries of both types, which are analogs of spacetime and internal symmetries, respectively. It may also be interesting to study symmetries of intermediate layers, if one wishes to impose symmetries on learned representations. Equivariance fits into this picture because it turns a transformation at input into a transformation at output. The ensemble of equivariant NNs is invariant under ρ_d action on the input if the partition function is invariant under the induced ρ_D action on the output.


Example.  Consider any architecture of the form 

    ϕ(x) = ∑_i w_i φ_i(x),      w_i ∼ P(w)  even,

for any neuron φ, which could itself be a deep network. The _2 action 
ϕ↦ -ϕ may be absorbed into parameters w_g = -w_i with dw P(w) invariant by evenness, which is a global symmetry provided that the domain is invariant. This theory has only even correlators. 

Less trivial examples will be presented below, when we present specific networks and compute correlators.



 §.§ Examples


In the statistics of neural networks, we have covered three topics: Gaussian limits, non-Gaussian corrections from violating CLT assumptions, and symmetries. We will present this essential data in some examples and then discuss similarities and differences. For canonical cases in ML, like networks with ReLU activation or similar, see <cit.> and references therein.

Gauss-net.
The architecture and parameter densities are 

    ϕ(x) = w^(1)_iexp(w^(0)_ij x_j + b^(0)_i)/√(exp[2(σ_b_0^2 + σ^2_w_0/x^2 )]),

for parameters drawn i.i.d. from 

    w^(0)∼𝒩(0,σ^2_W_0/),     w^(1)∼𝒩(0, σ^2_w_1/2N),     b^(0)∼𝒩(0, σ^2_b_0).

The two-point function is

    G^(2)(x_1, x_2) = σ_w_1^2/2 e^-1/2σ_w_0^2 (x_1-x_2)^2

and we see that the theory has a correlation length 

    ξ = √(/σ_w_0^2).

The connected four-point function is

    G^(4)_c =   1/4 Nσ_w_1^4 [ 3e^4σ_b_0^2 e^-σ_w_0^2/2d∑_i x_i^2 -2(x_1 x_2 +  6  perms)
       - (e^-1/2dσ_w_0^2
        ((x_1-x_4)^2+(x_2-x_3)^2) +  2  perms 
        ) ].

We see that the 2-pt function is translation invariant, but not the 4-pt function.


Euclidean-Invariant Nets. 
We are interested in constructing input layers that are sufficient to ensure invariance under SO(d) and translations, i.e., under the Euclidean group. Consider an input layer of the form 

    ℓ_i(x) = F( w^(0)_i) cos(∑_j w^(0)_ij x_j + b^(0)_i),     i ∈1,…,N,

where the sum has been made explicit since i's are not summed over, with 

    w^(0)_ij∼ P(w^(0)_ij)      b^(0)_i ∼Unif[-π,π].

It is easy to see that x_i→ R_ij x_j for R∈ SO(d) can be absorbed into a redefinition of the w^(0)_ij's, and the partition function is invariant under this action provided that F( w^(0)_i) and P(w^(0)_ij) are invariant. Furthermore, arbitrary translations x_j → x_j+ϵ_j give a term w^(0)_ijϵ_j that can be absorbed into a redefinition of the b^(0)_i that leaves the theory invariant. See <cit.> for the 2-pt and 4-pt functions.

We emphasize two nice features:


  * Larger Euclidean Nets. Any network that builds on ℓ without reusing its parameters is Euclidean-invariant.

  * Spectrum Shaping. In computing G^(2)(p), b_i gets evaluated on p, and F may be chosen to shape the power spectrum (momentum space propagator) arbitrarily.

We refer the reader to <cit.> for general calculations of ℓ-correlators and to the specializations below for simple single-layer theories. We turn the ℓ_i(x) input layer into a scalar NN by

    ϕ(x) = ∑ w^(1)_i ℓ_i(x)

with drawn w^(1)_i ∼ P(w^(1)).

Cos-net.
The architecture is defined by specifying (<ref>) to F=1 

    ϕ(x) =  w^(1)_i cos( w^(0)_ij x_j + b^(0)_i),

and specific parameter densities

    w^(1)∼(0,σ_w_1^2/N)      w^(0)∼(0,σ_w_0^2/d)      b^(0)∼Unif[-π,π].

The two-point function is equivalent to that of Gauss-net

    G^(2)(x,y) = σ_w_1^2/2 e^-1/2σ_w_0^2 (x_1-x_2)^2.

The connected four-point function is 

    G^(4)|_c =    1/8 Nσ_w_1^4 [ 3  (e^-1/2dσ_w_0^2
       (x_1+x_2-x_3-x_4)^2+  2 perms) 
       -(2 e^-1/2dσ_b^2
       ((x_1-x_4)^2+(x_2-x_3)^2)+  2 perms) ].

The 2-pt function and 4-pt function are fully Euclidean invariant.

Equipped with both Cos-net and Gauss-net, we'd like to compare the two:


  * Symmetry. Cos-net is Euclidean-invariant ∀ N, by construction, while Gauss-net enjoys this symmetry only when N→∞; in that limit it only depends on G^(2)(x,y).

  * Large-N Duality. As N→∞, both theories are drawn from the same Gaussian Process, i.e., with the same G^(2)(x,y).


Scalar-net.
Here is one final example you might like, specializing (<ref>)

    ϕ(x) =  √(2 vol(B_Λ^d)/(2π)^d σ_w_1^2)1/√( w^(0)_i^2 + m^2)  w^(1)_i cos( w^(0)_ij x_j + b^(0)_i),

and specific parameter densities

    w^(1)∼(0,σ_w_1^2/N)      w^(0)∼Unif(B_Λ^d)      b^(0)∼Unif[-π,π],

where B_Λ^d is a d-ball of radius Λ. The theory is translation invariant by construction, and so we compute the power spectrum of the two-point function G^(2)(x-y) to be 

    G^(2)(p) = 1/p^2+m^2.

We see that we have a realization of the free scalar field theory in d Euclidean dimensions. See Sec. <ref> for an extension to ϕ^4 theory.
    



§ DYNAMICS OF NEURAL NETWORKS



Having covered expressivity and statistics, we turn to dynamics. Focusing on the most elementary NN dynamics, we ask
How does a NN evolve under gradient descent? First we will study a simplification known as the neural tangent kernel (NTK), and then will use it in the case of MSE loss to solve a model exactly. We'll discuss drawbacks of the NTK, and then improve upon them with a scaling analysis that ensures feature learning.

We will study the dynamics of supervised learning with gradient  descent, with data 

    = {(x_α,y_α)}_α=1^||,

and loss function

    [ϕ] = 1/||∑_α=1^||ℓ(ϕ(x_α), y_α),

We optimize the network parameters θ by gradient descent,

    dθ_i/dt = -η∇_θ_i[ϕ].

It is also convenient to define 

    Δ(x) = - δℓ(ϕ(x),y)/δϕ(x),

where y is to be understood as the label associated to x, which yields 

    dθ_i/dt= η/||∑_α=1^||Δ(x_α) ∂ϕ(x_α)/∂θ_i.

as another form of the gradient descent equation, by chain rule. Δ(x) is the natural object of gradient descent in function space.

We use Einstein summation throughout this section unless stated otherwise (which will happen).



 §.§ Neural Tangent Kernel


We now arrive at a classic dynamical result in ML theory, the neural tangent kernel <cit.>. We train the network by gradient descent 

    dϕ(x)/dt   = ∂ϕ(x)/∂θ_idθ_i/dt
       = η/||∑_α=1^||Δ(x_α)  Θ(x,x_α),

where 

    Θ(x,x_α) = ∂ϕ(x)/∂θ_i∂ϕ(x_α)/∂θ_i

is the neural tangent kernel (NTK).

Given the short derivation, this is clearly a fundamental object, but it seems terrible to work with, because it is:


  * Parameter-dependent. For modern NNs, there are billions of parameters to sum over.

  * Time-dependent. Of course, the learning trajectory is θ(t), and therefore the NTK time evolves.

  * Stochastic. Since θ(t) begins at θ(0) sampled at initialization, the NTK inherits the randomness.

It's also non-local, communicating information about the loss at train points x_α to the test point x. In summary, the NTK is unwieldy.

The reason it is a classic result, however, is that it simplifies in the N→∞ limit. In this limit, neural network training is in the so-called

    Lazy regime:     |θ(t) - θ(0)| ≪ 1,

i.e. the number of parameters is large but the evolution keeps them in a local neighborhood. In such a regime the network is approximately a linear-in-parameters model <cit.>

    lim_N→∞ϕ(x)≃ϕ_lin(x) := ϕ_θ_0(x) + (θ-θ_0)_i ∂ϕ(x)/∂θ_i|_θ_0,

and we have 

    lim_N→∞Θ(x,x') ≃Θ(x,x')|_θ_0.

That is, the infinite-width NTK is the NTK at initialization, provided that the network evolves as a linear model. Furthermore, in the same limit the law of large numbers often allows a sum to be replaced by an expectation value, e.g., 

    lim_N→∞Θ(x,x')|_θ_0 = β_θ(x,x') =: Θ̅(x,x'),

for computable β(x,x'), yielding network dynamics governed by 

    dϕ(x)/dt= -η/||∑_α=1^||δ l(ϕ(x_α),y_α)/δϕ(x_α)  Θ̅(x,x_α),

where Θ̅ is the so-called frozen NTK, a kernel that may be computed at initialization and fixed once-and-for-all. This is a dramatic simplification of the dynamics.

However, you should also complain.
ComplaintThe dynamics in (<ref>) simply interpolates between information at train points x_α and test point x,
according to a fixed function Θ̅. This isn't “learning" in  the usual NN sense, and there are zero parameters. In particular, since the NN only affects (<ref>) through Θ̅, which is fixed, nothing happening dynamically in the NN is affecting the evolution. We say that in this limit the NN does not learn features in the hidden dimensions (intermediate layers), since their non-trivial evolution would cause the NTK to evolve.

Example. 
Let's compute the frozen NTK for a single-layer network, to get the idea. The architecture is 

    ϕ(x) = 1/√(N)∑_i=1^N ∑_j=1^d w_i^(1)σ(w_ij^(0) x_j).

We make the sums explicit here because one is very important.
The NTK is 

    Θ(x,x')    = ∑_i∂ϕ(x)/∂ w_i^(1)∂ϕ(x')/∂ w_i^(1) + ∑_ij∂ϕ(x)/∂ w_ij^(0)∂ϕ(x')/∂ w_ij^(0)
       = 1/N∑_i=1^N (∑_j,l=1^d σ(w_ij^(0)x_j)σ(w_il^(0)x'_l)
            + ∑_j=1^d x_j x'_j  w_i^(1)w_i^(1) σ'(w_ij^(0) x_j) σ'(w_ij^(0)x'_j) ) 
       =: 1/N∑_i β_i(x,x').

If you squint a little, you'll see that the i-sum is a sum over the same type of object, β_i(x,x'), whose i dependence comes from all these i.i.d. parameter draws in the i-direction. By the law of large numbers, we have that in the N→∞ limit

    Θ̅(x,x') = β_i(x,x'),

with no sum on i.
We emphasize 
ØThe NTK in the N→∞ limit is deterministic (parameter-independent), depending only on P(θ).
Sometimes, the expectation may be computed exactly, and one knows the NTK that governs the dynamics once and for all.



 §.§ An Exactly Solvable Model


Let us consider a special case of frozen-NTK dynamics with MSE loss,

    ℓ(ϕ(x),y) = 1/2(ϕ(x)-y)^2.

Then the dynamics (<ref>) becomes 

    dϕ(x)/dt = -η/||∑_α=1^|| (ϕ(x_α)-y_α) Θ̅(x,x_α).

The solution to this ODE is 

    ϕ_t(x) = ϕ_0(x) + 1/||Θ̅(x,x_α) Θ̅(x_α, x_β)^-1(1-e^-ηΘ̅t)_βγ(y_γ-ϕ_0(x_γ)),

where computational difficulty is that Θ̅(x,x_α) is a ||× || matrix and takes O(||^3) time to invert. 
The solution defines a trajectory through function space from ϕ_0 to ϕ_∞. The converged network is 

    ϕ_∞(x) = ϕ_0(x) +Θ̅(x,x_α) Θ̅(x_α, x_β)^-1(y_β-ϕ_0(x_β)).

This is known as kernel regression, a classic technique in ML. 
In general kernel regression, one chooses the kernel. In our case, gradient descent training in the N→∞ limit is kernel regression, with respect to a specific kernel determined by the NN, the NTK Θ̅. 

On train points we have memorization

    ϕ_∞(x_α) = y_α    ∀α.

On test points x, the converged network is performing an interpolation, communicating residuals R_β on train points β through a fixed kernel Θ̅ to test points x. The prediction depends on ϕ_0, but may be averaged over to obtain 

    μ_∞(x) := ϕ_∞(x) = Θ̅(x,x_α) Θ̅(x_α, x_β)^-1y_β,

provided that ϕ_0=0, as in many initializations for the parameters.
Let's put some English on the remarkable facts,

    
  * μ_∞(x) is the mean prediction of an ∞ number of ∞-wide NNs trained to ∞ time.
    
  * If ϕ_0 is drawn from a GP, then ϕ_∞ is as well. The mean is precisely μ_∞(x), see <cit.> for the two-point function and covariance.




 §.§ Feature Learning


The frozen NTK is a tractable toy model, but it has a major drawback: it does not learn features.
In this section we perform a more general study of learning dynamics, with a focus on choosing wise N-scaling such that features are non-trivially learned during gradient descent. To do this we will engineer three properties: that features (pre-activations) are finite at initialization, predictions evolve in finite time, and features evolve in finite time. 

This section is denser than the others, so I encourage the reader to remember that we are aiming to achieve these three principles if they become overwhelmed with details. Throughout, I am following lecture notes of Pehlevan and Bordelon <cit.>  with some changes in notation to match the rest of the lectures; see those lectures for more details, <cit.> for original literature from those lectures that I utilize, and <cit.> for related work on feature learning.

We study a deep feedforward network with L layers and width N, which in all is a map

    ϕ:^D →

(note the input dimension D; d is reserved for below) defined recursively as 

    ϕ(x)    = 1/γ_0 N^dz^(L)(x) 
     
    z^(L)(x)    = 1/N^a_L w_i^(L) σ(z_i^(L-1)(x))
    
    z^(ℓ)_i(x)    = 1/N^a_ℓ W^(ℓ)_ijσ(z_j^(ℓ-1)(x)) 
    
    z_i^(1)(x)    = 1/N^a_1√(D) W^(1)_ijx_j

where Einstein summation is implied throughout this section   (unless stated otherwise) and all Latin indices run from {1,…,N} except in the j-index in the first layer, when they are {1,…,D}. The parameters are drawn 

    w_i^(ℓ)∼(0,1/N^b_L)      W^(ℓ)_ij∼(0,1/N^b_ℓ).

We scale the learning rate as 

    η = η_0 γ_0^2 N^2d-c

with γ_0,η_0 O(1)  constants. We use a parameterization that will be convenient where d has already been introduced but c is a new parameter. For notational brevity we will sometimes use a Greek index subscript in place of inputs, e.g. z^(ℓ)_α:= z^(ℓ)(x_α). The z's are known as the pre-activations, as they are the inputs to the activation functions σ.

We have a standard MLP but have parameterized our ignorance of N-scaling, governed by parameters (a_ℓ,b_ℓ,c,d). We will use this freedom to set some reasonable goals:

    
  * Finite Initialization Pre-activations. z^(ℓ)∼ O_N(1)   ∀ l.
    
  * Learning in Finite Time. dϕ(x)/dt∼ O_N(1).
    
  * Feature Learning in Finite Time. dz^(ℓ)/dt ∼ O_N(1)    ∀ l.

These constraints have a one-parameter family of solutions, which becomes completely fixed under an additional learning rate assumption. We take each constraint in turn.

Finite Pre-activations. Since the weights are zero mean,  trivially

    z^(ℓ)_α = 0     ∀ l.

We must also compute the covariance. For the first pre-activation, we have 

    z_iα^(1)z_jβ^(1) = 1/D N^2a_1w^(1)_imw^(1)_jn  x_mαx_nβ = δ_ij1/DN^2a_1+b_1 x_mαx_mβ.

A similar calculation for higher layers l  gives

    z_iα^(ℓ)z_jβ^(ℓ)   = 1/ N^2a_ℓw^(1)_imw^(1)_jn σ(z^(ℓ-1)_mα)σ(z^(ℓ-1)_mβ)
       = δ_ij1/N^2a_ℓ+b_ℓ-11/Nσ(z^(ℓ-1)_mα)σ(z^(ℓ-1)_mβ)
       = δ_ij1/N^2a_ℓ+b_ℓ-1Φ^(ℓ-1)_αβ,

where 

    Φ^(ℓ)_αβ := 1/Nσ(z^(ℓ)_mα)σ(z^(ℓ)_mβ)

is a feature kernel. We are constructing a proof-by-induction that the pre-activations are O_N(1), so at this stage we may assume that the pre-activations z^(ℓ-1)∼ O_N(1) and therefore Φ^(ℓ-1)∼ O_N(1) since it is the average of N O_N(1) quantities σ(z^(ℓ-1)_mα). With this in hand, 

    2a_1+b_1=0      2a_ℓ + b_ℓ = 1   ∀ l > 1,

ensures that the pre-activations z^(ℓ) are O_N(1), as is empirically required for well-behaved training.

As an aside: in the N→∞ limit feature kernels asymptote to deterministic objects, akin to the frozen NTK behavior we have already seen, and intermediate layer pre-activations z_α^(ℓ) are also Gaussian distributed. Therefore in that limit, the statistics of a randomly initialized neural network is described by a sequence of generalized free field theories where correlations are propagated down the network according to a recursion relation; see, e.g., <cit.>.

Learning in Finite Time. Similar to our NTK derivation, we have 

    dϕ(x)/dt = η/||∑_α=1^||Δ(x_α) ∂ϕ(x_α)/∂θ_i∂ϕ(x)/∂θ_i,

We have that 

    dϕ(x)/dt∼ O_N(1)   ↔   γ_0^2 N^2d/N^c∂ϕ(x_α)/∂θ_i∂ϕ(x)/∂θ_i∼ O_N(1).

A short calculation involving a chain rule, keeping track of the different types of layers, and tedium yields

    η_0γ_0^2 N^2d/N^c∂ϕ(x_α)/∂θ_i∂ϕ(x)/∂θ_i   = 1/N^c[ 1/N^2 a_L - 1Φ_μν^(L-1)
       + ∑_ℓ=2^L-11/N^2 a_ℓ - 1 G_α x^(ℓ)Φ_α x^(ℓ-1) + 1/N^2 a_1 G_α x^(1)Φ_α x^(0)]

where 

    G_μν^(ℓ) = 1/N g^(ℓ)_iμ  g^(ℓ)_iν,
          g^(ℓ)_iμ = √(N) ∂ z^(L)_μ/∂ z^(ℓ)_iμ

arises naturally when doing a chain rule through the layers and x in the subscript is a stand-in for an arbitrary input x. Some additional work <cit.> shows that g, G ∼ O_N(1). Since we've shown that the feature kernels are also O_N(1), then if 

    2a_1 + c = 0,      2a_ℓ + c = 1    ∀ l > 1,

we have that dϕ(x)/dt∼ O_N(1), i.e., predictions evolve in finite time, and therefore learning happens in finite time.

Features Evolve in Finite Time. 
Finally, we need the features to evolve. For the first layer we compute

    dz_iμ^(1)/dt   =1/N^a_1√(D)dW_ij/dt x_jμ 
     = 1/N^2a_1+c-d+1/2η_0γ_0/||∑_ν=1^||Δ_ν g_iν^(1)Φ_μν^(0),
 
where the only N-dependence arises from the exponent. Given (<ref>), 

    d=1/2

is required to have  dz^(1)/dt ∼ O_N(1), a constraint that in fact persists for all dz^(ℓ)/dt. We note that we have
No feature learningif d< 1/2,
which applies in particular to the NTK regime d=0.

Summarizing the Constraints. Putting it all together, by enforcing finite pre-activations and that predictions and features evolve in finite time, we have constraints given by

    2a_1 + b_1    = 0 
    
        2a_ℓ + b_ℓ   = 1   ∀ l > 1 
    
        2a_1 + c    = 0 
    
        2a_ℓ + c    = 1   ∀ l > 1 
    
        d    = 1/2.

These constraints are solves by a one-parameter family depending on a∈

    (a_ℓ, b_ℓ, c_ℓ)    = (a,1-2a,1-2a)   ∀ l > 1 
     
    (a_1, b_1, c_1)     = (a-1/2, 1-2a, 1-2a).

If one makes the addition demands that η∼ O_N(N^2d-c) is O_N(1), so that you don't have to change the learning rate as you scale up, then c=1 is fixed and there is a unique solution. This scaling is known as the maximal update parameterization, or μ P <cit.>.







§ NN-FT CORRESPONDENCE



Understanding the statistics and dynamics of NNs has led us naturally to objects that we are used to from field theory. The idea has been to understand ML theory, but one can also ask the converse, whether ML theory gives new insights into field theory. With that in mind, we ask
What is a field theory?
At the very least, a field theory needs 
    
  * Fields, functions from an appropriate function space, or sections of an appropriate bundle, more generally. 
    
  * Correlation Functions of fields, here expressed as scalars
    
    G^(n)(x_1,…,x_n) = ϕ(x_1)…ϕ(x_n).


You might already be wanting to add more beyond these minimal requirements – we'll discuss that in a second. For now, we have 
Answera FT is an ensemble of functions with a way to compute their correlators.
In the Euclidean case, when the expectation is a statistical expectation, one my say
Euclidean Answera FT is a statistical ensemble of functions.
Our minimal requirements get us a partition function

    Z[J] = e^∫ d^dx J(x)ϕ(x)

that we can use to compute correlators, where at this stage we are agnostic about the definition of ·. In normal field theory, the · is defined by the Feynman path integral 

    Z[J] = ∫ϕ  e^-S[ϕ] + ∫ d^dx J(x)ϕ(x),

which requires specifying an action S[ϕ] that determines a density on functions exp(-S[ϕ]). But that's not the data we specify when we specify a NN. The NN data (ϕ_θ, P(θ)) instead defines 

    Z[J] = ∫ dθ P(θ) e^∫ d^dx J(x)ϕ_θ(x).

These are two different ways of defining a field theory, and indeed given (ϕ_θ,P(θ)) one can try to work out the associated action, in which case we have dual description of the same field theory, as in the NNGP correspondence.
The parameter space description is already quite useful, though, as it enables the computation of correlation functions even if the action isn't known. In certain cases it enables the computation of exact correlators in interacting  theories.

Okay, you get it, this is a different way to do field theory. Now I'll let you complain about my definition. You're asking 
Shouldn't my definition of field theory include X?
I'm writing this before I give the lecture, and my guess is you already asked about a set of X's, e.g. 

    X ∈{Quantum ,
         Lagrangian,
         Symmetries,
         Locality, …}.

The problem is that with any such X, there's usually some community of physicists that doesn't care. For instance, not all statistical field theories are Wick rotations of quantum theories; not all field theories have a known Lagrangian; not all field theories have symmetry; not all field theories are local. So I'm going to stick with my definition, because at a minimum I want fields and correlators.

Instead, if your X isn't included in the definition of field theory, it becomes an engineering problem. Whether you're defining your specific theory by S[ϕ], (ϕ_θ,P(θ)), or something else, you can ask
Can I engineer my defining data to get FT + X?
For X=Symmetries you've already seen ways to do this at the level of actions in QFT1 and at the level of (ϕ_θ,P(θ)) in these lectures.


For a current account of recent progress in NN-FT, see the rather long Introduction of <cit.> and associated references, as well as results.



 §.§ Quantum Field Theory


We've been on Euclidean space the whole time[This can be relaxed, see e.g. for a recent paper defining an equivariant network in Lorentzian signature <cit.>.], so it's natural to wonder in what sense these field theories are quantum. In a course on field theory, we first learn to canonically quantize and then at some later point learn about Wick rotation, and how it can define Euclidean correlators. The theory is manifestly quantum.

But given a Euclidean theory, can it be continued to a well-behaved quantum theory in Lorentzian signature, e.g. with unitary time evolution and a Hilbert space without negative norm states? If we have a nice-enough local action, it's possible, but what if we don't have an action? We ask:
Given Euclidean correlators, can the theory be continued to a well-behaved Lorentzian quantum theory?
This is a central question in axiomatic quantum field theory, and the answer is that it depends on the properties of the correlators. The Osterwalder-Schrader (OS) theorem <cit.> gives a set of conditions on the Euclidean correlators that ensure that the theory can be continued to a unitary Lorentzian theory that satisfies the Wightman axioms.
The conditions of the theorem include 


  * Euclidean Invariance. The correlators are invariant under the Euclidean group, which after continuation to Lorentzian signature becomes the Poincaré group.

  * Permutation Invariance of the correlators G^(n)(x_1,…,x_n) under any permutation of the x_1,…,x_n.

  * Reflection Positivity. Having time in Lorentzian signature requires picking a Euclidean time direction τ. Let R(x) be the reflection of x in the τ=0 plane. Then reflection positivity requires that

    G^(2n)(x_1,…,x_n,R(x_1),…,R(x_n)) ≥ 0.

Technically, this is necessary but not sufficient. An accessible elaboration can be found in notes <cit.> from a previous TASI.

  * Cluster Decomposition occurs when the connected correlators vanish when any points are infinitely far apart.

If all of these are satisfied, then the pair (ϕ_θ, P(θ)) that defines the NN-FT actually defines a neural network quantum field theory <cit.>.
In NN-FT, permutation invariance is essentially automatic and Euclidean invariance may be engineered as described in Section <ref>. Cluster decomposition and reflection positivity hold in some examples <cit.>, but systematizing their construction is an important direction for future work.

There is at least one straightforward way to obtain an interacting NN-QFT. Notably, if (ϕ_θ, P(θ)) is a NNGP that satisfies the OS axioms (this is much easier <cit.>) with Gaussian partition function

    Z_G[J] = ∫ dθ P(θ) e^∫ d^dx J(x)ϕ_θ(x),

then one may insert an operator associated to any local potential V(ϕ), which deforms the action in the expected way and the NN-FT to 

    Z[J]    = ∫ dθ P(θ) e^∫ d^dx V(ϕ_θ(x)) e^∫ d^dx J(x)ϕ_θ(x)
       =: ∫ dθP̃(θ) e^∫ d^dx J(x)ϕ_θ(x),

where the architecture equation ϕ_θ(x) lets us sub out the abstract expression for a concrete function of parameters, defining a new density on parameters P̃(θ) in the process. The interactions in V(ϕ) break Gaussianity of the NNGP that was ensured by a CLT. This means a CLT assumption must be violated: it is the breaking of statistical independence in P̃(θ). The theory Z[J] defined by (ϕ_θ, P̃(θ)) is an interacting NN-QFT, since local potentials that deformed Gaussian QFTs still satisfy reflection positivity and cluster decomposition.



 §.§ ϕ^4 Theory


With this discussion of operator insertions, it's clear now how to get ϕ^4 theory. We just insert the operator

    e^∫ d^dx ϕ_θ(x)^4

into the partition function associated to the free scalar of Section <ref>; this operator with the architecture (<ref>) technically requires an IR cutoff, though other architectures realizing the free scalar may not. The operator insertion deforms the parameter densities in (<ref>) and breaks their statistical independence, explaining the origin of interactions in the NN-QFT. See <cit.> for a thorough presentation.



 §.§ Open Questions


In this section we've discussed a neural network approach to field theory in which the partition function is an integral over probability densities of parameters, and the fields (networks) are functions of these parameters. We have summarized some essential results, but there are many outstanding questions:


  * Reflection Positivity. Is there a systematic way to engineer NN-FTs (ϕ_θ, P(θ)) that satisfy reflection positivity?

  * Cluster Decomposition. Can  systematically understand the conditions under which cluster decomposition holds in NN-FT?

  * Engineering Actions. How generally can we define an NN-FT that is equivalent to a field theory with a fixed action?

  * Locality in the action can be realized at infinite-N, but is there a fundamental obstruction at finite-N?

On the other hand, an NN-FT approach to CFT and to Grassmann fields is well underway and should appear in 2024.



§ RECAP AND OUTLOOK


Neural networks are the backbone of recent progress in ML. In these lectures we took the perspective that to understand ML, we must understand neural networks, and built our study around three pillars: the expressivity, statistics, and dynamics of neural networks.

Let's recap the essentials and then provide some outlook. In Section <ref> on expressivity, we asked “How powerful is a NN?" We presented the Universal Approximation Theorem (UAT) and gave a picture demonstrating how it works. We also presented the Kolmogorov-Arnold Representation Theorem (KART), which recently motivated a new architecture for deep learning where activations are learned and live on the edges. We wonder:
OutlookCybenko's UAT was around for over 20 years before the empirical breakthroughs of the 2010's. Existence of good approximators wasn't enough, we needed better compute and optimization schemes to find them dynamically.  However, given tremendous progress in dynamics, should we systematically return to the approximation theory literature, to help motivate new architectures, as e.g., in KART and KAN?

In Section <ref> on statistics, I reminded the reader that a randomly initialized NN is no more fundamental than a single role of the dice. It is a random function with parameters that is drawn from a statistical ensemble of NNs, and we asked “What characterizes the statistics of the NN ensemble?” I reviewed a classic result of Neal from the 90's that demonstrated that under common assumptions a width-N feedforward network is drawn from a Gaussian Process — a Gaussian density on functions — in the N→∞ limit. I explained that work from the last decade has demonstrated that this result generalizes to many different architectures, as a result of the Central Limit Theorem. Non-Gaussianities therefore arise by violating an assumption of the Central Limit Theorem, such as N→∞ or statistical independence, and by violating them weakly the non-Gaussianities can be made parametrically small. In field theory language, NNGPs are generalized free field theories and NN Non-Gaussian processes are interacting field theories. I also introduced a mechanism for global symmetries and exemplified many of the phenomena in the section.

In Section <ref> I presented some essential results from ML theory on NN dynamics, asking “How does a NN evolve under gradient descent?” When trained with full-batch gradient descent and a particular normalization, the network dynamics are governed by the Neural Tangent Kernel (NTK) that in general is an intractable object. However, in N→∞ limits it becomes a deterministic function and the NTK at initialization governs the dynamics for all time. An exactly solvable model with mean-squared error loss is presented that includes the mean prediction of an infinite number of infinitely wide neural networks trained to infinite time. The mean network prediction is equivalent to kernel regression, and the kernel communicates information from train points to test points in order to make predictions. However, from this description of the NTK a problem is already clear: nothing is being learned, and in particular late-time features in the hidden dimensions are in a local neighborhood of their initial values.  I showed how a detailed N-scaling analysis allows one to demand that network features and predictions update non-trivially, leading to richer learning regimes known as dynamical mean field theory or the maximal update parameterization.
OutlookThese recent theories of NN statistics and dynamics sometimes realize interesting toy models, but with known shortcomings from a learning perspective, though the N-scaling analysis in the feature learning section is promising. Surely there is still much to learn overall, e.g., we have said very little in these lectures about architecture design principles. For instance, some architectures, such as the Transformers <cit.> central to LLMs, are motivated by structural principles, whereas others are motivated by theoretical guarantees. 
 
 What are the missing principles that combines statistics, dynamics, and architecture design to achieve optimal learning?




Finally, in Section <ref> I explained how neural networks provide a new way to define a field theory, so that one might use ML theory for physics and not just physics for ML theory. This NN-FT correspondence is a rethink of field theory from an ML perspective, and I tried to state clearly about what is and isn't known about obtaining cherished field theory properties in that context. For example, I explained how by engineering a desired NNGP, such as the free scalar, one may do an operator insertion in the path integral and interpret the associated interactions as breaking the statistical independence necessary for the CLT. By such a mechanism, one may engineer ϕ^4 theory in an ensemble of infinite width neural networks.
OutlookCan NN-FTs motivate new interesting physics theories or provide useful tools for studying known theories?

Our discussion in this Section is directly correlated with the topics of these lectures. It reviews recent theoretical progress, but the careful reader surely has a sense that theory is significantly behind experiment and there is still much to do. It is a great time to enter  the field, and to that end I recommend carefully reading the literature that I've cited, as well as the book Geometric Deep Learning <cit.> that covers many current topics that will be natural to a physicist, including grids, groups, graphs, geodesics, and gauges. Many aspects of that book are related to principled architecture design, which complements the statistical and dynamical approach that I've taken in these lectures. I also recommend the book Deep Learning <cit.> for a more comprehensive introduction to the field.

I end with a final analogy between history and the current situation in ML. We are living in an ML era akin to the pre-PC era in classical computing. Current large language models (LLMs) or other deep networks are by now very powerful, but are the analogs of room-sized computers that can only be fully utilized by the privileged. Whereas harnessing the power of those computers in the 60's required access to government facilities or large laboratories, only billion dollar companies have sufficient funds and compute to train today's LLMs from scratch. Just as the PC revolution brought computing power to the masses in the 80's, a central question now is whether we can do the analog in ML by learning how to train equally powerful small models with limited resources. Doing so likely requires a deeper understanding of theory, especially with respect to sparse networks.



Acknowledgements: I would like to thank the students and organizers of TASI for an inspiring scientific environment and many excellent questions. I am grateful for the papers of and discussions with friends whose works have contributed significantly to these lectures, especially Yasaman Bahri, Cengiz Pehlevan, and Greg Yang.  I'd also like to thank my collaborators on related matters, including Mehmet Demirtas, Anindita Maiti, Fabian Ruehle, Matt Schwartz, and Keegan Stoner. I would like to thank Sam Frank, Yikun Jiang, and Sneh Pandya for correcting typos in a draft. Finally, thanks to GitHub Copilot for its help in writing these notes, including the correct auto-completion of equations and creation of  figures! Two-column landscape mode was chosen to facilitate boardwork, but I think it also makes a pleasant read, HT @ <cit.>. I am supported by the National Science Foundation under CAREER grant PHY-1848089 and
Cooperative Agreement PHY-2019786 (The NSF AI Institute for Artificial Intelligence and Fundamental
Interactions).



Disclaimers: In an effort to post these notes shortly after TASI 2024, there are more typos and fewer references than is ideal. Other references and topics may be added in the future to round out the content that was presented in real-time. I also struck a playful tone throughout, because lectures are supposed to be fun, but may have overdone it at times. I welcome suggestions on the above, but updates will still aim for clarity and brevity, targeted at HET students.



§ CENTRAL LIMIT THEOREM


Let us recall a simple derivation of the Central Limit Theorem (CLT), in order to better understand the statistics of neural networks. Consider a sum of random variables

    ϕ = 1/√(N)∑_i=1^N X_i,

with X_i = 0. The moments μ_r and cumulants κ_r are determined by the moment generating function (partition function) Z[J] = e^Jϕ and cumulant generating function W[J] = log Z[J], respectively, as 

    μ_r     = (d/dJ)^r Z[J]|_J=0
    κ_r     = (d/dJ)^r W[J]|_J=0.

If the X_i are independent random variables, then the partition function factorizes 
Z_∑_i X_i[J] = ∏_i Z_X_i[J], and the cumulant generating function of the sum is the sum of the cumulant generating functions, yielding

    W_∑_i X_i[J] = ∑_i W_X_i[J] 
    κ_r^∑ X_i = ∑_i κ_r^X_i.
 
If the X_i are identically distributed, then the cumulants κ_r^X_i are the same for all i and we account for the 1/√(N) appropriately, we obtain

    κ_r^ϕ = κ_r^X_i/N^r/2-1.

This yields 

    lim_N→∞κ_r>2^ϕ = 0,

which is sufficient to show that ϕ is Gaussian in the large-N limit. In physics language, cumulants are connected correlators, and (<ref>) means that Gaussian (free) theories have no connected correlators.

In neural networks we will be interested in studying certain Gaussian limits. From this CLT derivation, we see two potential origins of non-Gaussianity:

    
  * 1/N-corrections from appearance in κ_r^ϕ.
    
  * Independence breaking since the proof relied on (<ref>).








utphys




 Paper Title: MambaCapsule: Towards Transparent Cardiac Disease Diagnosis with Electrocardiography Using Mamba Capsule Network.


1
.001


 





mode = title]MambaCapsule: Towards Transparent Cardiac Disease Diagnosis 
with Electrocardiography Using Mamba Capsule Network                      




























1]Yinlong Xu[
orcid=0009-0008-1987-7562
]






22321336@zju.edu.cn








[1]
    organization=College of Computer Science & Technology, Zhejiang University,
    city=Hangzhou,
    country=China
[2]
    organization=Department of Gastroenterology, First Hospital of Quanzhou Affiliated to Fujian Medical University,
    city=Quanzhou,
    country=China
[3]
    organization=School of Public Health, Zhejiang University,
    city=Hangzhou,
    country=China
[4]
    organization=State Key Laboratory of Transvascular Implantation Devices of The Second Affiliated Hospital and Liangzhu Laboratory, Zhejiang University School of Medicine,
    city=Hangzhou,
    country=China


[5]
    organization=Innovation Institute for Artificial Intelligence in Medicine, Zhejiang University,
    city=Hangzhou,
    country=China
    
[6]
    organization=School of Medicine, Stanford University,
    country=USA
    
[7]
    organization=School of Computer Engineering and Science, Shanghai University,
    city=Shanghai,
    country=China

[8]
    organization=College of Future Industry, Gachon University,
    country=Korea


  

2]Xiaoqiang Liu
liuxiaoqianghusina@163.com


1]Zitai Kong
kongzitai@zju.edu.cn

3]Yixuan Wu
wyx_chloe@zju.edu.cn

4]Yue Wang
ywang2022@zju.edu.cn


6]Yingzhou Lu

lyz66@stanford.edu

7,8]Honghao Gao
gaohonghao@shu.edu.cn
honghaogao@gachon.ac.kr

3]Jian Wu
wujian2000@zju.edu.cn

5]Hongxia Xu
Einstein@zju.edu.cn
[1]



[cor1]Corresponding author

















§ ABSTRACT 
Cardiac arrhythmia, a condition characterized by irregular heartbeats, often serves as an early indication of various heart ailments. With the advent of deep learning, numerous innovative models have been introduced for diagnosing arrhythmias using Electrocardiogram (ECG) signals. However, recent studies  solely focus on the performance of models, neglecting the interpretation of their results. This leads to a considerable lack of transparency, posing a significant risk in the actual diagnostic process. To solve this problem, this paper introduces MambaCapsule, a deep neural networks for ECG arrhythmias classification, which increases the  explainability of the model while enhancing the accuracy.Our model utilizes Mamba for feature extraction and Capsule networks for prediction, providing not only a confidence score but also signal features. Akin to the processing mechanism of human brain, the model learns signal features and their relationship between them by reconstructing ECG signals in the predicted selection. The model evaluation was conducted on MIT-BIH and PTB dataset, following the AAMI standard. MambaCapsule has achieved a total accuracy of 99.54% and 99.59% on the test sets respectively. 
These results demonstrate the promising performance under the standard test protocol.


















Mamba Capsule ECG Classification Explainable AI



[
    [
    August 5, 2024
==================





§ INTRODUCTION

Cardiac arrhythmia refers to the disruption of the cardiac rhythm caused by abnormal cardiac electrical activity <cit.>. For the reason that arrhythmia serves as an indicator to advent heart consequence, its accurate detection and classification are imperative for mitigating potential risks and complications. Electrocardiogram(ECG) is widely used for recording the electrical cardiac activity<cit.>. It provides critical information about the cardiac function and health by measuring the cardiac electrical signals during each heartbeat cycle, which makes it a reliable tool for diagnosing heart health.

Recent researches have shown great advantages of deep learning in arrhythmia classification<cit.>, leveraging the information contained in ECG signals. The development of powerful and effective deep learning methods has the potential to significantly improve the accuracy and efficiency of arrhythmia diagnosis, and ultimately leads to better patient outcomes.  

Although the numerous previous models have made substantial progress in arrhythmia classification, most of them solely focus on the performance  neglecting the interpretation of their results, which leads to a considerable lack of transparency and poses a low reliability in the actual diagnostic process<cit.>.

To alleviate this limitation, we proposed a encoder-decoder based neural network called MambaCapsule in this paper. Inspired by the mechanism of brain processing<cit.>, which tells what features exactly the eyes see and tells the reason why the brain chooses to classify one item to this label not the other one, we novelly designed our model to implement both processes by using the main architecture for prediction and a reconstruct part for explaination. Our evaluation was carried on MIT-BIH dataset<cit.> and PTB<cit.> dataset, reached 99.54% and 99.59% accuracy. By applying a reconstruct network, we could intuitively know what the model sees and why it makes that decision.

The main contributions of this paper are as follows:


=0pt

  ∙ We introduced a novel encoder-decoder based network architecture based on Mamba<cit.> and Capsule networks<cit.>, changing the form of the output from the probability sequence to capsule vector sequence, which contains both the classification and feature attribute.

  ∙ We applied a downstream network to reconstruct the ECG signal. The reconstructed signal can be used to explain what our model learns and the reason why our model makes its prediction.

  ∙ We applied a fusion states SSM architecture in process of feature extraction. The evaluation indicators show the feature extracted has considerable advantages on the MIT-BIH and PTB datasets.
  



§ RELATED WORK



 §.§ Mamba Nerual Network

Mamba has made excellent performance in numerous traditional fields since it was proposed by <cit.>. Different from traditional and the most popular structure Transformer, the advantages of Mamba are mainly reflected in the computational efficiency and the ability to deal with long sequences. For the reason above, many researches have been carried to replace traditional components with Mamba. <cit.> proposed Graph-Mamba, which improved the remote context modeling ability of attention mechanism by integrating Mamba block and node selection mechanism, and solved the problems of high computing cost and limited scalability of attention mechanism. 
<cit.> proposes a new model-based meta-reinforcement learning method, achieving higher returns and better sample efficiency (up to 15x) with little need for hyper-parameter tuning.
<cit.> proposes a multi-modal large language model VL-Mamba based on state space model, demonstrating competitive performance on a variety of multi-modal benchmark tasks and the great potential of state-space models in multi-modal learning.

Mamba also show its advantages in vision tasks <cit.>. <cit.> proposed a generic vision backbone with bidirectional Mamba blocks (Vim) to solve the location sensitivity and global context requirements of visual data. Vim performs well in various tasks, with significant improvements in compute and memory efficiency. <cit.> introduced VideoMamba into video understanding task. Its linear-complexity operator enables efficient long-term
modeling, which is crucial for high-resolution long video understanding.




 §.§ Capsule Neural Network

Capsule network was first proposed in a 2D handwritten digital image classification task and has excellent performance in the view-invariance <cit.>. Later on, it was widely applied to point cloud <cit.>, image processing <cit.>, and tabular learning <cit.>, graph learning <cit.>. Specifically, <cit.> proposed a multi-scale capsule network that is more robust and efficient for feature representation in image classification, demonstrating a competitive performance on FashionMNIST
and CIFAR10 datasets. <cit.> introduce
a capsule network that can detect various kinds of attacks. The model uses many fewer parameters than traditional convolutional neural networks with similar performance.

Capsule neural network also has a intrinsic advantage for explaining its results. For instance, <cit.> produced a sentiment capsule to identify the informative logic unit and the sentiment based representations in user-item level for rating prediction. By introducing capsules in different layers, <cit.> demonstrated the possibility of transforming deep networks into transparent networks. This work helps make DNNs' internal processes more understandable and provides the ability to explain their decisions.




 §.§ ECG based Cardiac Disease Diagnosis

Numerous researches have been carried in the field of ECG signal classification and the most challenging part of it is to extract the features from the signal. The feature extraction methods have experienced from the manual features and automatic features. Manually designed features extracting usually applied by traditional ML methods like Support Vetor Machine <cit.> and Random Forest <cit.>.
<cit.> proposed an effective methods to extract the abnormal beat eigenvectors of low-dimensional ECG and classify them using support vector machine. <cit.> proposed a ML model using multi-domain features combined with least-square support vector machine (LS-SVM), which extracts the time and frequency domain features and selects the most relevant ones.

With the success of deep learning in vision and NLP fields, its ability has been admitted and has been applied to many other fields, including the ECG signal classification <cit.>. 
<cit.> proposed a 1D residual CNN based on deep residual networks with a 31-layer one-dimensional residual convolutional neural network combined with 2-lead ECG signals for the classification.
<cit.> proposed a novel model combining residual networks, compressed activation blocks and bidirectional short-duration memory networks. It was evaluated on multiple databases and the results showed that the framework performed well especially for a small number of categories.

Although these methods have their own way to explain the process of the prediction, their lack of accuracy and inadequacy of features extracting hinder their practical applications. Some efforts also focused on the class imbalance problom on ECG datasets. For instance, <cit.> introduced a deep learning model called ECGTransForm ,which combines multi-scale convolution, channel calibration module and two-way Transformer mechanism to improve the ability to capture spatio-temporal features in ECG data and solves class imbalance problem by introducing context-aware loss function. However, these methods were employed through simple data reinforcement using statistical methods, the reinforced data benefits to model performance but has no specific physiological information through the process of the reinforcement.
 


§ METHODS




 §.§ Mamba

State space models (SSMs) <cit.> are defined as 1D-time-invariant systems, which serve as the predecessor of Mamba. These systems aim to map a 1D input sequence x(t)∈ℝ^L to a 1D output sequence y(t)∈ℝ^L through a hidden state h(t)∈ℝ^N, which can be represented as the following equations:

    h(t) = Ah(t)+Bx(t)
       y(t)=Ch(t)


where A∈ℝ^N× N, B∈ℝ^N× L, C∈ℝ^L× N. Considering that deep neural networks operate in discrete space, researchers applied a zero-order preservation on the matrix A and B, turning the equations to the form as:

    h_t=Ah_t-1+Bx_t
       y_t=Ch_t


Where A=exp(△ A) and B=(△ A)^-1(exp(△ A)-I)· (△ B) for△ represents the time step in the continuous space. SSMs also code the remote dependency by initializing matrix A as HiPPO<cit.> and allow training in the form of a convolution kernel to accelerate the training process. However, the matrix A, B and C do not change whatever the input data are, leading to problem of low sensitivity. To solve the problem, Mamba applies a selective method to SSMs, which makes the matrix B and C and △ adapt themself based on the input as follows:

    B = s_B(x)
       C = s_C(x)
       △ = τ_△(Parameter+s_A(x))


Where s_B, s_C and s_A are linear projections, and τ_△ is a SoftPlus function. The matrix A does not depend on the input directly, it is influenced by the △ in the process of the zero-order preservation, making all main parameter inputs adaptable<cit.>.






 §.§ Capsule Networks

The whole network contains three parts: the convolution network, the capsule network and the dynamic routing. The difference between the capsule network and the traditional neural network is that the unit of the neural network is a scalar neuron x∈ℝ, while the unit of the capsule network is a capsule, which is a vector x∈ℝ^N.

In a capsule network, there are many capsules in one layer, and each capsule contains two types of information: the given vector's orientation and the vector's length as the probability of the capsule, the next capsule first applies a linear transformation of the previous capsule output as:

    û_j|i,L+1=W_ij, Lu_i, L


where u_i,L∈ℝ^D is the i-th output capsule at layer L, W_ij,L∈ R^D̂× D serves as the transformation matrix and the û_j|i, L+1∈ℝ^D̂ represents the contributes from the parent i-th capsule to the j-th one at layer L+1. The model then gets the weight of each contributes through a clustering like routing algorithm. At present, the popular routing algorithms include dynamic routing algorithm and self-routing algorithm. This paper adopts dynamic routing algorithm, which obtains the final weight coefficient through several rounds of iteration. In each iterations, the weight coefficient c is:

    c_ij=exp(b_ij)/∑_k exp(b_ik)


In the first iteration the b_ij is initialized to 0 and will be updated at the end of each iteration. Then the coefficient is used to calculate the temp capsule:

    s_j=∑_i c_ijû_j|i


Where s_j∈ℝ^D̂ serves as a temp result for an iteration, the capsule network designs a special activation function to transform the length of the capsule into [0-1] as:

    v_j = ||s_j||^2/1+||s_j||^2s_j/||s_j||


Where v_j∈ℝ^D̂ serves as the real output capsule in each iteration. The equation <ref> separates the long and short length of the capsules in the dim of 0 to 1, making the long capsule close to 1 and short capsule close to 0. Then the b_ij is updated as:

    b_ij = b_ij + v_jû_j|i


The equation <ref> serves as an clustering-like algorithm for after each iteration the b_ij grows bigger in capsule whose position is  closest to the others. And after the setting iterations have done, the v_j will serve as the output of the capsule networks.



 §.§ Model Architecture


In this section, we present a classification model based on Mamba and Capsule networks. The overall framework, as shown in Figure <ref>, consists of three main parts: the feature network, the capsule network and the reconstruction network.  Before be fed into the feature network, the input ECG signal
is firstly up-projected to contain scope knowledge not a signal sequence, the input shape changes from x∈ℝ^batch, length to x̂∈ℝ^batch, length, dim.

Then N Mamba Layers constitute a pipeline to extract features from the x̂ and each Layer contains a Mamba Block and a Layer Norm. Previous work typically apply a single state Mamba as the encoder backbone, however, these fails to extract wider and narrower features. To solve this problem, we propose a fusion states SSM to construct the Mamba Block. For the i-th Block, it first applies a 1D-convolution on the upstream i-1-th feature sequence x_i, which does not change the dimension of the sequence. Then the sequence will be handled by m different SSM Blocks. The different SSM Blocks focus on different scales of time step and have different convolution shape to obtain more abundant and wider features. Then the features will be down sampled and then linked together to add the x_i in the same shape. The whole process can be formulated as:

    x̂_i = Conv_1D(x_i)
        u_j = DownSample_j(SSM_j(x̂_i))
        x_i+1 = Concat(∑_j^mu_j) + x_i


Where u_j ∈ℝ^batch, length, dim/mand the m is the factor of the dim to assure x_i+1∈ℝ^batch, length, dim.

The feature network extracts the signal features and sends the features to the capsule network. The capsule network outputs it in the form of capsules in the end. 




The loss of our model consists of two parts, the marginal loss of focusing on model output and the label, and the reconstruct loss focusing on the reconstructed signal and the original signal.

Since our output is in the form of a capsule network, we must first take the geometric length of the output as the probability of the classification label, and then the difference between the set positive label threshold and negative label threshold is made. The final form of margin loss function is as follows:


    Loss_margin =    ∑_k T_kmax(0, m^+-||v_k||)^2+
       λ(1-T_k)max(0, ||v_k||-m^-)^2


Where v_k∈ℝ^D is the k-th output capsule represents one class of the classification, T_k=1 if the label of the input is k, and m^+=0.9, m^-=0.1 at the beginning. λ is used to prevent the lengths of vectors from shrinking too much, which is set to 0.5 in the beginning.

Reconstruct loss needs to choose the label with the highest probability or we choose it for other reasons. Then we put the output into the reconstruct model, which is a 3 fully connection layers model with Relu activation. After that, calculate the MSEloss of the reconstructed signal output of the reconstruct model and the original input signal, the reconstruct loss is formulated as follows:

    x̂_reconstruct = Model(v_choose)
        Loss_reconstruct=MSELoss(x̂_reconstruct, x_origin)




§ MODEL SETUP AND EVALUATION



 §.§ Training Strategy

Since the output probability density of the capsule network is not the same as that of the traditional neural classification network, and the bias can directly affect the output, we adjusted bias by cosine schedule during the training process, so that the threshold of bias kept increasing. At the same time, due to the slow training speed of capsule network in the initial stage of training, the cosine annealing algorithm after linear increase is adopted to change the learning rate. The changes of bias and learning rate are shown in the figure.<ref>





 §.§ Performance Metrics

To evaluate the model, we applied five performance indexes: accuracy(ACC), sensitivity (SEN), F1-score(F1), precision (PPV) and specificity (SPEC). These indexes are defined as follows:


    ACC=TP+TN/TP+TN+FP+FN
       SEN=TP/TP+FN
       F1=2×ACC× SEN/ACC+SEN
       PPV=TP/TP+FP
       SPEC=TN/TN+FP


In these formulas, TP represents the number of true positives (correctly labeled positive); TN represents the number of true negatives (correctly labeled negative); FP represents the number of false positives (incorrectly labeled positive); and FN represents the number of false negatives (incorrectly labeled negative).


 

 §.§ Dataset

 Our training and testing datasets are preprocessed in the same routine and are randomly splited in a ratio of 0.8. The detailed description of the dataset is in Table<ref>:


  §.§.§ MIT-BIH Arrhythmia Dataset

The MIT-BIH arrhythmia dataset <cit.> is widely used for arrhythmia classification in recent researches. The dataset contains 48 half-hour two-channel ECG recordings digitized at 11 bit resolution in 10 mV range and a rate of 360 samples per second per channel. Two cardiologists annotated each record separately (about 110,000 annotations in total). MIT-BIH has annotated 41 categories which can be converted to five-classes classification problems following the ANSL/AAMI EC57: 2012 (The Association for the Advancement of Medical Instrumentation) standard. These distinctive classes are: Normal Sinus Rhythm (N), Supraventricular Premature or Ectopic Beat (S), Ventricular Premature or Ectopic Beat (V), Fusion of Ventricular and Normal Beat (F), Unknown Beats (Q). Some researches were carried on three or four classes due to imbalanced sample size, making the comprehensive evaluation difficult. In this paper, we chose to adapt to all the five classes.



  §.§.§ PTB Diagnostic ECG Database

The PTB diagnostic ECG database is a popular ECG recording database collected by the National Metrology Institute of Germany<cit.>. The dataset contains 549 records from 290 subjects (aged 17 to 87, mean 57.2; 209 men, mean age 55.5, and 81 women, mean age 61.6; ages were not recorded for 1 female and 14 male subjects). Each subject is represented by one to five records. Each record includes 15 simultaneously measured signals: the conventional 12 leads (i, ii, iii, avr, avl, avf, v1, v2, v3, v4, v5, v6) together with the 3 Frank lead ECGs (vx, vy, vz). Each signal is digitized at 1000 samples per second, with 16 bit resolution over a range of ± 16.384 mV. The PTB dataset consists of two main classes, i.e., (1) Normal (N), which represents normal ECG recordings, and (2) the Myocardial Infarction (M), which represents ECG recordings that exhibit signs of myocardial infarction, indicating the presence of a heart attack.









§ EXPERIMENTAL RESULTS



 §.§ Performance Evaluation


Many previous work focused on only four or less classes performance, aiming to reduce the perplexity of the problem or concentrate on specific arrhythmia types. However, it leads to great difficulties to compare the ability and universality of different models. In our work, we operate our evaluation on five full classes.



Table <ref> shows the confusion matrix of our best model prediction. And the detailed evaluations on each class of the MIT-BIH dataset are presented in Table<ref>. The result shows our model has excellent ability of discrimination, especially in class N, V and Q the Sensitive, Precision and Specificity reached almost 100%, which can be attribute the large training sample size. The performance of Supraventricular Premature or
Ectopic Beat (S) is not so outstanding, which can be the result of lack of training samples and the Fusion of Ven-tricular. The average performance of Normal Beat (F) can be the reason its original features resemble to the Normal class (N).





Table <ref> and Table <ref> also show our model great performance on PTB dataset. The confusion matrix indicates that our model predicts nearly all of positive and Negative samples, with an macro-avg accuracy of 99.59%.















The performance of our model is also compared with several previous studies, which is presented in Table <ref>, presenting comparative advantages over the other architectures. And the performance of each label can be seen in Table <ref>, which can be observed from the result that our model has considerable advantages compared with others in majority classes.



 §.§ Ablation Study

The ablation experiment was carried out to test the effectiveness of our model structure. Since Mamba was mainly used as the encoder and capsule networks as the decoder in our model, the ablation experiment was carried out in these two aspects on MIT-BIH dataset.



  §.§.§ Encoder Structure

This experiment aims to compare the performance of different model structures, including the number of stacked mamba layers, the number of mamba states, the steps per layer, whether to adopt the layernorm and dropout structures, and the use of different models as encoders.

In order to assess the number of mamba states and steps in each layer, we adjusted these parameters while keeping the encoder layers fixed at 4. The experimental results depicted in the figure<ref>demonstrate a significant impact of varying numbers of states and steps on the outcomes. Moreover, a layer composed of diverse mamba blocks outperforms a single layer due to its ability to focus on different ECG data features across various time scales. Consequently, this wider temporal coverage achieved by combined blocks leads to enhanced performance. However, if the states are too wide it will miss significant information.

To assess the impact of encoder layers on model performance, we held the parameters of each layer constant and varied the number of encoder layers from 2 to 8 while considering whether to implement layer normalization and dropout. The corresponding results are depicted in the figure <ref>. From our analysis of training time and memory usage for a four-layer encoder, it is evident that the metric increases with additional layers; however, model performance does not improve proportionally with increased depth. In fact, the optimal performance is achieved at six layers due to limitations in information capacity exceeding what can be learned from available data as well as increased computational demands associated with larger models.

In order to evaluate the effect of different types of encoder, we have also tried to use resnet, transformer and cnn as encoders. The experimental results are shown in the figure<ref>. The results show that mamba has the best effect when it is used as encoder.





  §.§.§ Decoder Structure

The purpose of this experiment was to compare the changes in the model training and the performance caused by using capsule instead of traditional decoder.

We tried to replace the capsule network with MLP after freezing encoder parameters. Experiments showed that there was no significant difference between the capsule network and MLP under the same encoder with F1-score 93.50 compared to 93.62, but the training time of capsule network was longer than that of MLP. This is determined by the routing structure of the capsule network and the number of parameters. 



 §.§ Explainable Study

The interpretability of this paper is unique from other recent papers on ECG diagnosis, mainly through the reconstruction of ECG signals in different ways to explain what knowledge the model has learned on MIT-BIH dataset. For the reason of the limitation of the memory, we only consider P-wave and R-wave in our reconstruction, discarding the T-wave which is far from the center of the signal.



  §.§.§ Feature Relation Mining


When processing the ECG graphs, the human brain not only attends to focus on individual wave peaks but also considers the interrelationship between them. The view invariance of capsule network aligns remarkably well with this characteristic, which is why we selected it as the decoder. In our experiment, we conducted forward and backward disturbance on the input data to observe the disturbance reconstruction of ECG signals using our model and DAE model. The experimental results are depicted in the figure <ref>, where left and right shift represent forward and backward shift respectively. Repeated reconstructions of ECG signals by MambaCapsule exhibit remarkable consistency, indicating that Mambacapsule has learned not only the intrinsic features but also their interrelationships. Conversely, this characteristic is absent in the reconstructed ECG signal produced by DAE model.



  §.§.§ Cardiac Disease Manifestation Type Inspection


When human brain categorize an item, it can discern the reasons for its classification and differentiation. Due to the inherent output characteristics of Capsule network, the output not only encompasses the probability density of each category but also encapsulates the distinctive features associated with that particular category. Consequently, by reconstructing ECG signals from unified inputs belonging to different categories, we are able to precisely comprehend why the model deems a certain category more probable. The experimental results are depicted in Figure <ref>. In Figure <ref>, both the ECG signal and model prediction are labeled as N, where P, R wave can be observed clearly and can be distinguished it represents a normal signal. However in Figure <ref>, which is reconstructed by the label S and the length of the capsule is adjusted to be equal to the capsule of label N. The false reconstruction represents a vanish in P-wave, which is a symbol of Supraventricular Premature or
Ectopic Beat. The reconstructions of N and P labels demonstrate an ablity of knowledge explaination and thus, each capsule can be seen as the information collection of different labels and the length of it is how much the information is concluded in the signal.



§ CONCLUSION


In this work, we have proposed an effective-while-explainable solution to ECG classification by introducing MambaCapsule, which emphasizes the capability of multi-scale Mamba in feature extracting and the interpretability of the form of Capsule network. By applying evaluation and reconstruction, the result of our model reveals great advantages in both classification score and mechanism explainablity. However, it's necessary to realize the limitation of our model, a the long training time and big memory occupied. In our future work, we aim to address these challenges by applying pre-trained parameters and optimize routing structure in Capsule network. As an automatic heart rhythm abnormality detection system, our model can offer doctors and patients accurate and reliable results, which have a wide range of application prospects.






cas-model2-names









 Paper Title: Learning Ordinality in Semantic Segmentation.















Learning Ordinality in Semantic Segmentation
    Rafael Cristino^1 Ricardo P. M. Cruz^1,2* Jaime S. Cardoso^1,2*
^1 Faculty of Engineering, University of Porto
^2 INESC TEC, Porto
 rpcruz@fe.up.pt, jaime.cardoso@fe.up.pt
    
===============================================================================================================================================================================



§ ABSTRACT 
Semantic segmentation consists of predicting a semantic label for each image pixel. Conventional deep learning models do not take advantage of ordinal relations that might exist in the domain at hand. For example, it is known that the pupil is inside the iris, and the lane markings are inside the road. Such domain knowledge can be employed as constraints to make the model more robust. The current literature on this topic has explored pixel-wise ordinal segmentation methods, which treat each pixel as an independent observation and promote ordinality in its representation. This paper proposes novel spatial ordinal segmentation methods, which take advantage of the structured image space by considering each pixel as an observation dependent on its neighborhood context to also promote ordinal spatial consistency.
When evaluated with five biomedical datasets and multiple configurations of autonomous driving datasets, ordinal methods resulted in more ordinally-consistent models, with substantial improvements in ordinal metrics and some increase in the Dice coefficient. It was also shown that the incorporation of ordinal consistency results in models with better generalization abilities.




§ INTRODUCTION


Semantic segmentation, or scene parsing, is the task of attributing a semantic label to each of the pixels in an image, resulting in a segmentation map. One common problem with these deep learning segmentation models is the lack of generalization ability, which means that the network fails to make appropriate predictions when parsing a situation that did not occur in the training dataset <cit.>. A hypothesis is that the neural network model does not have the necessary intrinsic domain knowledge of the task – it failed to infer appropriate high-level relations from the data used to train it. 

In many situations, there is an explicit ordering between the output classes, and by training the network with methods that uphold the ordinal constraints, the network may be able to learn better higher-level concepts, such as the ordinal relation between different objects (the lane marks and are inside the lane, etc) and the relative placement of objects (the sidewalk is to the side of the road, etc.).

Typically, ordinal problems have mostly been studied in the context of (image) classification <cit.>, where the task is to classify an observation (image) as one of 𝒞_1 ≺𝒞_2 ≺…≺𝒞_K ordered classes (for example, the severity of a disease), as opposed to nominal classes in the case of classic nominal classification. To our knowledge, only one work <cit.> attempted to introduce such ordinal relations for semantic segmentation, where 𝒞_1 ⊃𝒞_2 ⊃…⊃𝒞_K. However, the work does not explicitly perform spatial segmentation: it would be expected that an area labelled as 𝒞_k would only have a direct boundary to the areas segmented as 𝒞_k-1 and 𝒞_k+1, as exemplified in fig:ordseg_viz_example.






    
  * The ordinal relation between different objects, the lane marks and the lane itself – the lane marks are inside the lane;
    
  * The model should be more undecided between certain pairs of object classes, indecision between car/truck rather than between car/person;
    
  * The relative placement of objects, the sky or the sidewalk.





The transition from image classification to segmentation inherently evolves the prediction task from classifying independent samples (each image) to classifying structured dependent samples (each pixel in an image). The former can only take action in the representation space of each pixel, while the latter can also consider the structured space and the relations between the samples. Further developing this idea, the structured image space can be generalized to a graph, where each pixel is a vertex and is connected to its adjacent pixels.

The contributions presented in this paper can be summarized in:


  * The task of ordinal regression is formalized into ordinal consistency for each individual decision and ordinal consistency for the entire structure of the input, i.e. spatial consistency in the case of images (section <ref>).

  * Focusing on the latter case, two loss regularization terms (section <ref>) and one metric are proposed (section <ref>).

  * A thorough evaluation is performed for five biomedical datasets and an autonomous driving dataset.


The document is structured as follows: sec:sota introduces the state of the art; sec:proposal delineates the proposal; sec:experiments describes the conducted experiments; sec:results shows and analyzes the experimental results; and sec:conclusion revisits the proposed methods and summarizes the key conclusions.





§ STATE OF THE ART



Cross entropy is one of the most commonly used loss functions for image classification and segmentation problems. Defining cross entropy for a semantic segmentation problem,

    CE(y_n, p̂_n) = - 1/HW∑_i=1^H ∑_j=1^W ∑_k=1^K 1(y_n,i,j=k) log(p̂_n,k,i,j),

where p̂ is the model output as probabilities, in shape (N, K, H, W), where N is the batch size, K is the number of classes, and (H, W) are, respectively, the height and width of each image; y is the ground truth segmentation map, in shape (N, H, W), where each value y_n,i,j corresponds to the ground truth class k ∈{1,⋯, K} of the pixel at position (i,j) of observation n; and 1(x) is the indicator function of x.

Cross-entropy maximizes the probability of the ground truth class for each pixel in the observation, ignoring the distribution of the predictions for the other classes. This is a potential area where new loss functions can improve by restricting the probabilities of the non-ground truth class according to the domain knowledge of the task.



 §.§ Ordinal Classification Methods
 

Various research works seek to imbue deep neural networks with ordinal domain knowledge in the ordinal classification domain.



  §.§.§ Unimodality
 

The promotion of unimodality in the distribution of the model output probabilities has achieved good results in ordinal classification tasks <cit.>. This can be advantageous in ordinal problems because the model should be more uncertain between ordinally adjacent classes. For example, it would not make sense for a model to output a high probability for Low and High risk of disease, but a small probability for Medium risk of disease. fig:unimodality_pixel shows the difference between multimodal and unimodal distributions.



To promote unimodal output probability distributions, some authors have imposed architectural restrictions, which restrict the network output to a Binomial or Poisson probability distributions <cit.>. Other authors have promoted ordinality by penalizing the model when it outputs a non-unimodal distribution through augmented loss functions. One such case is the CO2 loss, which penalizes neighbor class probabilities if they do not follow unimodal consistency <cit.>,

    L_CO2(y_n, p̂_n) = L_CE(y_n, p̂_n) 
    + λ  L_O2(y_n, ŷ_n),

where L_O2 is the regularization term,

    L_O2(y_n, p̂_n)    = ∑_k=1^K-11(k ≥ y_n)  ReLU(δ + p̂_n,k+1 - p̂_n,k) 
       + ∑_k=1^K-11(k ≤ y_n)  ReLU(δ + p̂_n,k - p̂_n,k+1),

with δ being an imposed margin, assuring that the difference between consecutive probabilities is at least δ, and ReLU is defined as ReLU(x) = max(0, x).



  §.§.§ Ordinal Encoding


An approach to introducing ordinality to neural networks for classification involves regularizing the input data by encoding the ordinal distribution in the ground truth labels <cit.>. Defining k^⋆ as the ground truth class for a given sample, this input data encoding encodes each class as 1(k < k^⋆), whereas generic one-hot encoding encodes each class as 1(k = k^⋆).
This approach can be adapted for segmentation problems by similarly encoding the ground truth masks at a pixel level <cit.>. 

Using ordinal encoding for segmentation does not guarantee that the output probabilities are monotonous, the probability of ordinal class k, P_k, may be less than P_k+1. The consistency of the output class probabilities can be achieved by using,

    P(𝒞^+_k+1) = P(𝒞^+_k+1|𝒞_k^+) P(𝒞_k^+),
  
where P(𝒞^+_k+1|𝒞_k^+) is the (k+1)-th output of the network and P(𝒞_k^+) is the corrected probability of class k <cit.>. 

The state-of-the-art ordinal segmentation approaches have treated pixels as independent observations and promoted ordinality in their representation. However, this may be insufficient when applied to the structured image space, where pixels are dependent observations and a new level of consistency, spatial consistency, can be achieved.





§ PROPOSAL



Firstly, we present the formal foundation of our work (sec:foundation), then introduce the proposed ordinal segmentation methods, which are categorized into representation consistency (sec:proposal:pixel_wise), and structural consistency (sec:proposal:spatial). Finally, the ordinal segmentation problem is adapted to domains with arbitrary hierarchies (sec:proposal:hierarchies).



 §.§ Foundation



We start by recovering the definition of ordinal models, as introduced in <cit.>.
In a model consistent with the ordinal setting, a small change in the input data should not lead to a “big jump” in the output decision. Assuming f(𝐱) as a decision rule that assigns each input value 𝐱∈ℝ^d to the index ∈{1,2,…,K} of the predicted class, the decision rule is said to be consistent with an ordinal data classification setting in a point 𝐱_0 only if

    ∃ ε>0,  max_𝐱∈ℬ_ε(𝐱_0) f(𝐱)-min_𝐱∈ℬ_ε(𝐱_0) f(𝐱)≤1,

with ℬ_ε representing the individual feature-space neighborhood centered in 𝐱_0 with radius ε. Equivalently, the decision boundaries in the input feature-space 𝐱 should be only between regions of consecutive classes. Note that the concept of consistency with the ordinal setting is independent of the model type (probabilistic or not) and relies only on the decision region produced by the model. The state-of-the-art methods discussed in the previous Section focus only on this consistency (albeit often indirectly, by working on a related property, like the unimodality in the output probability space).



However, this consistency is only part of our knowledge in the ordinal segmentation setting.
In fig:ordinal-consistency, the feature description 𝐱∈ℝ at every pixel is mapped to the class according to the representation function

    f(x)=
    𝒞_1,  if    x<100
    𝒞_2,  if    100≤ x<200
    𝒞_3,  if    x≥ 200,

which is also depicted in fig:ordinal-consistency(a). Note that the model is ordinal-consistent in the pixel description space.
At most, a small change in x will change the decision to an adjacent class.

Consider now that the representation learned at each pixel v is given as 

    𝐱(v)=
    𝐱(v_0),      if   v∈ left half of the image 
    
    4𝐱(v_0),     if otherwise,

where 𝐱(v_0) is arbitrarily defined.
The model always sets the right half of the image to four times the values in the left half. Therefore, as illustrated in fig:ordinal-consistency(b), even being consistent at the pixel representation level, for some images (e.g., when the description of the pixels on the left takes the value 75), the decision will be 𝒞_1 for the left half and 𝒞_3 for the right half, which is to be avoided in an ordinal segmentation scenario. Therefore, we generalize the definition of ordinal-consistent models to models acting on structured observations, such as images or, more generally, graphs.

Consider each image as a graph 𝒢=(𝒱, ℰ), where 𝒱 denotes the set of graph vertices and ℰ⊂𝒱×𝒱 denotes the set of graph edges. 
Vertices in the image graph represent pixels, and because the values of a pixel are usually highly related to the values of its neighbors, there are undirected edges from a pixel to its neighboring pixels (often 4 or 8). The corresponding graph is then a 2D lattice.
Consider the goal of learning a function of signals/features which takes as input:


  * A feature description 𝐱_i = g(v_i) for every node v_i

  * A representative description of the graph structure 𝒢=(𝒱, ℰ)

and produces a node-level output f(𝐱_i).
Also, remember that the closed neighborhood of a vertex v in a graph 𝒢 is the subgraph of 𝒢, 𝒩[v], induced by all vertices adjacent to v and v itself, i.e., the graph composed of the vertices adjacent to v and v, and all edges connecting vertices adjacent to v.

We now define a model as ordinal consistent if the following two conditions are simultaneously met:


  * representation consistency: as before, if, for every point 𝐱_0 ∈ℝ^d, ∃ ε>0, max_𝐱∈ℬ_ε(𝐱_0) f(𝐱)-min_𝐱∈ℬ_ε(𝐱_0) f(𝐱)≤1, with ℬ_ε representing the feature-space neighborhood centered in 𝐱_0 with radius ε.

  * structural consistency: if, for every node v_i,
 max_v∈ N[v_i] f(g(v))-min_v∈ N[v_i] f(g(v)) ≤ 1

Figure <ref>(b) illustrates structural inconsistency and consistency – the representations of the pixels change smoothly over the image as given by

    𝐱(v)=
    𝐱(v_0),         if  v ∈left third of the image 
    𝐱(v_0) + 1,     if  v ∈middle third of the image 
    𝐱(v_0) + 2,     if  v ∈right third of the image.




 §.§ Representation Consistency for Ordinal Segmentation
 

In ordinal segmentation, ordinal representation consistency methods encompass those methods that act on the individual pixel representation 𝐱(v) and decision, they impose restrictions on the pixel, taking into account its own characteristics and disregarding the context of the neighboring pixels. Such methods include the ordinal pixel encoding and consistency methods discussed in the state-of-the-art analysis <cit.>. The previously introduced CO2 loss (<ref>) can be trivially adapted to segmentation by performing the regularization term for each pixel,

    L_O2(y_n, p̂_n) =    1/HW∑_i=1^W∑_j=1^H
    [ ∑_k=1^y_n,i,jReLU(δ + p̂_n,k,i,j - p̂_n,k+1,i,j) 
    
    +    ∑_k=y_n,i,j^K-1ReLU(δ + p̂_n,k+1,i,j - p̂_n,k,i,j) ].




 §.§ Structural Consistency for Ordinal Segmentation
 

The structural neighborhood concerns the neighborhood that relates one element with the others in the graph structure. In the case of semantic segmentation, the structural neighborhood is the set that contains all pixels connected to a given pixel. 



  §.§.§ Contact Surface Loss Using Neighbor Pixels
 





The contact surface loss is defined by penalizing the prediction of two neighboring pixels of non-ordinally adjacent classes. This penalization term is performed by multiplying the output probabilities for neighbouring pixels,

    L_CSNP(p̂_n,p̂_m) = ∑_n,m : 
    v_n∈ N [v_m]p̂_n^⊺ C p̂_m

where p̂_n^⊺ C p̂_n is a symmetric bilinear form, C is a K× K cost matrix, p̂_n∈ℝ^K is the vector of probabilities predicted at node v_n (similarly for 𝐩_m). C is defined as

    C_i,j = ReLU (|i-j| - 1) = 
    
    0,    if  |i-j|≤ 1 
    
    C_i,j = |i-j|-1,     otherwise,

to penalize ordinal-inconsistent neighboring probabilities,
where ReLU(·) is the Rectified Linear function, ReLU(x) = max(0,x). As an example, for K=4 classes, we obtain 

    C= [ 0 0 1 2; 0 0 0 1; 1 0 0 0; 2 1 0 0 ].

The intuition is to penalize high probabilities in spatially-close classes that are not adjacent.






  §.§.§ Contact Surface Loss Using the Distance Transform
 



Another approach is leveraging the distance transform, which is an image map where each value represents the distance from each pixel in the target image to its closest pixel with value 1, calculated with the customizable distance function d <cit.>. Defining the distance transform (DT) of the output probability map of class k, 

    = min_p_2 : p̂_n,k,p_2≥δ d(p_1, p_2),

where p = (i,j) and δ is the threshold parameter that selects the high-confidence pixels (typically 0.5), allowing the distance transform to be calculated for a high-certainty version of the output segmentation mask.




This provides a distance between pixels that have a high probability for a certain class and the closest pixel with a low probability of that class. That is, it provides an approximate distance between pixels of different classes. The model is trained to maximize this distance by multiplying the class probabilities map with the opposing class's distance transform,

    L_CSDT(p̂_n) = -1/|𝒮|∑_k_1, k_2 ∈𝒮 C_k_1, k_2   (p̂_n,k_1DT(p̂_n,k_2) 
        + p̂_n,k_2DT(p̂_n,k_1)),

where 𝒮 is the set of pairs (k_1, k_2) ∈{1,⋯, K} such that k_2-k_1 > 1.

At this stage, the  term maximizes the distance indefinitely. This is problematic because this may cause exploding distances, drawing the masks away from each other and possibly completely deviating from the ground truth. This can be solved by limiting the distance transform to a maximum distance, γ. This way, the loss only penalizes masks closer to each other than the γ value. For that reason, an updated distance transform was used, which saturates,

    ^γ = min (, γ).




 §.§ Partially Ordered Domains
 

The previous ordinal segmentation methods were proposed for ordinal domains with a (linear) total order in the set of classes, domains where 𝒞_1 ⊃𝒞_2 ⊃…⊃𝒞_K. This section proposes extending the previous work on ordinal losses to partially ordered output sets (the road may contain both lane marks and vehicles, but there is no relation between lane marks and vehicles).

Domains with a partial order in their set of classes must provide the segmentation methods with this partial order, the ordinal relations that motivate the ordinal constraints, as exemplified by Figure <ref>.



The ordinal segmentation metrics and losses proposed to total order sets can be extended to partially ordered sets.
Define first ℓ_m,n as the length of the shortest path from class m to class n in the Hasse diagram, or 0 if no such path exists. Note that ℓ_m,n≠ℓ_n,m and ℓ_m,n∨ℓ_n,m  = 0.

The previously introduced loss terms (L_O2, L_ and L_) are extended to arbitrary hierarchies as L_O2', L_' and L_'.

The L_O2' term can be redefined from (<ref>) by applying it between two neighboring classes, i.e. only when ℓ_m,n = 1, and if the edge (m,n) is part of a Hasse path that includes the ground-truth label k^⋆.

The C cost matrix in the L_' term can be defined as 

    C_m,n = ReLU(max(ℓ_m,n,ℓ_n,m)-1).
 

The adaptation L_' is similar to the adopted in L_O2'. 

Note that all these extensions revert to the base versions when applied to a totally ordered set.



 §.§ Evaluation Metrics




  §.§.§ Unimodal Pixels

We propose measuring the representation consistency using as a metric the percentage of Unimodal Pixels (UP), originally proposed by <cit.>, which consists of the fraction of times that the probability distribution produced by the model is unimodal.


  §.§.§ Contact Surface Metric

To evaluate the structural consistency, we propose to measure the percentage of ordinally invalid inter-class jumps between adjacent pixels, a metric for the contact surface between the masks of non-ordinally adjacent classes. Ordinally valid jumps are considered to be jumps between classes whose ordinal distance equals 1. If the ordinal distance between the classes of adjacent pixels exceeds 1, then that is an ordinally invalid jump. This requires that each pixel and its immediate neighborhood be examined during calculation. Defining the Contact Surface (CS) metric,

    CS(ŷ_n)    = 1/21 (CS_dx(ŷ_n)_i,j > 1)/1 (CS_dx(ŷ_n)_i,j > 0)
       + 1/21 (CS_dy(ŷ_n)_i,j > 1)/1 (CS_dy(ŷ_n)_i,j > 0),

where ŷ = _k=1^K(p̂), and CS_dx and CS_dy are the ordinal index variation, ordinal distance, from the current pixel (i, j) to the neighborhood, respectively, through the x and y axis,

    CS_dx(ŷ_n)_i,j = |ŷ_n,i, j - ŷ_n,i, j+1|
    CS_dy(ŷ_n)_i,j = |ŷ_n,i, j  - ŷ_n,i+1, j|







§ EXPERIMENTS
 



 §.§ Datasets



Various real-life biomedical datasets with ordinal segmentation tasks, where there is a clear ordering between classes, were identified from the literature <cit.>. table:sota:ordinal_datasets introduces the five biomedical datasets used to validate the proposed methods, along with a sample image and its corresponding segmentation mask.



To evaluate the ordinal methods on autonomous driving domains, the BDD100K <cit.> and Cityscapes <cit.> datasets were used:


  * BDD100K – is a multi-task, large-scale, and diverse dataset, obtained in a crowd-sourcing manner. Its images are split into two sets, each supporting a different subset of tasks: (1) 100K images – 100,000 images with labels for the object detection, drivable area, and lane marking tasks, and (2) 10K images – 10,000 images with labels for the semantic segmentation, instance segmentation, and panoptic segmentation tasks. The 10K dataset is not a subset of the 100K, but considerable overlap exists.

  * Cityscapes – is a large-scale and diverse dataset, with scenes obtained from 50 different cities. It provides 5,000 finely annotated images for semantic segmentation.


Two variants of the BDD100K dataset were considered: (1) BDD10K for the ordinary semantic segmentation task (10,000 images); and (2) BDDIntersected, which is the intersection of the 100K and 10K subsets and supports both the semantic segmentation and drivable area tasks (2,976 images). The models trained with BDD10K were subsequently tested with Cityscapes to validate the methods' generalization ability with out-of-distribution (OOD) testing. Furthermore, to evaluate how the proposed methods influence learning with scarce data, a dataset scale variation experiment was conducted with the BDD10K dataset.

To transpose semantic segmentation in autonomous driving to an ordinal segmentation problem, ordinal relations must be derived from the classes in the dataset. When analyzing an autonomous driving scene, fig:driving_scene1_bdd100k, we can, a priori, derive that, usually:



  * The vehicles will be on the road or in parking spaces;

  * The drivable area will be on the road;

  * The ego lane will be in the drivable area;

  * The sidewalk will be on either side of the road;

  * The pedestrians will either be on the sidewalk or the road;

  * The remainder of the environment surrounds the road.


Taking this domain knowledge into account, tab:segclasses introduces the ,  and  ordinal segmentation mask setups, including the ordinal relationship between classes in the form of a tree.  fig:driving_scene1_bdd100k(b) shows the  ordinal segmentation mask setup for the autonomous driving scene in fig:driving_scene1_bdd100k(a).







 §.§ Experimental Setup
 

The experimental results were obtained using the UNet architecture <cit.> with four groups of convolution blocks (each consisting of two convolution and one pooling layers) for each of the encoder and decoder portions[An open-source PyTorch implementation of the UNet architecture was used, <https://github.com/milesial/Pytorch-UNet>.]. All datasets were normalized with a mean of 0 and a standard deviation of 1 after data augmentation, consisting of random rotation, random horizontal flips, random crops, and random brightness and contrast. The networks were optimized for a maximum of 200 epochs in the case of the biomedical datasets and a maximum of 100 epochs in the case of the autonomous driving datasets, using the Adam <cit.> optimizer with a learning rate of 1e-4 and a batch-size of 16. Early stopping was used with a patience of 15 epochs. After a train-test split of 80-20%, a 5-fold training strategy consisting of 4 training folds and one validation fold was applied for each training dataset. For each fold, the best-performing model on the validation dataset was selected. An NVIDIA Tensor Core A100 GPU with 40GB of RAM and an NVIDIA RTX A2000 with 12GB of RAM were used to train the networks.

The metrics used were: (1) the Dice coefficient, which evaluates the methods with respect to the ground truth labels, (2) the contact surface, which evaluates the methods with respect to the spatial ordinal consistency, and (3) the percentage of unimodal pixels metrics, which evaluates the methods with respect to the pixel-wise ordinal consistency.

The cross-entropy loss and the methods by Fernandes  <cit.> were used as the baseline models. The methods to be evaluated had their parameterization, including the range of regularization term weights (λ), empirically determined. These methods are:



  * The semantic segmentation adaptation of the ordinal representation consistency L_CO2 loss function, with the imposed margin δ = 0.05, as recommended by the authors <cit.>;

  * The proposed ordinal structural loss for segmentation, L_CSNP
and L_CSDT, with the distance transform threshold δ = 0.5 and the maximum regularization distance γ = 10;

  * The mix of representation and structural methods, through the L_CO2 + L_CSNP loss combination of the two regularization terms.






§ RESULTS



Tables <ref>–<ref> depict the main results. Since each loss term proposed is actually a penalty that is weighted with cross-entropy (loss=CE+λ term), each loss was evaluated using grid-search, λ∈{0.1,1,…,10^4}, selecting the best value for each metric.

Table <ref> shows that the proposed losses generally offer an improvement, as judged by the Dice coefficient. Notice that this metric is computed for each pixel, explaining why CO2 (a loss that focuses on the pixel-wise ordinal representation consistency) generally performs better. Two additional tables are presented with the proposed ordinal metrics, evaluating the consistency for the representation space (% unimodality in Table <ref>) and the structural space (contact surface in Table <ref>). Naturally, CO2 again shows a very good performance for representation consistency, but the other losses start to become competitive as the structural consistency is evaluated using the contact surface metric.



Additional results are shown in plots that depict the influence of the weight of the regularization term, λ, on each respective metric. Figures fig:biomedical:plots:dice, fig:biomedical:plots:cs and fig:biomedical:plots:unimodal show, respectively, the Dice coefficient, contact surface, and unimodal metrics for the biomedical datasets. We can see that the ordinal methods successfully optimize the ordinal metrics, resulting in more ordinally consistent models. The dice coefficient metric does not improve much; however, excessive regularization can occur at some λ values, resulting in a lower Dice metric.

fig:bdd100k:plots:dice shows the Dice coefficient for the autonomous driving datasets, including the out-of-distribution domain testing. There, it can be seen that CO2 is the method with the highest impact on the generalization capability of the model, achieving a maximum absolute gain of 4.2% (11.5% in relative terms) at λ = 10, which means that using this loss helps the model generalize better to previously unseen scenarios. fig:results:co2:samples showcases a set of model inference outputs for this scenario, comparing cross-entropy outputs to CO2 with λ = 10.

To evaluate whether the inclusion of domain knowledge during the training of the models would help the network learn better with scarce data for autonomous driving scenarios, an experiment that consisted of varying the scale of the dataset used to train the models was performed. For each method, the best-performing lambda in the out-of-distribution scenario was selected, the rationale being that those models are the best at generalizing to unseen scenarios, which is useful when training with low amounts of data. For the L_CO2 and L_CSNP losses, λ = 10.0 was used, and for the L_CSDT, λ = 0.1 was used. fig:bdd100k:plots:scalevar:dice shows the Dice coefficient results for the dataset scale variation experiment.

When testing with the BDD10K dataset, the L_CO2 and L_CSNP losses achieve better Dice coefficient results at scales 0.25 and 0.5 when compared with the cross-entropy baseline, which suggests that, indeed, these losses help the network learn better when data is scarce. Especially when using the CO2 loss, resulting in absolute gains of 1.2% (5.7% in relative terms) in the Dice coefficient performance at scale 0.25 over cross-entropy. CO2 continues beating cross-entropy Dice performance through scales 0.1 and 0.05.

Cross-entropy generalizes better than the spatial methods when testing with the Cityscapes dataset in an out-of-distribution scenario for lower scales. However, the CO2 loss continues to generalize better than the cross-entropy throughout all scales, achieving a maximum absolute gain of 5.3% (15.7% in relative terms) in the Dice coefficient at scale 0.75. Still, it can be seen that the generalization ability of CO2 decreases at a higher rate than its performance on BDD10K – training with fewer data has a higher impact on the model's generalization ability.

Regarding the ordinal metrics, the scale variance does not significantly affect either metric.

In terms of compute complexity, since these are different losses, they do not affect inference time. Furthermore, any change in training time was too small to be detected.










 §.§ Discussion


A comparison between the biomedical and autonomous driving datasets results concludes that the biomedical results are significantly better in terms of the Dice coefficient performance – the performance gains with the autonomous driving datasets are still positive, but not as large, since these datasets are considerably more complex, with a greater variety of scenarios and segmentation classes.

The representation space, pixel-wise, L_CO2 performed well in both the biomedical and autonomous driving datasets, especially when considered in an out-of-distribution domain – it could potentially be applied in a real-world scenario in order to improve the generalization capabilities of perception algorithms.

The structured space, spatial, L_CSNP and L_CSDT losses, may not apply to the autonomous driving scenario in their current form, at least at high regularization weights. The scene perspective from the car makes it so that there is a large amount of valid contact surface between non-ordinally adjacent classes – in a 2D projection of the real world, the absolute minimization of these contact surfaces may not be the best solution since occlusions and different perspectives may originate legitimate contact between non-ordinally adjacent classes. Relaxed adaptations of these methods that consider this type of contact could be devised in the future. However, in the biomedical datasets, these methods performed well, with their greatest difficulty being the existence of occlusions.

As seen, the choice of λ value, regularization weight, is critical for the performance of the proposed methods. For this reason, in order to be applied to different domains, there should be an empirical study of the influence of the lambda value in the segmentation performance in that specific domain and the choice of a value that is a balance between the ordinal metrics and the Dice coefficient, a value that promotes some unimodality and spatial consistency but also does not hurt Dice performance to the point where it is unusable.





§ CONCLUSION



This paper explored ordinality in the context of structured data, proposing a formal definition of an ordinal model for generic graph structures. The concept was then specifically studied for semantic segmentation, where each image can be interpreted as a graph.

In this domain, two categories of loss functions for ordinal segmentation were studied: (1) an ordinal representation consistency loss, where each pixel is treated individually by promoting unimodality in its probability distribution, and (2) a structural consistency loss, where each pixel is considered in the context of its neighborhood and the contact surface between non-ordinally adjacent classes is minimized.

For that purpose, the following loss terms were proposed:


  * the segmentation adaptation of the representation L_CO2 loss;

  * the structural L_CSNP loss, which considers only the immediate neighbor pixels;

  * the structural L_CSDT loss, which considers the global neighborhood.

In addition, two metrics were proposed to evaluate the network output's ordinal consistency: (1) the percentage of unimodal pixels and (2) the contact surface between the segmentation masks of non-ordinally adjacent classes.

The proposed methods were initially validated on five biomedical datasets and two autonomous driving datasets, resulting in more ordinally consistent models without significantly impacting the Dice coefficient. To evaluate the methods' impact on the models' generalization capability, the resulting models were tested in an autonomous driving out-of-distribution scenario. Furthermore, the autonomous driving models were also tested with scaled-down versions of the BDD100K dataset to evaluate how the network learns with scarce data. The ordinal methods achieved maximum improvements in the Dice coefficient with an absolute value of 5.3% (15.7% in relative terms) in the out-of-distribution domain.

To summarize, incorporating ordinal consistency into semantic segmentation models showed promising results, including developing more generalizable models that exhibit improved learning capabilities with limited data availability. Since these are loss terms, they do not add time complexity to inference time.

Future research topics could include: (i) the development of more flexible spatial ordinal segmentation methods, allowing for limited contact between non-ordinally adjacent classes, such as in the case of occlusions and different perspectives; and (ii) the development of novel methods that leverage ordinal constraints not necessarily consisting of augmented loss functions.

Furthermore, the current work could contribute to other types of consistency. The partially ordered domains explored in this work were structural (e.g., a vehicle may be located on the road or in parking spaces), but a representational partially ordered domain could correspond to hierarchical segmentation <cit.>, where each pixel has a taxonomy (e.g., a motorcycle and bicycle are both two-wheels; two-wheels and four-wheels are both types of vehicles).

Also, part segmentation could be considered a different type of structural consistency, where parts of an object are next to each other. Super-pixels have been used to impose constraints, such as the “head” appearing above “upper body” or “hair” being above “head” <cit.>. Similarly, the function of Restricted Boltzmann Machines has been used to favor certain classes based on their spatial locations <cit.>. This work could be considered an alternative that promotes such structural consistency using loss penalties.



§ SOURCE CODE






IEEEtran




 Paper Title: On the Problem of Text-To-Speech Model Selection for Synthetic Data Generation in Automatic Speech Recognition.



Spin-wave Goos-Hänchen effect induced by 360 degree domain walls in magnetic heterostructures
    Jie Lu
    August 5, 2024
=============================================================================================




§ ABSTRACT 
The rapid development of neural text-to-speech (TTS) systems enabled its usage in other areas of natural language processing such as automatic speech recognition (ASR) or spoken language translation (SLT).
Due to the large number of different TTS architectures and their extensions, selecting which TTS systems to use for synthetic data creation is not an easy task.
We use the comparison of five different TTS decoder architectures in the scope of synthetic data generation to show the impact on CTC-based speech recognition training.
We compare the recognition results to computable metrics like NISQA MOS and intelligibility, finding that there are no clear relations to the ASR performance.
We also observe that for data generation auto-regressive decoding performs better than non-autoregressive decoding, and propose an approach to quantify TTS generalization capabilities.




§ INTRODUCTION


The usage of synthetic data from text-to-speech (TTS) systems in the context of automatic speech recognition (ASR) has been explored in many different ways.
*Work done while being an internship student at JAIST
This ranges from simply generating new data <cit.> to jointly training ASR and TTS systems<cit.>.
Prior work improved the integration of the synthetic data into the ASR systems, for example by different weighting of synthetic inputs <cit.>, or the usage of internal ASR model statistics to reject synthetic samples not matching the real data<cit.>.
As most publications aim to introduce new methodologies, in each case they usually use one specific TTS architecture and one specific ASR architecture.
The number of fundamentally different ASR architectures is rather limited and roughly categorized into: Hybrid deep neural network hidden Markov model (DNN-HMM) <cit.>, connectionist-temporal-classification (CTC) <cit.>, Transducer <cit.> and Attention-Encoder-Decoder (AED) <cit.>. For TTS there is a vast amount of different architectures and models, especially when including different approaches of multi-speaker modeling<cit.>.
From the perspective of working on ASR this makes it very difficult to choose which TTS system to use for synthetic data generation.
In addition, human evaluations are not suited to evaluate data for training purposes, as conducting the evaluation usually takes more time than running the training itself. 
Then, evaluation for naturalness and human subjective evaluations alone can not be expected to be a sufficient criterion for utility estimation of synthetic data.
Instead, aspects like speaker diversity, temporal diversity or expressiveness in general are assumed to be relevant to yield good results by integrating synthetic data into the ASR process<cit.>. Unfortunately, those aspects are difficult to quantify.

Furthermore, LJSpeech <cit.> is often the only common dataset to compare new TTS architectures, which can not be expected to give representative results for the multi-speaker case<cit.>.
This is especially relevant as one might not have training data suited for TTS training at hand for a particular domain or language, but instead has to use the ASR data.

Motivated by those problems, we present an experiment pipeline to systematically analyze the relation of TTS generated data to ASR training.
For this we compare different decoder architectures, which are based on: Non-autoregressive (NAR) Transformer similar to FastSpeech-2 <cit.>, autoregressive (AR) long-short term memory (LSTM) similar to Non-attentive Tacotron <cit.>, Glow-TTS <cit.> as flow-based decoder and Grad-TTS <cit.> as diffusion based decoder.
We keep the rest of the TTS systems as simple as possible, using a Transformer encoder and convolution based duration prediction network, as used in <cit.>.

Given the large amount of possible TTS architectures, we choose these systems for multiple reasons:
First, they have a widespread recognition in the scientific community. Then, they already share similar encoder and up-sampling concepts, sometimes even using the same code, which makes comparison easier. Finally, they do not require training with an integrated vocoder model (as e.g. in VITS) which would make a direct comparison more difficult.  
While comparisons of different TTS architectures with respect to naturalness and expressiveness exist <cit.>, this is to our knowledge the first work covering a broader range of different TTS output characteristics and quality levels targeted for ASR training.
Also, we perform a direct comparison of NAR vs AR decoding for data generation with a consistent sequence mapping method.

Lastly, we present a way of measuring generalization of the TTS systems with respect to ASR training data generation by using different conditions for synthesizing the training data. The Sisyphus <cit.> based setup and additional materials are available on Github[<https://github.com/rwth-i6/i6_experiments/tree/main/users/rossenbach/experiments/jaist_project>].


§ TEXT-TO-SPEECH SYSTEMS




 §.§ General architecture


In the design of the TTS systems we follow what we found to be most prevalent in recent literature. After the introduction of Transformer blocks with convolution for TTS <cit.> and the establishing of duration based state up-sampling <cit.>, many systems continued using these components. Thus, for each system we use the same layout of a Transformer based-encoder, a convolution based duration predictor and a discrete encoder state up-sampling method. Only the decoder, which converts the up-sampled encoder states h_1^T into the final log-Mel spectrograms x_1^T, is changed. Figure <ref> shows the TTS architecture used in this work. We do not use explicit f_0 or energy prediction. The overall structure follows exactly Glow-TTS, in which many parts were taken from TransformerTTS <cit.> or FastSpeech-2. We also used the code of the official Glow-TTS repository as reference for our implementation.


 §.§ Decoder variants




A large amount of proposed TTS decoder architectures can be categorized into three groups: Direct prediction, flow and diffusion. Direct prediction means that the output of the TTS neural network is a log-Mel spectrogram which is trained to directly match the target x_1^T. Common losses are mean absolute error or mean squared error. The generation is a single forward pass through the decoder network g_θ, which can be written as:

    x̂_1^T = g_θ(h_1^T)

The generation can also be done in an auto-regressive manner:

    x̂_t = g_θ(h_1^t, x̂_1^t-1)

For flow-based TTS systems, the decoder is an invertible neural network that can be applied in both directions from the spectrograms x_1^T into latent space variables z_1^T and vice-versa. During training, the target features are transformed into the latent space and matched against a normal distribution with a mean μ_1^T = μ_θ(h_1^T) and unit variance. The target objective is to maximize the likelihood of the latent space samples given the distributions. For inference, the latent space values are sampled from the distribution with a mean function μ_θ, temperature τ, and passed backwards through the flow f_θ:

    ẑ_1^T ∼𝒩(μ_θ(h_1^T),τ1)    and   x̂_1^T = f^-1_θ(ẑ_1^T)

For the diffusion-based TTS system we follow the Grad-TTS architecture. A normal distribution as in the flow-based approach is used to sample a noisy output spectrogram X̂_I with X = x_1^T. The noise spectrogram is cleaned in I steps using a differential equation containing the solver network s_θ and pre-defined noise schedule β_i to reach the final output X̂_0:

    X̂_I    ∼𝒩(μ_θ(h_1^T),τ1)
    X̂_i-1   = X̂_i - 1/2I(μ_θ(h_1^T) - X̂_i - s_θ(h_1^T, X̂_i))β_i

For each of the decoder principles we implemented one model for which the details will be presented later in Section <ref>.

























§ EVALUATION PIPELINE AND METRICS



















To evaluate the TTS systems in the context of ASR, we train a CTC-based ASR system purely on data generated by TTS. We generate the same amount of data  as used for the baseline training, and measure the performance of the ASR systems in word error rate (WER) for recognizing real human speech. In the optimal case, the performance of the ASR system trained on the synthetic data would match the performance of the system trained on real data. In order to evaluate the generalization capabilities of TTS towards new conditions, we generate the synthetic data in 3 ways: a) using exactly the same text and speaker assignment as in training, b) using the same text but shuffling the speakers and finally c) using the same amount of text from a different corpus in the same domain. That way we can measure how good the TTS systems are at generalizing to new conditions different from the training. The main evaluation criterion to measure the utility of the TTS-generated data is the WER of the trained ASR systems calculated on the official LibriSpeech <cit.>  test sets. In addition, we use metrics directly on the synthetic data to evaluate their relation to the final WER. We use NISQA <cit.> as a tool to generate automatic mean-opinion-score (MOS) values. While computer generated MOS values are not a replacement for subjective evaluations, they can be used as tool to determine naturalness as possible indicator for the ASR performance. Subjective evaluation itself can not be useful to evaluate synthetic data in a development process, as doing the evaluation often takes more effort than simply training the ASR system. We also use a larger ASR model trained on the full LibriSpeech corpus to recognize the synthesized cross-validation data to check for completeness and pronunciation correctness. We refer to this metric as synthetic WER (sWER). In the literature this is often called intelligibility metric, and sometimes character error rate is used instead of WER , e.g. in <cit.>.












































§ EXPERIMENTS




 §.§ Data


We use the LibriSpeech train-clean-100 as baseline data to train the TTS and ASR models, and the transcriptions of train-clean-360 as additional text data. This is a common setup that has been used extensively in previous literature <cit.>. We sub-sample the exact same amount of utterances, roughly 28k, from train-clean-360 to have the same amount of unseen text as text in the training data. We take 4 utterances of each of the 251 speakers in train-clean-100 out of the TTS training data to have a cross-validation set to report MOS and sWER. We perform no further pre-processing of the data, unlike other work <cit.>. I.e. no excessive silence is removed, and no speakers are discarded. We use the official LibriSpeech lexicon for word-to-phoneme conversion and use Sequitur <cit.> to generate phonemes for words that are not in the lexicon. For ASR, we augment the phoneme set with end-of-word markers as presented in <cit.>.



 §.§ Automatic speech recogition

As ASR system we use a CTC model with a Conformer-based<cit.> encoder with 12 blocks, a base dimension of 384 and a convolutional frontend with a sub-sampling factor of 4. For recognition we use the lexicon-constrained Flashlight <cit.> CTC decoder via the available Torchaudio interface. The search process includes the official LibriSpeech 4-gram LM model. Since it has been shown that synthetic data generation is successful mostly for AED-ASR <cit.>, we chose the context-free phoneme-based CTC-ASR specifically for the reason that it should be influenced less by changing or adding new text.



 §.§ Training and model settings



All models are trained using 80-dimensional log-Mel spectrogram features, for TTS with 12.5ms shift and for ASR with 10ms shift and SpecAugment <cit.> applied.
The TTS models are trained with Adam <cit.> for 400 epochs. For all models except Grad-TTS we use linear learning rate (LR) scheduling, which we found to perform better than inverse square reduction. We increase the LR from 5e-5 to 5e-4 for the first 100 epochs, and reduce linearly from 5e-4 to 5e-7. For Grad-TTS we use a constant LR of 1e-4. The ASR models are trained using AdamW  with weight-decay 1e-3 for roughly 80 epochs and a maximum learning rate of 7e-4. We generate an alignment by training the Glow-TTS system once using implicit alignment search. In each TTS training the fixed alignment is used for up-sampling and as target for the duration predictor. For vocoding, we convert the log-Mel to linear features using a separate BLSTM-based network and apply the Librosa Griffin & Lim (G&L) <cit.> function with 32 iterations and momentum of 0.99.



  §.§.§ TTS parameter settings


For the different decoder architectures the number of parameters, training time and inference time differ, as shown in Table <ref>. Making a fair comparison is difficult, as for each system we would need to do extensive tuning experiments until we reach the optimal performance. Thus, we stick mostly to the sizes presented in previous publications.

Due to the high cost of subjectively evaluating TTS systems, ablation studies on the model sizes are usually not presented in scientific publications.



Base model: For the TTS Transformer encoder we use 6 layers with an internal dimension of 256, preceded by 3 convolution layers with 256 channels and kernel size 5. The duration prediction network uses 2 convolutions with kernel size 3 and 384 channels.


Transformer decoder: For the Transformer-based decoder we simply re-use the same structure as for the encoder Transformer, having 6 layers with an internal base dimension of 256.

The system is trained using L1 loss on the output spectrograms.


Autoregressive LSTM decoder: For the AR-LSTM decoder we are using 2 layers of 1024-dimensional Zoneout-regularized LSTM layers. We use a reduction factor of 2 to predict two feature frames per decoder step. The pre-net for the AR feedback loop and the convolutional NAR post-net are derived from <cit.>. We apply L1 loss for both the decoder output and the combined prediction with the post-net.


Non-autoregressive LSTM decoder: This decoder is similar to the AR decoder, but we remove the AR feedback loop network, and replace the uni-directional Zoneout LSTM cell by a normal bi-directional LSTM with dropout. Each LSTM direction has 512 dimensions. For Transformer, AR- and NAR-LSTM, a 256-dimensional speaker embedding vector is concatenated to each upsampled encoder state.


Glow-TTS decoder: For a flow-based decoder, including speaker conditioning, we follow exactly the architecture of Glow-TTS <cit.>. The decoder consists of 12 invertible coupling blocks with an internal dimension of 256. We merge each 2 consecutive frames for a latent-space dimension of 160.


Grad-TTS decoder: For a diffusion-based model we follow the architecture and settings of Grad-TTS <cit.>. As the original architecture is not designed for multi-speaker TTS, modifications were necessary for convergence. We added two additional Transformer layers on top of the 6 encoder layers which are conditioned on the speaker embedding. That way, we have a speaker-conditioned mean prediction network. We also used the same speaker embeddings to condition the u-net, which is used for the differential equation solving.
We found τ = 0.7 to be the optimal noise scale for both Glow-TTS and Grad-TTS.



§ RESULTS




 §.§ Synthetic data, automatic MOS and recognizability

































In a first comparison we evaluated the systems via the synthetic data only ASR training, the NISQA MOS and the sWER.
Table <ref> shows the results of each of the five decoders compared to the reference values of the real data. For all metrics we used text data unseen during training.
With respect to the final ASR training evaluation, all TTS systems have a reasonable performance and there is no strong outlier.
In general, using the synthetic data for training results in a system with double the error rate when training TTS only on the available ASR data, which is a consistent result to previous literature <cit.>.
<cit.> showed that even when using vast amounts of training data to create strong foundation-model sized TTS models, the generated synthetic data seems to be not yet on par with real data.
The AR-LSTM decoder produces the best synthetic training data, which is unexpected as the Glow-TTS and Grad-TTS should produce more expressive and variable audio due to their internal sampling process.
Also, Glow-TTS is the only system that exactly follows the given literature, while the other systems have small adaptations or are simplifications of the original model architecture.

The third and fourth column in Table <ref> show the NISQA MOS and the sWER on the TTS cross validation set. It is visible that there is no direct relation of the two metrics compared to the final ASR performance.

For example, when comparing Grad-TTS to the Glow-TTS and Transformer decoder, there is a lower MOS and a much higher sWER, but still the final system performance is better. This means both metrics are not reliable to determine which system to pick for data generation.



 §.§ Decoder characteristics

In Figure <ref> we show a selected sequence synthesized by four different decoders. The Transformer decoder spectrograms are smooth, lacking a lot of detail in the high-frequency area. The AR-LSTM spectrogram in comparison has much more details. The spectrogram from Glow-TTS shows visible noise resulting from the latent space sampling. It also has less distinct boundaries between the phonemes, but rather fuzzy transitions, which might be an explanation for the worse recognizability. The Grad-TTS has less distorted spectrograms than Glow-TTS, but we found that for very short sequences the output is sometimes just a noisy grid pattern. Also the voice lines become very blurred together for lower frequencies, causing a very unnatural sound, which matches with the low automatic MOS score. Thus we assume, that the NISQA MOS and the sWER are very sensitive to certain artifacts that have less impact in on the actual ASR training. One can see that all four TTS decoders produce spectrograms with different characteristics, even with other components being identical. Thus, it is important to not only consider a single TTS architecture when trying to show a general relation between the ASR performance and metrics calculated on the level of the TTS system or the synthetic data.

































 §.§ Generalization




Table <ref> shows how good the TTS systems generalize with respect to the synthetic data generation task. Recall that condition a) means to exactly reproduce the training data, so with perfect memorization during training it would be possible to produce identically performing data. It is visible that across all models the values for a) and c) are mostly not too far from each other, so the generalization abilities are high. This gives a hint that current models are under-fitting and much larger models might yield better results, even on very constrained datasets such as LibriSpeech 100h. For synthesizing new text, so comparing b) to c), most models show no or only a small change. Only for the AR LSTM and Glow-TTS decoder there is meaningful degradation in the WER, mostly for the "other" test. For the Glow-TTS a possible reason could be that it suffers from pronunciation issues, which get even stronger when synthesizing new text. Testing for the generalization capabilities of TTS this way might not be used only for the comparison of different models like in this work, but allow for easier hyperparameter tuning which is usually not done extensively in TTS literature.



 §.§ Discussion


When training with synthetic data only, we achieved a WER performance which is less than factor 2 worse compared to training on the real data. In relation, <cit.> reported reaching ratios of around 2 on the LibriSpeech-960 task, and <cit.> reported ratios of more than 3. In this work we specifically investigated decoder architectures. In future work this gap can be reduced by extending the results of this work with extensive hyperparameter tuning as well as prosody <cit.> and more sophisticated duration modeling <cit.>. We have seen that NISQA MOS and the sWER are not reliable to determine the usefulness of the TTS system for ASR data generation. This means there are more and/or different metrics needed to give accurate estimates about how good the synthetic data can be utilized for ASR training. While in our work synthesizing new data and training the ASR systems was done in less than a day, this becomes much more of a problem when training large models on magnitudes more of data. Thus, we think it is important to find ways to estimate the capabilities of TTS models before starting large-scale data generation.



§ CONCLUSION

In this work we evaluated different TTS decoder architectures w.r.t. their ability to generate synthetic data for ASR training. We compared five different TTS decoder architectures and found that for our setup the AR-LSTM-based system with L1 loss performed best in creating training data for ASR. We were able to create synthetic data that performs less than a factor of 2 worse than real data while training the TTS only on the same data as available for the ASR training. We observed that the systems' outputs differ from each other in many aspects such as naturalness or pronunciation accuracy. Still, the metrics we used to capture them showed a surprisingly low correlation to the performance of the ASR systems trained on such outputs. This indicates that better metrics have to be found in order to do a meaningful evaluation of TTS systems before using it to create synthetic training data. We also introduced a way to measure the generalization capabilities of TTS systems for data generation, which can be used to optimize TTS towards being able to generate data that reaches parity with real data.




§ ACKNOWLEDGEMENTS




This work was partially supported by NeuroSys, which as part of the initiative “Clusters4Future” is funded by the Federal Ministry of Education and Research BMBF (03ZU1106DA), and by the project RESCALE within the program AI Lighthouse Projects for the Environment, Climate, Nature and Resources funded by the Federal Ministry for the Environment, Nature Conservation, Nuclear Safety and Consumer Protection (BMUV), funding ID: 67KI32006A. Part of this work is supported by JSPS KAKENHI Grant Numbers JP21H05054 and JP23K21681. The authors thank Lukas Rilling, Peter Vieting and Benedikt Hilmes for their additional input and feedback.




IEEEtran









 Paper Title: Cost-Effective Hallucination Detection for LLMs.









Cost-Effective Hallucination Detection for LLMs
    Simon Valentin^1[1]
        Jinmiao Fu^2[1]
        Gianluca Detommaso^3These authors contributed equally to this research. 
        Shaoyuan Xu^2
        Giovanni Zappella^1
        Bryan Wang^2
    
    ^1Amazon Web Services, Berlin, Germany 

    

    ^2Amazon, Seattle, USA 

    

    ^2Helsing, Berlin, Germany 

    


=======================================================================================================================================================================================================



§ ABSTRACT 
Large language models (LLMs)
can be prone to hallucinations — generating unreliable outputs that are unfaithful to their inputs, external facts or internally inconsistent.
In this work, we address several challenges for post-hoc hallucination detection in production settings.
Our pipeline for hallucination detection entails: first, producing a confidence score representing the likelihood that a generated answer is a hallucination; second, calibrating the score conditional on attributes of the inputs and candidate response; finally, performing detection by thresholding the calibrated score. We benchmark a variety of state-of-the-art scoring methods on different datasets, encompassing question answering, fact checking, and summarization tasks. We employ diverse LLMs to ensure a comprehensive assessment of performance.
We show that calibrating individual scoring methods is critical for ensuring risk-aware downstream decision making. 
Based on findings that no individual score performs best in all situations, we propose a multi-scoring framework, which combines different scores and achieves top performance across all datasets.
We further introduce cost-effective 
multi-scoring,
which can match or even outperform more expensive detection methods, while significantly reducing computational overhead. 





§ INTRODUCTION

Despite their impressive capabilities,  large language models (LLMs) can be prone to generating hallucinations — undesirable outputs that are incorrect, unfaithful, or inconsistent with respect to the inputs (or the output itself) <cit.>. 
These unreliable behaviors pose significant risks for adopting LLMs in real-world applications. 
Challenges in detecting hallucinations lie, among other things, in hallucinations taking different forms, being context-dependent and sometimes being in conflict with other desirable properties of generated text <cit.>. 
Hallucinations may be harmless in some contexts, but can be undesired or potentially dangerous in other applications (e.g., erroneous medical advice).
Detecting and quantifying hallucination risk is thus a critical capability to enable safe applications of LLMs and improve generated outputs.



Prior work has proposed various approaches for detecting and mitigating hallucinations in LLM-generated outputs, including verifying faithfulness to inputs <cit.>, assessing internal coherence <cit.>, consulting external knowledge sources <cit.>, and quantifying model uncertainty <cit.>.
However, deploying these methods in production settings is far from trivial due to several challenges:
First, there is limited comparative evaluation illuminating how different detection methods perform. 
Second, existing approaches for detecting hallucinations differ greatly in their computational demands, and guidelines are lacking on cost-effectiveness trade-offs to inform method selection for real-world applications with constraints.
Third, hallucination detection in the real world often requires careful consideration of risks and false positive/negative trade-offs, requiring methods to provide well-calibrated probability scores.
Fourth, many applications of LLMs take the form of calls to black-box APIs, which sometimes requires methods to employ workarounds to assess the model's confidence in its generated output. 

In this work, we provide a framework for detecting hallucinations in the outputs of any LLM in a model-agnostic manner.
Our approach relies on quantifying the probability that a generated answer is a hallucination. 
After computing initial scores, we employ state-of-the-art calibration techniques to provide calibrated probabilities of the generation containing a hallucination, which can subsequently be used for decision making or other downstream tasks. 
We evaluate a variety of scores proposed in the literature for hallucination detection on several metrics, across multiple datasets encompassing question answering, fact checking, and summarization tasks. We employ a range of different LLMs to ensure a comprehensive assessment of performance. 

Critically, as we show that no single score performs best across all datasets, 
we introduce
multi-scoring, 
a simple way of combining multiple calibrated scores, which achieves superior performance compared to 
any individual score alone.
Furthermore, 
we 
propose cost-effective multi-scoring, which 
finds the subset of best-performing scores for any fixed cost budget, and combines them in a multi-scoring fashion. Our empirical demonstrations reveal that cost-effective multi-scoring not only matches but often surpasses the performance of individual scores that incur significantly higher costs. Consequently, our proposed method achieves superior hallucination detection outcomes while maintaining a substantially lower cost footprint.


We summarize our contributions as follows: 1. We benchmark a variety of hallucination detection methods across the literature on several metrics, over different datasets and LLMs. 2. We introduce multi-scoring, a novel approach that aggregates multiple complementary scores and outperforms individual scores. 3. We further propose cost-effective multi-scoring, which optimally balances detection performance and computational constraints. 





§ DETECTING LLM HALLUCINATIONS




 §.§ Formalizing Hallucination Detection

We study the problem of quantifying the probability that a generated output from a language model contains hallucinations.
More formally, let 
x represent an input token sequence to a language model 𝒢, and let 
z represent a generated output text sequence from the model. 
We define a binary random variable y ∈{0,1} that indicates whether z is a “permissible” output (y=1) or contains a “hallucination” (y=0). 

Our goal is to develop a scoring function to model the probability that a given output text contains a hallucination conditioned on the input.[Note that while we are interested in whether the entire output contains any hallucination, one could easily adapt the methods at different granularities, such as the sentence or phrase level.]
This is critical, as in real-world scenarios, we need to set risk-aware thresholds, balancing false positive/negative rates to accommodate the production requirement.
Having access to scores allows us to flexibly set the threshold.
Conceptually, the key reason to model this probabilistically is that there is inherent uncertainty in determining whether a given text contains a hallucination or not. 
Even human raters may disagree on the assessment, based on their own knowledge and definition.
Some key contributors to this epistemic uncertainty are: First, that no system has complete world knowledge to perfectly assess factual correctness. 
Second, that there is ambiguity in whether something is a hallucination. 
Finally, any automatic scoring model may make occasional errors, so probabilistic scores reflect confidence. 
Therefore, while conditional on x and z the true hallucination label y is fixed, our estimate of y remains uncertain. The probabilistic score thus reflects this epistemic uncertainty — the degree of belief that z contains a hallucination given the available knowledge:
p(y = 0 |x, z; ψ), where ψ denotes parameters of the scoring function. We refer to this conditional probability function as the hallucination score, denoted s_ψ(x, z).
The hallucination score can then be applied to downstream tasks, including making risk-aware binary decisions.

We discuss concrete instances of hallucination scores in the following subsection.
Generally, the form of s_ψ can vary between hallucination detection approaches. 
In particular, s_ψ may depend on multiple candidate texts generated for the same input, as implemented by a number of hallucination detection methods proposed in the literature.
We denote K candidate generated outputs as Z = [z_1,...,z_K]. 
Then s_ψ(x, Z) could quantify inconsistencies within texts in Z using different metrics ψ, as discussed below.



 §.§ Scoring Methods

Many hallucination detection methods proposed in the literature make use of LLMs to “judge” the output of an LLM (either the same or a different one). 
For clarity, we thus distinguish between generator and detector LLMs. 
The generator is defined as the model used to generate the original response. 
The detector LLM is the model used to score a generated text for the presence of hallucinations. 
In general, the generator and detector LLMs may coincide. 
However, it may be more desirable to use different LLMs in some scenarios, e.g., when computational cost is a greater concern, where a smaller LLM may be used to judge outputs of a more expensive LLM, or when using hallucination scoring methods that require white-/grey-box access, while the generator is black-box. 

Today, many interactions with LLMs take the form of API-calls.
Typically, only the output tokens are returned with no access to the logits of the predicted tokens,  treating the LLM effectively as a black box. 
Sometimes, inference parameters may be accessible, allowing for setting different temperature (among others) values, thereby providing some (grey-box) access to the model. 
Generally, hallucination detection methods vary in their required model access, ranging from black-box APIs to full white-box access to the model weights.
That is, while some methods only require token-level outputs, other methods may need access to the logits of the generated tokens, or some control over inference parameters like temperature. 

In our experiments, we evaluate a comprehensive set of hallucination scoring methods.
Generally, we do not make any assumptions about the generator LLM being white-, grey- or black-box. 
We divide methods into single-generation methods, which require only one generated output, and multi-generation methods which are based on multiple alternative generations. 





 §.§ Single-generation

We first provide an overview over a set of hallucination detection methods that are based on scoring the hallucination from a single given generated output. 



  
Inverse Perplexity This method provides a prominent instance of using a model's logits over output tokens to score the confidence in the output. Computed as the inverse of the exponentiated average negative log-likelihood of the LLM's response <cit.>,

    Perplexity^-1(W) = exp(1/N∑_i=1^N log p(w_i|w_i-1,...,w_1)),

inverse perplexity thus provides a sequence-length normalized expression of the model's confidence. 
Here, the model may either be the generator LLM or a different detector LLM.
Using a different LLM from the generator as a plug-in estimator implies that the confidence estimate is detached from the generator. 
If our generator provides access to the model's logits we can directly use them to score hallucinations.
Alternatively, we can use another LLM to generate the logits of output tokens. 



  
P(True) This method works by prompting an LLM whether an answer is correct or not, then making use of the logits of the next token <cit.>. 
Given a prompt like the following: 


We compute P(True) by first taking the softmax over the first generated output token's logits, 
and then normalizing across the “True” and “False” tokens.[Note that alternatively, we can prompt the LLM to bind the True/False tokens to symbols like response options A or B <cit.>. However, this did not change results in our experiments. Further, if the detector LLM provides only black-box access, one can sample the corresponding tokens and use the empirical distribution to approximate, but thereby incurring a higher computational cost.]
That is, let 𝐳^(1) = (z^(1)_1, …, z^(1)_|V|) be the logit vector over the vocabulary V for the first generated token when asking the LLM to evaluate whether the answer is correct. We then take the softmax over these logits, 
p(w |𝐳^(1)) = exp(z^(1)_w)/∑_w' ∈ Vexp(z^(1)_w').
Let w_True, w_False∈ V be the “True” and “False” token ids. Then we can compute P(True) as:

    P(True|𝐳^(1)) = p(w_True|𝐳^(1))/p(w_True|𝐳^(1)) + p(w_False|𝐳^(1)).


In addition to checking whether the response is correct, we provide variations of P(True), where we check whether the output contradicts the input in P(InputContradict), the output contradicts itself in P(SelfContradict), or the output contradicts generally established facts in P(FactContradict), with the followings prompts: 








  
NLI Text Classification Natural language inference (NLI) models provide an alternative to assess the correctness of the model output. As hallucination detection requires checking for contradictions, we compute the score as 1-S_contradict, where S_contradict refers to the softmax probability of the output conflicting with the question. Specifically, we use a DeBERTa model fine-tuned on an MNLI task <cit.> as the underlying NLI model, which we refer to as NLI (DeBERTa).




  
Verbalized Probabilities Instead of analyzing a model's logits, scores are elicited by asking an LLM to provide a confidence verbatim, i.e., by generating tokens that indicate the numerical confidence <cit.> with the following prompt:

We note that moderated LLMs accessed through APIs may in practice decline to answer if the context does not provide sufficient information, rather than assigning a low confidence score.  



 §.§ Multi-generation
 Multi-generation methods are based on quantifying the consistency across multiple generated outputs from the generator LLM. 
This follows the notion of white-/grey-box uncertainty quantification via logits that if the LLM is confident in its response, multiple generated responses will probably be alike and contain compatible facts. 
Conversely, for fabricated information, sampled responses are more likely to differ and contradict one another. Crucially, this rests on the assumption that the model's confidence is calibrated, as we discuss and evaluate below.



  
SelfCheckGPT This method exploits a DeBERTa NLI model to assess whether the answer A_0 is consistent with K alternative generated answers A_1, ..., A_K <cit.>. Each sentence of A_0 is compared against the full set of answers A_1, ..., A_K. A consistency score per sentence is obtained via an NLI model, then the final score is obtained by averaging over the sentence-wise scores. 



  
Similarity Degree This method is based on computing pairwise similarities between the multiple responses using NLI models, and then quantifying uncertainty based on the distribution of similarities <cit.>. Here, we compute the pairwise similarities between responses via the contradict class probability of an NLI model and construct a degree matrix, where each diagonal element corresponds to the total (sum) similarity of the corresponding response to all other responses. We use the degree of the candidate response A_0 as the confidence estimate, following favourable results in prior work <cit.>, but note that other metrics have also been proposed in the original paper.



  
NeMO Guardrails: Hallucination Rail 
Unlike SelfCheckGPT or Similarity Degree, the score here is computed via one LLM call, where we check for agreement between the concatenated K additional generations and the candidate output and not averaged over sentences or computed on pairs of responses. Specifically, we use the softmax probability normalized over yes/no tokens using the following prompt:





 §.§ Calibration

Initial hallucination scores may not be properly calibrated, which can lead to poor downstream decisions. Seminal work by Guo et al. <cit.> has demonstrated that neural models tend to be miscalibrated, particularly in the form of models being overconfident. 
Later work, focusing on language models, has confirmed this across a wide range of tasks, models and datasets <cit.>. 
Notably, there is also research which demonstrates that given particular prompts and in-distribution tasks, LLMs can be well-calibrated <cit.>. However, this has been shown to be brittle and dependent on context <cit.>. 

Formally, our score outputs probabilities p_ψ(ŷ=0|x, z) that z contains a hallucination, parameterized by ψ. 
The score is calibrated if, for any probability level p ∈ [0,1], the average observed frequency of hallucinations matches the predicted probability:

    𝔼[y | p_ψ(ŷ=0|x, z) = p] = p


A naive approach for obtaining scores would be to compute the model's probabilities marginally, ignoring the context/question x and generated response z and directly estimating p(y=1). 
However, this does not allow for conditioning on individual inputs to obtain calibrated probabilities p(y=1 |x, z) for specific x, and z, which is, however, impossible to guarantee <cit.>. 
Common calibration methods include temperature scaling (Platt scaling) of logit outputs <cit.>, isotonic regression <cit.> or histogram binning <cit.>, which operate marginally, and can thereby not account for different confidence levels for different inputs (e.g., with an LLM being more confident on certain domains than others).

An alternative is to partition the inputs into G groups and compute calibration separately for each group g∈ G. However, this assumes the groups are disjoint and does not handle inputs belonging to multiple groups, which is often necessary <cit.>.
More advanced calibration methods, such as multicalibration, which we use in this work, allow defining G potentially overlapping groups  <cit.>. 
Prior work has shown the effectiveness of modern calibration techniques in scoring LLM confidence, albeit in a white-box setting <cit.>.
We describe our calibration approach in the experimental section.



 §.§ Multi-Scoring: Combining Scores

Different scoring methods capture different aspects of hallucinations, e.g., incorrect, non-factual, non-grounded, irrelevant, inconsistent with other answers, etc. As a result, some scores may work better on some data or for some specific models, and worse on others. Therefore, we design a multi-scoring method to combine the complementary information from individual scores into a single, strong predictor.

Denote each available score by s_n, for n=1, … ,N. To obtain an aggregated score, we run a logistic regression using as features the concatenations of the logit of each score, i.e. [logit(s_n(x, z)), …, logit(s_N(x, z)) ], and the labels of the calibration dataset as target variables. [Note that we also conducted experiments with alternative models, such as XGBoost <cit.> and Random Forest <cit.>, but results did not show significant improvements, so we opted for logistic regression for simplicity.]





 §.§ Cost-Effective Multi-Scoring

Scoring methods based on multiple generations incur the cost of additional generations as well as the cost of checking their consistency. 
Similarly, multi-scoring can be a viable choice when there are only few LLM generations to check for hallucinations, but incurs considerable computational cost, especially when using multi-generation methods. 
To avoid prohibitive computational costs in practice, we propose cost-effective multi-scoring, where we set a fixed computational budget and compute the best performing combination of scores that stay within the specified budget. 

Given an input text x and generated text z, we have N scoring functions s_1(x, z), ..., s_N(x, z) with associated costs c_1, ..., c_N (e.g., number of generations required). We are given a total computational budget B. Our goal is to find the optimal subset of scores S^* ⊆{1,...,N} that maximizes detection performance while staying within budget B:


    S^∗ = _S: S ⊆{1, …, N}ℒ(f(s_i(x, z))_i ∈ S)   s.t.  ∑_i ∈ S c_i ≤ B


where ℒ measures loss on a validation set. This is a constrained optimization problem over subsets of scoring functions. When B=min_i c_i, it reduces to selecting the single best score at the lowest cost. When B=∑_i c_i, it recovers full multi-scoring. In between, the optimal subset S^* provides the best trade-off between performance and cost. 
We note that in general, this problem is computationally challenging, given the exponential runtime complexity. 
However, given that there are only generally a small set of potential scores (N ∼ 10) and the logistic regression is very fast to fit, computing all combinations takes only 1.8 seconds on a single Intel Xeon processor (3.1 GHz), iterating over all candidate solutions sequentially. 
When this approach is not feasible, some exemplary alternatives include classic greedy forward-selection methods or regularising the model via an L1 penalty while scaling the regularisation term to accommodate score cost. 
Overall, cost-effective multi-scoring allows for flexibly balancing multiple scores under computational constraints.



  
Quantifying Cost
While the cost c_i of scoring function s_i is presented abstractly above, accurately quantifying computational cost can be challenging in practice. 
The actual runtime per method is a reasonable proxy, however the runtime depends on model architecture, hardware acceleration, batching, etc. 
One approach is to directly benchmark each scoring function's average runtime empirically on the target hardware. 
However, this overlooks nuances like caching effects and ignores runtime variability due to implementational differences.
To simplify our analysis, and as LLM calls are generally much more expensive than calling smaller NLI models based on parameter size, we leverage the number of LLM calls required per method as a proxy. See Table <ref> for an overview. 
If more precision is desired, we suggest empirically running different scores and computing their computational cost in the actual application, as the cost will depend on the precise setup and a range of factors.




§ EXPERIMENTS




 §.§ Experimental Setup



  §.§.§ Datasets




  
TriviaQA TriviaQA <cit.> is a commonly used factual open-respon­se QA dataset. Originally set up as a reading comprehension task, it is today often used without context as a closed-book (free-recall) task <cit.>. 
We use the validation fold, containing 17944 question-answer pairs. We use  <cit.> to generate candidate answers. To decide whether a given response should be labelled as positive or negative, we check whether the correct answer is contained in the generated answer after removing formatting, following the original evaluation script <cit.>.  



  
FEVER The closed-response Fact Extraction and VERification dataset <cit.> provides a comprehensive benchmark of factual hallucination detection. We take the test fold, containing 14027 labelled examples of source documents, claims and whether they are  ,  or contain . For the purpose of hallucination detection, we look at all claims that are either  or .



  
HaluEval We include the hallucination detection dataset HaluEval <cit.> and use the summarization task, which contains 10000 labeled examples of source documents, summaries and labels for whether the summary contains hallucinations. 



  
BIG-bench It provides an evaluation of LLMs on a diverse set of tasks <cit.>. We select 11 tasks suitable for hallucination detection, leading a total of 8664 labelled examples from the validation fold. 



  §.§.§ Metrics

Practical applications of hallucination detection require calibrated scores, but often also binary decisions over whether a given output is hallucinated or not. 
To this end we report the Brier score <cit.>, binary decision metrics F1 score and accuracy. 



  §.§.§ LLMs

We conduct experiments with  <cit.>,  <cit.>,  <cit.> and  <cit.>.
All models are used with default configurations via HuggingFace Transformers <cit.>.



  §.§.§ Calibration

The calibration step is performed via the following multicalibration approach, using the Fortuna library <cit.>:
To obtain groups, we compute embeddings of the input text x and the generated response z, such that our embedding is e[ (x), (z) ]. We obtain embeddings from Universal AnglE Embedding <cit.>, which is the SOTA in the MTEB benchmark <cit.> at the time of writing. 
We subsequently reduce the dimension of x via UMAP <cit.> and perform soft-clustering via Gaussian Mixture Models, as a simple off-the-shelf algorithm (fitting the number of cluster components via BIC <cit.>).
The calibration error is measured for each group g∈ G separately. 
The model is then iteratively patched on the group with the largest error until the calibration error drops below a threshold for all groups.[Note that alternative approaches are possible <cit.>.] 
This provably converges to a calibrated model with theoretical guarantees <cit.>.
We fit the calibrator to a random calibration fold of 80% of the data and report only held-out test results. 
For all binary predictions, we set the threshold to the 50th percentile to not impose preferences over false true/negative rates, but note that in practice any threshold could be applied. 



 §.§ Individual Scoring Methods

Table <ref> presents the results of different hallucination detection methods on all datasets. Multi-generation methods (SelfCheckGPT-NLI, HallucinationRail and SimilarityDegree) are employed only for TriviaQA, since the latter provides the true answer to each question, which can be compared to the alternative generated answers to assess their agreement. In contrast, for HaluEval, BIG-Bench and FEVER, the true answer is not provided. Instead, we compare the given candidate answer from the dataset against the provided binary label of correctness.
If we exclude Multi-Scoring methods, multi-generation methods such as SelfCheckGPT-NLI and SimilarityDegree (with 10 generations) achieve the best performance on TriviaQA, P(True) is the best on HaluEval and BIG-Bench, and P(InputContradict) performs best on FEVER. 
Thus, we find that there is no single best scoring method across all datasets. 
This is likely because different scoring methods capture different aspects of hallucination, supporting the notion that hallucination is a multi-faceted concept. 
Multi-generation methods measure hallucination based on the consistency of the generator LLM's responses and on the ability of the comparison method to identify whether multiple alternative responses are in agreement. 
We find that SelfCheckGPT-NLI performs best in our experiments in comparison to SimilarityScore and HallucinationRail.
While all of these methods follow similar approaches, there are subtle differences in how they assess the consistency between different responses.
More generally, as we discuss below, multi-generation methods are appropriate only if one can assume that there is exactly one correct response, but can fail otherwise. 

Other methods are based on different notions of hallucination, such as NLI (DeBERTa) measuring the entailment of the response given the input. 
Interestingly, variants of P(True) can be appropriate for detecting different kinds of hallucinations. 
P(True) explicitly asks the evaluator LLM to check whether the response is correct. 
In some contexts, the evaluator LLM may have the ability to directly evaluate this. 
However, in other situations, as we see with the FEVER dataset, other methods such as P(InputContradict) can be more appropriate, if we are trying to directly target a specific form of hallucination, which may not be subsumed under a general “correct or not?” prompt. 
Other applied scenarios may target even more different (or more specific) kinds of hallucinations, though the variants we include in this work are designed to cover the space in a reasonable manner. 
Similarly, while the datasets included in these experiments were selected to cover a wide range of different kinds of hallucinations, real-world applications may show even new kinds of hallucinations, for which no public datasets are available.
Overall, these findings highlight the need for our multi-scoring method which can absorb the strength of each individual method and can be easily applied to a concrete hallucination detection setting while only requiring a relatively small amount of labeled data. 

Overall, neither the inverse perplexity score nor the NLI (DeBERTa) scores emerge as the best scores for any of the datasets we consider. While they may add information that can be exploited in (cost-effective) multi-scoring, as reported below, individually they perform worse than some of the other scores. 






 §.§ Multi-Scoring

To evaluate the proposed multi-scoring method, we conduct experiments on all datasets combining the scores via logistic regression. 
As presented in Table <ref>, the multi-scoring ensemble achieves an F1 score of 0.9106 on TriviaQA, outperforming the best individual F1 score of 0.8614 (achieved by SelfCheckGPT-NLI).
Similarly, Brier and Accuracy also show the highest performance for multi-scoring.
For HaluEval, we also find that multi-scoring outperforms the best individual score (P(True)) in all metrics. 
BIG-Bench shows more nuance, as multi-scoring outperforms the best individual score (P(True)) on Brier and Accuracy, but not on F1. 
This reflects the fact that the metrics capture different properties, and practical considerations may require a decision as to which metric should be prioritized in a given application.
For FEVER, multi-score again outperforms the best individual score (P(InputContradict)) in all metrics. 

Thus we see that combining scores performs better than the top performing individual scores, showing that combinations of different scoring signals complement each other and can boost performance. 
This demonstrates that combining complementary signals enables more robust hallucination detection, while balancing the strength of each scoring method.
Crucially, while some settings may benefit from combinations of different signals (such as when trying to detect different kinds of hallucinations in a given generated output), in other settings the data-driven selection of an informative signal may be sufficient (such as when trying to detect a more narrowly defined notion of hallucination). 
These scenarios are covered by our use of multi-scoring, which allows for arriving at optimally combined hallucination scores (or binary decisions) for a given application.
Meanwhile, as computational cost can be a concern with scaling LLM applications, it may not be desirable to always use a full ensemble of scores, and we would rather compute the most performant score at a fixed computational cost.



 §.§ Cost-effective Multi-Scoring

Shown in Table <ref>, cost-effective multi-scoring methods are also amongst the top performers, but at considerably lower cost compared to multi-score. We see that at cost C=1, measured as the number of LLM calls (but see our discussion about measuring cost above), we recover the best individual scores, which varies across each dataset. As we increase the budget, the cost-effective multi-scores converge to the performance of the multi-score itself. Notice that in several instances, cost-effective multi-score with C=2 already performs as good as multi-score itself. 
For TriviaQA, we see that performance increases on all metrics as we increase the computational budget, with cost-effective multi-score at C=5 outperforming SelfCheckGPT-NLI as the most performant individual scores at half the computational cost in all metrics but Brier.
HaluEval shows no improvement beyond a budget of C=2, which is likely due to the kind of hallucination to detect being more narrowly defined and well-capture by a small number of signals, thereby not benefiting from additional scores. 
For BIG-Bench, as discussed for multi-score, we recover the best individual score at C=1, and match multi-score at higher budgets.  
FEVER results indicate that, again, we recover the best individual score at budget C=1, and can already recover the full multi-score at a budget of C=3.

We now take a closer look at cost-effect multi-scoring when including multi-generation scoring methods.
The overall cost budget is varied over the entire range. For each budget, we solve the constrained optimization in Eq. <ref> to find the optimal subset S^* of scores.
We compare the hallucination detection F1 score achieved by the cost-effective ensemble versus individual scoring functions and the full ensemble with all scores. Figure <ref> shows the results for TriviaQA for responses generated via . With a minimal budget of B=1, cost-effective selection recovers the best single method, as expected. As the budget increases, it selectively adds more expensive functions, gradually improving F1, though the gains are marginal at higher budgets per unit cost. At the maximum budget, it recovers the unconstrained full ensemble performance. In between, cost-effective multi-scoring incorporates both less and more and expensive methods to maximize detection within the computational constraints.



These results demonstrate that the proposed cost-effective multi-scoring approach can intelligently balance the trade-off between computational expense and hallucination detection effectiveness. It outperforms individual scoring functions and makes selective use of more costly scores to maximize detection performance under a fixed computational budget.

A key consideration in practice would be whether to include potentially more costly multi-generation methods, such as SelfCheckG­PT-NLI. 
In particular as implied by the results presented in Table <ref>, we may be interested in reducing the number of required multiple generations while matching performance at lower computational costs. 
To this end, we compare SelfCheckGPT (as the single best performing method for TriviaQA) alone with SelfCheckGPT combined with P(True) at different numbers of additional responses generated via . 
Here, we count the evaluation of P(True) as one additional LLM call, just as generating one additional output from the generator LLM. 
As presented in Figure <ref>, at one additional generation, SelfCheckGPT (in the degenerate case of only one generated output) combined with P(True) thus recovers P(True). 
As the number of additional generations increases, the combination quickly outperforms SelfCheck alone. 
In particular, we observe that cost-effective multi-scoring with 3 LLM calls is already as good as SelfCheckGPT with 9 LLM calls in our experiment.
This highlights how we can save costs by combining multi-generation methods with other scores while requiring fewer additional generated responses from the generator LLM than if we wanted to achieve the same performance with multi-generation methods alone. 





 §.§ Exploration of Relationships between Scores


As discussed above, different scoring methods can target different kinds of hallucations. 
To explore this empirically, we here explore their relationships via Spearman rank correlations. 
As presented in Figure <ref>, calibrated scores across all different individual scores are positively correlated. 
Meanwhile, the magnitude of their correlations is smaller than would perhaps be  expected if one were to consider hallucinations as a uniform phenomenon. 
At the same time, we can see certain “clusters” of more strongly inter-correlated scores emerge.
For example, multi-generation methods including SelfCheckGPT, HallucinationRail and SimilarityDegree, which check the consistency across multiple generations, show comparably high correlations. 
P(True) and P(True) Verbalised and P(InputContradict) emerge as a similarly correlated cluster of stronger correlations. 
Overall, this supports the idea that different scores can capture distinct information, and the need to empirically select appropriate (combinations of) scores for a given application, as we propose with cost-effective multi-scoring.






 §.§ Experiments across Different LLMs

Most of the scoring methods considered in this work rely on detector LLMs to compute scores for hallucination detection. 
In Table <ref> of Appendix <ref>, thus present additional data for different LLMs, namely ,  and .
Overall, we see that hallucination detection performance is correlated with performance on general LLM benchmarks[E.g., <https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard>]. 
Thus, more overall capable LLMs are likely to perform better at hallucination detection than less capable ones.
However, concrete applications may need to take the inference cost of LLMs into account, where one may not always be able to use the most expensive model. 
We note that cost-effective multi-scoring could here also make use of scores computed via LLMs with different computational costs to find cost-effective combinations. 



 §.§ The Importance of Calibration

To analyze the impact of calibration, we conduct an ablation study by evaluating model performance with and without calibrating the individual scores.
Table <ref> in Appendix <ref> indicates that model performance clearly drops in most of the datasets and metrics without calibration. This demonstrates the benefit of calibration for more accurate and risk-aware downstream applications.



 §.§ Exploring Multi-Generation Assumptions





Our results indicate that methods based on the uncertainty among multiple generations can provide strong signals when such multiple generations are available. 
However, such methods can also suffer from problems in practice. 
Methods based on multiple generations make use of the uncertainty of the generator, that is the distribution over generations z given the input x, for model 𝒢, i.e., p_𝒢(z|x) and the consistency between actual sampled generations z. 
We are interested in classifying whether the candidate output contains a hallucination or not, 
that is estimating and making use of p(y |x, z). 
The generating model's uncertainty p_𝒢(z|x) can be a useful proxy for p(y |x, z), but only under particular conditions. 

On a technical level, sampling multiple answers requires access to the generator's temperature parameter, as temperatures of zero collapse the generations to a single response, as is mentioned, e.g., in <cit.>. 
Therefore, these methods are technically grey-box models, as they require some access to the model's inference parameters. 

More conceptually, the generator's uncertainty (or confidence <cit.>) over p_𝒢(z|x) reflects uncertainty over generated tokens. 
All recently proposed multi-scoring methods we are aware of are based on the idea of scoring the consistency across multiple generations using model-based metrics. 
However, self-consistency across multiple generations is neither a necessary nor sufficient criterion for a hallucination to be present. 
Sufficiency is not given as a model may consistently provide an incorrect response. 
Self-consistency across multiple generations is also not necessary, as many tasks allow for multiple hallucination-free answers that are contradictory to each other. 

As a relatively harmless example, in tasking a model to generate cooking recipes for lunch, the model may generate, among other things, a recipe for a salad and a recipe for a soup. Clearly, the steps in preparing these dishes contain contradictory information, while otherwise being free of hallucinations in themselves. 
Thus, if there is more than one correct response, the self-consistency assessment in multi-generation methods may falsely score an output as likely to be hallucinated. 
As there are numerous situations where there exist multiple correct responses, assuming that there is only one could lead to worse LLM responses, also via a loss of diversity.

Practically, methods based on multiple generations can be costly due to added computational overhead, as we have seen above.
Finally, these methods are based on the assumption that LLMs are calibrated. 
As such, multi-generation methods do not allow for detecting cases where the generator LLM is confident yet wrong. 









§ CONCLUSION


In this work, we compared a comprehensive set of scoring methods to provide calibrated probability scores for the presence of hallucinations in generated LLM outputs.
Overall, we have observed that no single hallucination detection score performs best across all datasets.
Our experiments showed that combinations of scores, as suggested with multi-scoring, can effectively combine complementary signals to yield higher hallucination detection performance than any individual score.
Further, we demonstrate that cost-effective multi-scoring can find the highest performing scores at a given computational budget.

More generally, our findings support the notion that hallucinations can be rather multi-faceted than present a uniform phenomenon.
Thus, detecting hallucinations may require different methods.
In concrete settings, one may be interested in even more fine-grained detection of particular types of hallucinations, such as in specific domains like code generation <cit.>.

The approach outlined in this work requires only a small amount of labeled data to calibrate hallucination scores and combine scores via (cost-effective) multi-scoring.
However, it is important to acknowledge the limitations of the current work. Future research could explore the effectiveness of this approach on more diverse datasets, investigate alternative scoring methods, or extend the methodology to multi-modal models. 
While improvements in LLM performance can be expected to also lower their rate of occurrence, hallucinations are unlikely to go away completely <cit.>.
Thus, even as models improve further, detecting hallucinations is likely to remain relevant for applying LLMs in a reliable and trustworthy manner in future.

In summary, our work presents a promising approach for detecting hallucinations in LLM outputs by combining multiple scoring methods in a cost-effective way, offering a pathway towards more reliable and trustworthy language models that can be applied in real-world settings with more confidence.


















§ ADDITIONAL RESULTS














 Paper Title: Learning to Embed Distributions via Maximum Kernel Entropy.




Floquet Engineering of Polaritonic Amplification in Dispersive Photonic Time Crystals
    Vladimir M. Shalaev
    August 5, 2024
=====================================================================================




§ ABSTRACT 
Empirical data can often be considered as samples from a set of probability distributions. Kernel methods have emerged as a natural approach for learning to classify these distributions. Although numerous kernels between distributions have been proposed, applying kernel methods to distribution regression tasks remains challenging, primarily because selecting a suitable kernel is not straightforward. Surprisingly, the question of learning a data-dependent distribution kernel has received little attention. In this paper, we propose a novel objective for the unsupervised learning of data-dependent distribution kernel, based on the principle of entropy maximization in the space of probability measure embeddings. We examine the theoretical properties of the latent embedding space induced by our objective, demonstrating that its geometric structure is well-suited for solving downstream discriminative tasks. Finally, we demonstrate the performance of the learned kernel across different modalities.




§ INTRODUCTION

Most discriminative learning methods conventionally assume that each data point is represented as a real-valued vector. In practical scenarios, however, data points often manifest as a 'set' of features or a 'group' of objects. A quintessential example is the task of predicting a health indicator based on multiple blood measurements. In this case, the single data point of a patient has multiple, or a distribution of, measurements. One approach to accommodate such cases involves representing each input point as a probability distribution. Beyond mere convenience, it is more appropriate to model input points as distributions when dealing with missing data or measurement uncertainty, as often encountered when facing the abundance of data, which commonly presents a challenge for data-rich fields such as genetics, neuroscience, meteorology, astrophysics, or economics.


The task of regressing a mapping of probability distributions to a real-valued response is known as distribution regression. Distribution regression has been successfully applied in various fields, such as voting behavior prediction <cit.>, dark matter halo mass learning <cit.>, human cancer cells detection <cit.>, brain-age prediction <cit.>, among others <cit.>. The versatility and effectiveness of this framework underscore its power in solving complex problems <cit.>. Kernel methods have become a widely used approach for solving distribution regression tasks by exploiting a kernel between distributions  referred to as a distribution kernel. Despite the multitude of proposed kernels, the practical application of kernel methods remains challenging due to the nontrivial choice of the appropriate kernel. While some efforts focus on identifying kernels with broad applicability and favorable statistical properties <cit.>, others aim to tailor kernels to the geometric characteristics of specific input spaces <cit.>. Remarkably, the question of learning data-dependent kernels has received limited attention. This study is thus driven by a fundamental question: What are the underlying principles that facilitate the unsupervised learning of an effective kernel, one that optimally encapsulates the data properties and is well suited for discriminative learning on distributions?

In this work, we leverage a key insight: an appropriate selection of the distribution kernel enables the embedding of a set of distributions into the space of covariance operators. Building on this theoretical idea, we claim that quantum entropy maximization of the corresponding covariance operator is a suitable guiding principle to learn data-dependent kernel. This, combined with a careful design of kernel parametrization, let us to devise a differentiable optimization objective for learning data-specific distribution embedding kernels from unlabeled datasets (i.e. unsupervised) (<Ref>). We show that the entropy maximization principle facilitates learning of the latent space with geometrical configuration suitable for solving discriminative tasks <cit.>. We empirically demonstrate the performance of our method by performing classification tasks in multiple modalities.

In summary, our unsupervised data-dependent distribution kernel learning framework introduces a theoretically grounded alternative to the common practice of hand-picking kernels. Such framework could be further leveraged for generalizing existing learning approaches <cit.> catalyzing the use of distribution-based representations within the broader scientific community.



§ PRELIMINARIES

We first introduce the main concepts necessary to formalize our learning framework: kernel mean embeddings and covariance operator embeddings.



 §.§ Kernel Embeddings of Distributions


Consider an input space 𝒳 and a positive-definite (p.d.) kernel k: 𝒳×𝒳→ℝ. Let ℋ be the corresponding reproducible kernel Hilbert space (RKHS) induced by such kernel. Consider a probability distribution P∈𝒫(𝒳). The kernel mean embedding map embeds the distribution P as a function in Hilbert space:


    μ_P≡μ(P) ∫_𝒳 k(x, ·)   dP(x) = ∫_𝒳ϕ(x)    dP(x) ,

where ϕ: 𝒳→ℋ is a feature map such that ϕ(x) = k(x, ·).

Importantly, if the kernel k is characteristic <cit.>, the mapping μ: 𝒫(𝒳) →ℋ is injective, implying that all information about the original distribution is preserved in ℋ. This last property underscores much of power under recent applications of kernel mean embeddings <cit.>. 
The natural empirical estimator for the kernel mean embedding approximates the true distribution with a finite sum of Dirac delta functions:

    μ̂_P1/N∑_i=1^N ϕ(x_i) ∈ℋ

where x_1, …, x_N ∼P are N empirical i.i.d. samples. The estimator has a dimension-free sample complexity error rate of 𝒪(N^-1/2) <cit.>.

Additionally, we denote d_k: 𝒳×𝒳→ℝ_≥ 0 a kernel metric induced by a kernel k,

    d_k(x, x^') = ϕ(x) - ϕ(x^') _ℋ  .

Note that d_k is a metric on 𝒳 if feature map ϕ is injective.



 §.§ Covariance Operators and Entropy

A second way of mapping a probability distribution to a Hilbert space can be defined by means of a covariance operators. For a given feature map ϕ(x) = k(x, ·): 𝒳→ℋ[Assuming k to be a continuous positive definite (p.d.) kernel and 𝒳 to be compact.], and a given probability distribution P∈𝒫(𝒳), the covariance operator embedding is defined as:

    Σ_P∫_𝒳ϕ(x) ⊗ϕ(x)   dP(x)

where ⊗ is a tensor product. Σ_P is a self-adjoint positive semi-definite (p.s.d.) operator acting on ℋ. Such operator can be seen as a mean embedding w.r.t. the feature map x ↦ϕ(x) ⊗ϕ(x) and therefore, for a universal kernel k, the map P↦Σ_P is injective (see <cit.>).

Similarly to <Ref>, the natural empirical estimator is:

    Σ̂_P = 1/N∑_i=1^Nϕ(x_i) ⊗ϕ(x_i)

where x_1, …, x_N ∼P are N i.i.d. samples.

For a translation-invariant kernel k(x, x^') = ψ(x - x^') normalized such that k(x, x) =  1, the covariance operator Σ_P is a density operator <cit.>. Henceforth, entropy measures can be applied to it, and the quantum Rényi entropy of the order α can be defined as:

    𝒮_α(Σ_P) 1/1 - αlogtr[ (Σ_P)^α]  = 1/1 - αlog∑_iλ_i^α

where {λ_i }_i are the eigenvalues of Σ_P.
The Von Neumann entropy can be seen as a special case of Rényi entropy in the limit α→ 1. However, in our work, we focus primarily on the second-order case of Rényi entropy, i.e. α = 2 (see <cit.> for an in-depth overview of the properties and theory of quantum entropies).



§ UNSUPERVISED DISTRIBUTION KERNEL LEARNING



 §.§ Distribution Regression


In this section, we discuss the key topics in distribution regression, including problem setup, the notion of a 2-stage sampling process, and the common solutions to regression employing kernel methods.

Distribution regression extends the common regression framework to the setup where covariates are given as probability distributions available only through samples. Formally, consider the task of finding a regressor f: 𝒫(𝒳) →𝒴 from the dataset of samples Γ_M = {(P_i, y_i)}_i=1^M where P_i ∈𝒫(𝒳) are distributions provided as a set of i.i.d. empirical samples x_1, …, x_N_i∼P_i (see <cit.> for a comprehensive analysis). A viable approach to solving this problem is to define a kernel K: 𝒫(𝒳) ×𝒫(𝒳) →ℝ that is universal in 𝒫(𝒳). By setting up such a kernel K, we can utilize kernel-based regression techniques, with SVM often being the preferred method for classification tasks <cit.> or Kernel Ridge Regression (KRR) when the space 𝒴 is continuous <cit.> [As pointed out in <cit.>, under mild conditions, KRR can also be used for classification problems.]. To this end, several kernels have been proposed over time (see details in <Ref>). 

One possibility, proposed by <cit.>, is to introduce a kernel in the input space, the so called embedding kernel k_emb: 𝒳×𝒳→ℝ and exploit the induced mean embeddings μ_emb: 𝒫(𝒳) →ℋ_emb to map input distributions to points in RKHS. Subsequently, to define a second level kernel, distribution kernel K_distr: ℋ_emb×ℋ_emb→ℝ between points in the RKHS ℋ_emb. The simplest choice for such a distribution kernel is the linear kernel:

    K_l(μ_P, μ_Q) ⟨μ_P, μ_Q⟩_ℋ_emb
    = ∬_𝒳×𝒳 k(x, x^')   d P(x)   d Q(x^') .

A standard alternative to the linear kernel is a Gaussian kernel with bandwidth parameter λ > 0:

    K_RBF(μ_P, μ_Q) exp( - λ/2‖μ_P - μ_Q‖_ℋ_emb^2 ) = exp( - λ/2 d_k_emb(μ_P, μ_Q)^2 )   ,

which was shown to be universal in 𝒫(𝒳) <cit.>. The Gaussian kernel K_RBF can be computed from the linear kernel K_l as d_k_emb(μ_P, μ_Q)^2 = K_l(P, P) + K_l(Q, Q) - 2 K_l(P, Q).

In practice, we only have access to a finite number of samples of each input distribution, thus the distribution kernel is approximated using the natural estimator for the kernel mean embedding. The related excess risk for the regression solution is analyzed in <cit.>.





 §.§ Dataset Embedding


Instead of using standard kernels designed to encapsulate the geometry of the input space, we consider learning a data-dependent kernel, tailored to the specific properties of the dataset. In a similar vein, <cit.> proposed learning an optimal kernel (or equivalently, a feature map) jointly with the classifier to address the text modality. In this work, we focus on an unsupervised problem, aiming to learn a data-dependent kernel between probability distributions without access to classification labels.

We first introduce proper parametrization to ensure both expressivity and robustness followed by the definition of the optimization objective. Leveraging the idea of 2-level kernel setup, we define the embedding kernel as

    k_θ: 𝒳×𝒳→ℝ = k_emb(f_θ(x), f_θ(x^')) .

where f_θ is a trainable encoder function f_θ: 𝒳→𝒵, 𝒵 is a latent encoding space, and k_emb is a kernel defined on the latent space k_emb: 𝒵×𝒵→ℝ. The encoder function f_θ transforms every input probability distribution P∈𝒫(𝒳) into a distribution over the latent space ℙ_θ∈𝒫(𝒵) [By the definition of the encoding process, P_θ is a push-forward measure. For an empirical probability distribution q=∑_iδ(x_i)∈𝒫(𝒳) and a measurable map f: 𝒳→𝒵, the push-forward measure f # q ∈𝒫(𝒵) is defined as ∑_iδ(f(x_i)).] (<Ref>). Furthermore, we denote RKHS corresponding to the kernel k_emb as ℋ_emb and the kernel mean embedding map as μ_emb (see <Ref>).

    P∈𝒫(𝒳)  P_θ∈𝒫(𝒵)  μ_emb(P) ∈ℋ_emb .

These transformations define mean embeddings for each input probability distributions through the first level, embedding kernel k_emb (<Ref>). The second level, distribution kernel K_distr: ℋ_emb×ℋ_emb→ℝ is defined over the mean embeddings μ_emb(P)'s. We can now consider the embedding dataset 𝒟_M = {P_i }_i=1^M as an empirical covariance operator (see <Ref>), i.e.

    𝒟_M Σ̂_𝒟 = 1/M∑_P∈𝒟_MK_distr(μ_emb(P), ·) ⊗ K_distr(μ_emb(P), ·)  .

As Σ̂_𝒟 encapsulates information about the entire dataset, we term it dataset embedding. With the assumption that the dataset 𝒟_M is sampled i.i.d. from the (unknown) true meta-distribution 𝒟, Σ̂_𝒟 is a natural estimator to approximate the true covariance operator Σ_𝒟. To simplify the notation we use Σ_𝒟 in place of Σ_𝒟 unless required by the context.

Both the embedding kernel k_emb and the distribution kernel K_distr remain fixed throughout the training, learning happens by adjusting the parametrization of the latent space encoder f_θ. Such separation ensures expressivity while conforming to all technical requirements for a distribution kernel. Throughout the paper, we make the following assumptions on the latent space 𝒵 and embedding kernel k.

Latent space 𝒵 is a compact subset of ℝ^d. Kernel k_emb: 𝒵×𝒵→ℝ is a p.d. characteristic translation-invariant kernel k_emb(z, z^') = f( z - z^'^2) such that -f^' is completely monotone on (0, ∞) (see Definition 2.2.4 of <cit.>) and ∀ z ∈𝒵:   k_emb(z, z) = 1.



To facilitate our optimization process (which will be explained shortly), we opt for 𝒵=𝕊^d-1 (the d-dimensional hypersphere) and Gaussian kernel both for k_emb and K_distr (see <Ref>). We retain other suitable choices as potential avenues for future research.



 §.§ Unsupervised Optimization Objective


This dataset level representation depends on the choice of first and second level kernels k, K and, in turn, on the trainable function f_θ parameterized by the set of parameters θ. In this work, we propose learning the parameters θ to maximize quantum entropy of the dataset embedding, i.e.,

    θ = {𝒮_2(Σ_𝒟)  - logtr[ (Σ_𝒟)^2 ] } .

As we will describe in brief, optimizing this target has clear benefits inherited from the underlying geometry of the setup. But, first, we show how to empirically compute 𝒮_2(Σ_𝒟). Building upon previous work <cit.>[See the proof of Proposition 6 in <cit.>.], we exploit the following property of the covariance estimator:

    tr[ (Σ_𝒟)^2 ] = tr[ ( 1/M K_𝒟)^2 ]

where K_𝒟∈ℝ^M × M is the distribution kernel matrix, with [K_𝒟]_ij = K_distr(μ_P_i, μ_P_j).  This equation follows directly from the fact that Σ_𝒟 and 1/M K_𝒟 share the same set of eigenvalues. Leveraging this relationship, we can define tractable unsupervised training loss, which we term Maximum Distribution Kernel Entropy (MDKE), with respect to the parameters of the encoder f_θ:

    ℒ_MDKE(θ)   - 𝒮_2(Σ_𝒟) = logtr[ ( 1/M K_𝒟)^2 ]  = log∑_i=1^Mλ_i^2 ( 1/M K_𝒟)  = log1/M K_𝒟_F^2

where the latter relies on the fact that the Frobenius norm A _F^2 = ∑_iλ_i^2(A), where λ_i(A) are eigenvalues A.

The MDKE objective is differentiable w.r.t. θ for commonly used kernels, provided that the encoder f_θ is differentiable as well. While the entropy estimator 𝒮_2(Σ_𝒟) is convex in the kernel matrix K_𝒟, the objective as a whole is generally not convex in θ. However, in practice, as we show in <Ref>, mini-batch Stochastic Gradient Descent (SGD) proves to be an effective method for optimizing this objective. The effectiveness of this optimization process is significantly influenced by the parameters of the Gaussian kernels. We elaborate on the methodologies for kernel bandwidth selection in <Ref>.

The Frobenius norm formulation in the loss <Ref> significantly reduces computational complexity. However, as we have observed in some of our experiments, it can lead to the collapse of small eigenvalues of K_𝒟, particularly near the optimal value of the objective. To address this challenge we introduced a regularized version of the loss ℒ_MDKE-R that incorporates optional regularization, based on the determinant K_𝒟 inspired by the connection with Fekete points (see details in <Ref>).



 §.§ Geometrical Interpretation
 

The optimization objective is specifically designed to minimize the variance within each distribution (inner-distribution variance) while simultaneously maximizing the spread of distributions over the compact latent space 𝒵=𝒮^d-1. This shaping of the distributions embeddings in the latent space facilitates easier separation in downstream tasks. In this section we show that the geometry of the optimal (w.r.t. the MDKE loss) configuration of mean embeddings in the RKHS attains describe properties. For doing so we leverage the notion of distributional variance  𝕍_ℋ (Definition 1 in <cit.>). 


For a set of M probability distributions 𝒟_M, distributional variance 𝕍_ℋ(𝒟_M) of the mean embeddings in the RKHS ℋ is given by

    𝕍_ℋ(𝒟_M) 1/Mtr[ G ] - 1/M^2∑_i=1^M∑_j=1^MG_ij ,

where G is the M × M Gram matrix of mean embeddings in ℋ, i.e. G_ij = ⟨μ_P_i, μ_P_j⟩_ℋ <cit.>.


Here we show that the distributional variance 𝕍_ℋ can be equally reformulated into two separate contributions:

    𝕍_ℋ(𝒟_M) ≡1/M∑_i=1^M‖μ_P_i‖_ℋ^2  - ‖μ_P̅‖_ℋ^2 ,

where P̅ denotes mixture distribution with elements of 𝒟_M being uniformly weighted mixture components (see proof in the <Ref>).

The relevance of distributional variance for MDKE objective is established by the following result. 


For a set of M probability distributions 𝒟_M, the second-order Rényi entropy 𝒮_2 of the empirical covariance operator embedding Σ̂_𝒟 induced by the choice of Gaussian distribution kernel K_RBF over points in the RKHS ℋ_emb, - as defined in <Ref>, - is upper bounded by the distributional variance 𝕍_ℋ_emb(𝒟_M), i.e.,


    1/2 γ𝒮_2(Σ̂_𝒟) ≤𝕍_ℋ_emb(𝒟_M)


where γ is the bandwidth of the distribution kernel K_RBF.

The proof of this proposition is provided in <Ref>. This result formalizes the fact that our objective increases distributional variance, pushing up the average squared norm of mean embedding of input distributions while minimizing squared norm of the mean embedding of the mixture. We further explore the geometrical implications of such optimization by formalizing connection between the variance of the distribution and the squared norm of the its mean embedding in RKHS.


Under <Ref>, the maximum norm of kernel mean embedding is attained by Dirac distributions {δ_z }_z ∈𝒵.

This result is trivial due to the fact that the set of mean embeddings is contained in the convex hull of { k_emb(z, ·) }_z ∈𝒵, and, under <Ref>, ∀ z ∈𝒵:  k_emb(z, ·) _ℋ_emb^2 = k_emb(z, z) = 1.


Under <Ref>, uniform distribution 𝒰(𝒵) is a unique solution of

    _P ∈𝒫(𝒵){μ_emb(P) _ℋ_emb^2 ≡∬_𝒵×𝒵 k_emb(z, z^')   d P(z)   d P(z^') }.


The key intuition here comes from the fact that minimization of the squared norm of mean embedding in RKHS could be seen as minimization of total interaction energy over the given surface where the potential is defined by the kernel k. Thus <Ref> is a special case of Theorem 6.2.1 of <cit.>. Similar setup was used w.r.t. Gaussian potential over the unit hypersphere in Proposition 1 from <cit.>. The reformulation in <Ref> together with <Ref> immediately suggests that the framework could be seen as an extension of the Hyperspherical Uniformity Gap <cit.> to infinite-dimensional spaces of probability distributions This extension maintains the goal of reducing variance among input distributions while maximizing the separability between their means. See <Ref> for a broader explanation of the connection.

More generally, utilizing the fact that under <Ref>, the kernel metric d_k_emb (see <Ref>) is a monotonically increasing function of Euclidean metric on the latent space, we establish a precise connection between the generalized variance <cit.> and the norm of the  mean embedding. Further details can be found in <Ref>.

An attempt to directly optimize 𝕍_ℋ_emb using SGD resulted in significantly weaker outcomes. While a thorough mathematical explanation necessitates further investigation, we contend that this issue aligns with the recurring challenge reported across various studies regarding direct optimization over Maximum Mean Discrepancy (MMD). We hypothesize that optimal solutions exhibit similar geometric configurations, while the entropy of the covariance operator providing a smoother objective. Nonetheless, 𝕍_ℋ_emb retains its value as an insightful and intuitive measure for describing the geometric configuration of the learned system.



 §.§ An Illustrative Example

We use a simple example to illustrate the connection between geometrical configurations of embedded distributions and distribution kernel entropy 𝒮_2(Σ_𝒟) (see <Ref>). We sample a number of points from 6 different Gaussian distributions and project on a sphere 𝕊^2 varying their projected variance γ. As γ decreases, the distributional variance of the overall distribution of Gaussians increases (<Ref>). For very small γ each distribution converges to a point (a Dirac distribution). This results in the entropy interpolating between lower and upper bounds, demonstrating how entropy behaves in response to changes in distribution variance. <Ref> showcases the behavior of two terms comprising distributional variance (<Ref>): the average kernel norm of the distributions alongside the kernel norm of the mixture. The increase in entropy and variance corresponds to a 'flattening' effect on the spectrum of the distribution kernel matrix. This example provides a simplified picture of how input distributions configurations influence kernel entropy.




 §.§ Limitations


Runtime complexity. The applicability of a data-dependent distribution kernel to solving discriminative tasks relies on the structure of the dataset being well-suited for distribution regression modeling. The model performs best when the number of input distributions is relatively small (e.g., thousands rather than millions), while the number of samples per distribution is large. It is crucial to note that the computational complexity of the proposed method, which is a common concern in practical applications, is most favorable for the tasks described. A detailed analysis of runtime complexity can be found in <Ref>.

Broader impact. We wish to emphasize that the distribution regression framework has emerged as a powerful tool for analysis and predictive modeling, especially in domains where traditional methods face challenges, including social science, economics, and medical studies. We urge researchers and practitioners applying distribution regression in these areas to give special consideration to issues such as bias, fairness, data quality, and interpretability, -  aspects that are currently under-researched in the context of distributional regression, largely due to the relative novelty.



§ RELATED WORK





 §.§ Distribution Regression

Distribution regression was introduced in <cit.>, while the seminal work of <cit.> provides a comprehensive theoretical analysis of this regression technique. A natural approach to solving distribution regression problems involves using kernels between measures. Notable examples include the Fisher kernel <cit.>, Bhattacharyya kernel <cit.>, Probability product kernel <cit.>, kernels based on nonparametric divergence estimates <cit.>, and Sliced Wasserstein kernels <cit.>. <cit.> proposed leveraging the mean embedding of measures in RKHS, and <cit.> provided theoretical guarantees for learning a ridge regressor from distribution embeddings in Hilbert space to the outputs. Distribution kernels have been successfully applied in various kernel-based methods, such as SVM <cit.>, Ridge Regression <cit.>, Bayesian Regression <cit.>, and Gaussian Process Regression <cit.>. They have also been adapted for different modalities like distribution to distribution regression <cit.>, sequential data <cit.>, and more. For an in-depth exploration of the diverse methodologies employed in distributional regression settings, we invite readers to consult <ref>.



 §.§ Matrix Information Theory

Quantum entropy, including Rényi entropy, is a powerful metric to describe information in a unique way (see <cit.> for foundational insights). <cit.> designed the measure of entropy using operators in RKHS to mimic Rényi entropy's behavior, offering the advantage of direct estimation from data. <cit.> applied von Neumann entropy of the density matrix to the covariance operator embedding of probability distributions, thereby defining an information-theoretic framework utilizing kernel methods. In machine learning, especially within self-supervised learning (SSL) setups, entropy concepts have recently found novel applications. Our study builds on most recent developments <cit.> by applying quantum Rényi entropy to the covariance operator in RKHS.



§ EXPERIMENTS


We here demonstrate that our proposed method successfully performs unsupervised learning of data-dependent distribution kernel across different modalities. The experimental setup is divided into two phases: unsupervised pre-training and downstream regression classification using the learned kernel.

For each dataset, we select a hold-out validation subset with balanced classes, while the remainder of the dataset is utilized for unsupervised pre-training. We use mini-batch ADAM <cit.> with a static learning rate of 0.0005. We report mini-batch based (instead of epoch based) training dynamics as our tasks do not require cycling over the entire dataset to converge to the optimal loss value. All experiments use Gaussian kernel both as an embedding kernel and distribution kernel, the hyperparameter selection is performed as described in <Ref>.

Once the samples encoder f_θ is learned, we employ it to compute distribution kernel Gram matrix, used as an input to the Support Vector Machine (SVM) for solving downstream classification tasks. A grid search with 5 splits (70/30) is conducted to optimize the strength of the squared l_2 regularization penalty C, exploring 50 values over the log-spaced range {10^-7, …, 10^5}. The best estimator is then applied to evaluate classification accuracy on the validation subset, which we report.

Additional experiments exploring the application of data-dependent distribution kernels in domains where distribution regression models are less common, such as image and text, are presented in <Ref>.



 §.§ Flow Cytometry



Flow Cytometry (FC) is a widely used technique for measuring chemical characteristics of mixed cell population. Because population-level properties are described through (randomized) sampling of cells, FC is used as a canonical setup of distribution regression. For this study we used a dataset <cit.> where more than 100.000 cells are measured per each patient (subject). For each cell a total of ten parameters are reported, hence, we treated each subject as an empirical distribution over ℝ^10. We considered downstream classification tasks on two different sets of labels. The first ('Tissue' classification) contains peripheral blood (pB) and bone marrow (BM) samples from N=44 subjects. The second ('Leukemia' classification) presents healthy and leukemia BM cell samples, N=50. Classes were balanced in both cases.
l8.2cm
 Distribution regression accuracy on Flow Cytometry datasets. Code available at https://github.com/hidden-for-review.






Model     2cTissue     2cLeukemia 
 2-5 
                   Acc.     Var.        Acc.     Var.    
 
GMM-FV             93.07%            ± 0.308            94.80%          ± 0.186                

SW1                87.10%            ± 0.530            95.07%          ± 0.111                

SW2                81.71%            ± 0.341            95.30%          ± 0.224                

MMD Linear         82.42%            ± 0.840            90.57%          ± 0.208                

MMD Gaussian       81.71%            ± 0.574            92.23%          ± 0.216                

MMD Cauchy         81.57%            ± 0.662            93.77%          ± 0.080                

MMD IMQ            82.89%            ± 0.698            91.43%          ± 0.217                

MDKE Rand        77.50%                ± 0.002                    89.50%                ± 0.003                

MDKE No Reg.    95.30%                ± 0.002                    92.46%                ± 0.002                
 
MDKE Reg.        98.89%                ± 0.010                    94.57%                ± 0.005                
 








We sampled 16 subjects for Tissue classification and 20 subjects for Leukemia for training. Unsupervised learning was performed over the entire dataset. The encoder f_θ was parametrized by a 2-layers neural network (NN) with ReLU nonlinearity and l_2 normalized output (on the unit hypersphere 𝕊^9). Per each subject we sampled a small percentage of cells, and we report performance for the sample size of 200. We repeated each training and testing phase for 100 times to track the variance induced by this aggressive subsampling.

To demonstrate the impact of unsupervised pre-training, we compared several methods across multiple configurations (<Ref>):

a) Kernels on distributions. This group includes Fisher kernels applied to parametric Gaussian Mixture Model (GMM) estimates, as suggested in <cit.>, along with Sliced Wasserstein-1 and Sliced Wasserstein-2 kernels <cit.>.

b) MMD kernels. Here, we employ a Gaussian embedding kernel for mean embeddings, marked as 'MMD' (Maximum Mean Discrepancy) in the table. This category includes various options for the distribution kernel, such as linear, Gaussian, Cauchy, and inverse multiquadrics.

c) MDKE. We explore various configurations of the encoder optimized with the MDKE objective. We report performance for randomly initialized encoder, and for unsupervised pre-trained encoder with and without regularization. Random initialization happens only once, and all subsequent accuracy measurements are taken using the same encoder.

The variance reported for each model is measured across multiple runs to demonstrate the effect of the sampling. Importantly, the optimization of the MDKE objective results in embedding kernels with significantly lower variance compared to non-data-specific kernels.




§ CONCLUSION

In this work, we presented an unsupervised way of learning data-dependent distribution kernel. While previous studies in distribution regression predominantly relied on hand-crafted kernels, our work, in contrast, demonstrates that entropy maximization can serve as a powerful guiding principle for learning adaptable, data-dependent kernel in the space of distributions. Our empirical findings show that this technique can not only serve as a pre-training step to enhance the performance of downstream distribution regression tasks, but also facilitate complex analyses of the input space. The interpretation of the learning dynamics induced by the proposed objective relies on a theoretical link between the quantum entropy of the dataset embedding and distributional variance. This theoretical link, which we have proven, enables us to approach the optimization from a geometrical perspective, providing crucial insights into the flexibility of the learned latent space encoding.

We hope that theoretically grounded way of learning data-dependent kernel for distribution regression tasks will become a strong alternative to the common practice of hand-picking kernels. More broadly, our results present a methodology for leveraging the distributional nature of input data along side the novel perspective on the encoding of complex input spaces. This highlights the potential to extend the application of more advanced learning methods, embracing the ever-increasing complexity of data by going beyond more conventional vector-based representations.



  
*Acknowledgments. 
We thank the Allen Institute for Brain Science founder, Paul G. Allen, for his vision, encouragement, and support.

abbrvnat







§ PROOFS


In this section, we present the proofs for the propositions outlined in our study. To ensure clarity, we will first restate the setup and introduce necessary concepts.

We define the input space as 𝒳, and 𝒫(𝒳) represents the space of probability distributions over 𝒳. Consider a dataset of M probability distributions, denoted as 𝒟_M = {P_i ∈𝒫(𝒳) }_i=1^M. With a p.d. characteristic embedding kernel k: 𝒳×𝒳→ℝ, the corresponding RKHS ℋ, and the feature map ϕ: 𝒳→ℋ, we define the mean embedding map μ: 𝒫(𝒳) →ℋ such that μ_P = μ(P) ∫_𝒳ϕ(x)   d P(x). A p.d. translation-invariant characteristic distribution kernel K: 𝒫(𝒳) ×𝒫(𝒳) →ℝ is defined using the mean embeddings of corresponding distributions. For simplicity, μ_i denotes μ(P_i).

Additional concepts essential for our proofs include the Gram matrix of mean embeddings G ∈ℝ^M × M, representing the inner products of mean embeddings in the dataset, i.e., G  [⟨μ_i, μ_j ⟩_ℋ]_ij. The kernel matrix K_𝒟∈ℝ^M × M with respect to the distribution kernel K is denoted as K_𝒟 [K(P_i, P_j)]_ij. We also recall the definition of distributional variance 𝕍_ℋ (see <Ref>):

    𝕍_ℋ(𝒟_M) 1/Mtr[G] - 1/M^2∑_i=1^M∑_j=1^MG_ij

These definitions and notations will be referenced throughout the proofs.



 §.§ Kernel Norms Gap and Distributional Variance


For both cases of input distributions being empirical probability distributions or continuous densities, we define mixture distribution P̅, with a slight abuse of notation:

    P̅(x) 1/M∑_i=1^M P_i(x)


For the mixture distribution P̅ and the Gram matrix G, the following relationship holds:

    μ_P̅_ℋ^2 = 1/M^2∑_i=1^M∑_j=1^MG_ij



Proof. We begin by recalling that the inner product between mean embeddings μ_i and μ_j in ℋ is given by:

    ⟨μ_i, μ_j ⟩_ℋ = ∬_𝒳×𝒳k(x, x^')   d P_i(x)   d P_j(x^')


Substituting this into the expression for the Gram matrix, we have:

    1/M^2∑_i=1^M∑_j=1^MG_ij   = 1/M^2∑_i=1^M∑_j=1^M∬_𝒳×𝒳k(x, x')    d P_i(x)   d P_j(x')
       = ∬_𝒳×𝒳 k(x, x')    d ( 1/M∑_i=1^MP_i(x) )   d ( 1/M∑_j=1^MP_j(x') ) 
       = ∬_𝒳×𝒳 k(x, x')    d P̅(x)   d P̅(x')  = μ_P̅_ℋ^2


This completes the proof of <Ref>.

By incorporating <Ref> into the definition of distributional variance (see <Ref>), and noting that the trace of G is the sum of squared norms of input distributions (i.e.,  tr[G] = ∑_i=1^M μ_P_i_ℋ^2), we obtain:

    𝕍_ℋ(𝒟_M) ≡1/M∑_i=1^Mμ_P_i_ℋ^2 - μ_P̅_ℋ^2

This result offers a new and intuitive perspective on distributional variance, conceptualizing it as the difference between the average squared norm of individual input distributions and the squared norm of the mixture distribution.



 §.§ Pairwise Distance and Distributional Variance




For a dataset 𝒟_M of M probability distributions, we define the average pairwise distance between kernel mean embeddings in ℋ as 𝕁_ℋ(𝒟_M), given by:

    𝕁_ℋ(𝒟_M) 1/M^2∑_i=1^M∑_j=1^M||μ_i - μ_j||_ℋ^2



For the distributional variance 𝕍_ℋ of the dataset 𝒟_M, the following relationship holds:

    𝕍_ℋ(𝒟_M) ≡1/2·𝕁_ℋ(𝒟_M)



Proof. Starting with the definition of 𝕁_ℋ(𝒟_M):

    𝕁_ℋ(𝒟_M)    = 1/M^2∑_i=1^M∑_j=1^M||μ_i - μ_j||_ℋ^2
       = 1/M^2∑_i=1^M∑_j=1^M⟨μ_i - μ_j, μ_i - μ_j ⟩_ℋ
       = 1/M^2∑_i=1^M∑_j=1^M( ⟨μ_i, μ_i ⟩_ℋ + ⟨μ_j, μ_j ⟩_ℋ - 2⟨μ_i, μ_j ⟩_ℋ) 
       = 1/M^2( 2 M ∑_i=1^M⟨μ_i, μ_i ⟩_ℋ - 2∑_i=1^M∑_j=1^M⟨μ_i, μ_j ⟩_ℋ) 
       = 2 ( 1/M∑_i=1^M⟨μ_i, μ_i ⟩_ℋ_diagonal of G - 1/M^2∑_i=1^M∑_j=1^M⟨μ_i, μ_j ⟩_ℋ)

Recognizing that ∑_i=1^M⟨μ_i, μ_i ⟩_ℋ = tr[G], we can equate expression in the brackets to 𝕍_ℋ(𝒟_M), leading to:

    𝕍_ℋ(𝒟_M) ≡1/2·𝕁_ℋ(𝒟_M)

<Ref> will be instrumental in further proofs, establishing a crucial link between distributional variance in ℋ and quantum entropy of the covariance operator embedding Σ_𝒟.



 §.§ Distribution Kernel Entropy Upper-Bound


In this section, we provide the proof for the key theoretical result stated in <Ref>.

Consider a dataset 𝒟 consisting of probability distributions {P∈𝒫(𝒳) }_i sampled i.i.d. from an unknown meta-distribution 𝔻. We assert that the second-order Rényi entropy 𝒮_2 of the empirical covariance operator embedding Σ_𝒟, induced by the choice of Gaussian distribution kernel K_RBF over points in the RKHS ℋ, is upper-bounded by the distributional variance 𝕍_ℋ(𝒟):

    1/2 γ𝒮_2(Σ̂_𝒟) ≤𝕍_ℋ(𝒟)

where γ is the bandwidth of the distribution kernel K_RBF.

Starting from the properties of 𝒮_2(Σ̂_𝒟) stated in  <Ref>:

    𝒮_2(Σ̂_𝒟) = 𝒮_2 ( 1/M K_𝒟) = -log∑_i=1^M∑_j=1^M( 1/M K_ij) ^2  = -log( 1/M^2∑_i=1^M∑_j=1^MK_ij ^2)


Applying Jensen's inequality, considering the concavity of the log, to <Ref>, we have:

    𝒮_2(Σ̂_𝒟)    ≤ - 1/M^2∑_i=1^M∑_j=1^MlogK_ij^2
       = - 1/M^2∑_i=1^M∑_j=1^Mlogexp(-γ/2μ_i - μ_j_ℋ^2)^2
       = - 1/M^2∑_i=1^M∑_j=1^Mlogexp(-γμ_i - μ_j_ℋ^2)
       =  γ( 1/M^2∑_i=1^M∑_j=1^Mμ_i - μ_j_ℋ^2) 
       = γ·𝕁_ℋ(𝒟) 
       = 2 γ·𝕍_ℋ(𝒟)

<Ref> uses the definition of K_RBF from <Ref>, <Ref> is derived from the definition of the average pairwise distance in <Ref>, and <Ref> follows from <Ref>. This completes the proof of <Ref>, establishing the upper bound for the second-order quantum Rényi entropy of the covariance operator embedding Σ_𝒟 in terms of the distributional variance in RKHS ℋ. 

Technical Note: It is important to note that 𝒮_2 is typically measured using the base-2 logarithm (log_2) rather than the natural logarithm. However, the proof remains accurate with a proper re-scaling to account for the change in the logarithmic base.



 §.§ Generalized Variance in RKHS



In this section we prove the connection between generalized variance and norm of the kernel mean embedding.

So far we established the squared norm of mean embedding is maximized by Dirac points (zero variance) and is minimized by uniform distribution (max variance). We now formalize the intuition that larger squared norm in embedding kernel RHKS corresponds to smaller variance in the latent space. We first note that under  <Ref>, the kernel metric d_k_emb (see <Ref>) induced by the choice of k_emb is a monotonically increasing function of Euclidean metric on the latent space, thus is representative of the geometry of encoded input distributions. Which let us use a generalized notion of variance here.



(<cit.>) Let X be a random variable which takes values in a Fréchet space ℱ equipped with seminorm ·_α. And suppose that X is square-integratable, in a sense that 𝔼 X _α^2 < ∞. Let μ∈ℱ be a Pettis integral of X (i.e. generalization of the mean). Generalized variance of X w.r.t. seminorm ·_α is defined as following

    Var_α[X] 𝔼 X - μ_α^2   .



Note that for a random variable in RKHS defined as a push-forward of a probability distribution P∈𝒫(𝒵) over the latent space, i.e. X = z ∼ϕ#P satisfies conditions of <Ref> with ℱ = ℋ, ·_α = ·_ℋ_emb, and a Pettis integral being a kernel mean embedding. Denote the described variance as Var_ℋ_emb[P]. We now show that


Under <Ref>, for every P∈𝒫(ℤ),

    Var_ℋ_emb[P] = 1 - μ_emb(P) _ℋ_emb^2.





  
Proof.

    Var_ℋ_emb[P]    = 𝔼_x ∼ Pϕ(x) - μ_P _ℋ_emb^2 
       = ∫ϕ(x) - μ_P _ℋ_emb^2   d P(x) 
       = ∫⟨ϕ(x) - μ_P, ϕ(x) - μ_P ⟩_ℋ_emb  d P(x) 
       = ∫( ϕ(x) _ℋ_emb^2 + μ_P _ℋ_emb^2 - 2 ⟨ϕ(x), μ_P ⟩_ℋ_emb)   d P(x) 
       = ∫ϕ(x) _ℋ_emb^2   d P(x)  + ∫μ_P _ℋ_emb^2   d P(x)  - 2 ∫⟨ϕ(x), μ_P ⟩_ℋ_emb  d P(x) 
       = ∫ 1   d P(x)  + μ_P _ℋ_emb^2 ∫ 1   d P(x)  - 2  ⟨∫ϕ(x)   d P(x) , μ_P ⟩_ℋ_emb
       = 1 + μ_P _ℋ_emb^2 - 2 μ_P _ℋ_emb^2 
       = 1 - μ_P _ℋ_emb^2

Which concludes the proof.

The proposition demonstrates a motivation behind training the encoder to minimize variance in the latent space by maximizing the (average) squared norm of the mean embeddings.



§ PRACTICAL ASPECTS OF LEARNING





 §.§ Kernel Hyperparameter Selection


In practice, the effectiveness of the optimization process is significantly influenced by the parameters of the Gaussian kernels. Setting the bandwidth parameter γ either too low or too high can hinder the model's ability to learn effectively. This issue is a well-known challenge in working with kernel methods, where determining the optimal kernel bandwidth has been an area of extensive study.

In our work, we have employed an empirical approach, which involves adjusting γ based on the idealized structure of the dataset after its projection onto the latent space. Specifically, we set γ such that 1/γ equals 10 times the average distance to the nearest neighbor in the set of points sampled uniformly on 𝕊^d-1 (inspired by the experimental approach in <cit.>). The number of points is chosen to be the number of distributions in the training set for the distribution kernel. For the embedding kernel, it is set to the number of distributions in a batch multiplied by the number of samples per distribution used for unsupervised training.



 §.§ Optional Regularization




As the theoretical maximum of 𝒮_2 entropy is attained when the spectrum of 1/M K_𝒟 is uniform, the optimal encoder with respect to ℒ_MDKE also tends to maximize the determinant of the kernel Gram matrix K_𝒟. This relationship is intuitive when considering that the determinant of a matrix is the product of its eigenvalues. The points that maximize such determinant K_𝒟 are theoretically known as Fekete points <cit.> and, in our case, are relative to the distribution kernel. Fekete points have been shown by <cit.> to be an optimal configuration for learning kernel interpolants, making them particularly suitable for downstream tasks framed as kernel regression. As such, encoders optimized under the ℒ_MDKE objective facilitate more accurate and robust performance in subsequent regression tasks.

In practice, we found that optimizing the MDKE objective posed certain numerical challenges, particularly due to the tendency of too small eigenvalues in the distribution kernel matrix K_𝒟 to collapse near the optimal value of the objective when using large batch size for training. To mitigate this issue and prevent undesirable optimization behavior, we have introduced a regularization term to the original objective. This term is inspired by the concept of Fekete points configuration, leading to the following loss function:

    ℒ_MDKE-R(θ)     -𝒮_2(Σ_𝒟) + ϵ·Ω(Σ_𝒟) 
       = log1/M K_𝒟_F^2 - ϵ·log| 1/M K_𝒟|


Here, ϵ serves as a hyperparameter to control the strength of regularization. The regularization term Ω(Σ_𝒟) is designed to stabilize the optimization process by counteracting the effects of the collapse of small eigenvalues. The empirical evidence supporting the effectiveness of regularization in stabilizing the training dynamics for the sentence representation learning experiment (as detailed in <Ref>) is showcased in <Ref>.



 §.§ Runtime Complexity


Scalability of the kernel methods is typically a key concern when it comes to practical applications. The runtime complexity of the proposed method could be decomposed into two components:


  * Computation of the distribution kernel Gram matrix. This involves computing O(N^2) inner products between distributions where N is a number of distributions. Each inner product involves computing matrix of pairwise distances between samples from each distribution, in the case of computing the Gaussian kernel over points on the hypersphere, the runtime complexity is the complexity of multiplying matrices ℝ^M × d where M is a number of samples. All computations could be efficiently parallelized.

  * Solving regression using distribution kernel Gram matrix. The complexity comes from matrix inverse and is typically estimated to be O(N^3) with a range of methods proposed to reduce the complexity <cit.>.

With this being said, we want to highlight that the runtime complexity of the proposed method is most favorable exactly for those tasks and datasets where distributional regression is an appropriate modeling approach. A significant  number of samples per  distribution ensures a high accuracy in approximating the kernel between a pair of distributions, and, at the same time, a relatively small number of distributions in the dataset alleviates issues related to storing the distribution kernel Gram matrix in memory and performing computations on it




§ CONNECTION TO OTHER FRAMEWORKS



 §.§ Distributional Regression Landscape



As the distributional regression task differs from common Machine Learning (ML) setups where inputs are given as vectors, the effective practical solution requires unique considerations.

The most obvious approach would be to ignore the fact that inputs are given as distributions and learn classifier on the space of samples, with a proper aggregation of posteriors (e.g. with a simple sum over histograms). This approach, while being simple, has been shown not to yield practically useful results, and was explicitly excluded from reported performance on different tasks by different authors <cit.>.


Viable approaches to solving distributional regression could be, approximately, split up into the following categories.



  
Discriminate generative models. The idea is to fit each input distribution to a parametric family, e.g. Gaussian Mixture Model (GMM), to use available closed-form solutions to compute kernel or similarity or distance between distributions. This group of methods originated from the work on hidden Markov models and it's applications to processing sequence modalities, like text, DNA, proteins, and more. Early work <cit.> on learning discriminative classifier for generative models leveraged the fact that parametric models forms a Riemannian manifold with local metric given by Fisher information, they derived kernel function termed Fisher kernel suitable for running SVM between generative models. Driving motivation was classification between hidden Markov models, with the developed method being applied to DNA and protein sequence analysis. Following the same modeling approach of analysing DNA sequences with discriminative models between Markov models, <cit.> proposed to use Bhattacharyya distance between distributions from exponential families to derived so-called Expected Likelihood Kernel. <cit.> explored the method of computing kernel between distributions as the integral of the product of pairs of distributions, termed Probability Product Kernel. A new family of kernels was applied to the same setup of discriminating task on text modality, hidden Markov models for biological data. The model was also successfully applied for analysis of linear dynamical systems for time series data. Critical advantage of this work was access to computationally effective way of computing kernel for the distributions without having access to analytical closed-form parametrization (only relying on samples). In <ref> we used fitting distributions to GMM with Fisher Kernel applied to learn parameters as a baseline for the performance on the task.



  
Point clouds. This group includes methods that model each input distribution as a set of points (also known as 'point cloud' or 'feature group' in computer vision) and use kernel or similarity function defined on sets. Large portion of the methods in this group arised in computer vision (CV) field when local features extractor were widely employed to pre-process images yielding either per-image histograms or sets of low dimensional vectors. The group includes kernels based on nonparametric divergence estimates, quantized set kernels, and so-called 'nearest-neighbor' set kernels to name a few <cit.>. Such kernel were employed to many CV tasks though the successfully application required not only a good kernel but also high-quality feature extractor. Special attention goes to methods leveraging  Wasserstein distance (including both kernel-based and similarity-based solutions). Wasserstein distance as a metric-aware discrepancy measure for probability distributions is a natural choice of deriving kernels or similarity functions between point clouds <cit.>. While being computationally problematic for large-scale problems, Sliced Wasserstein kernels were succesfully adopted <cit.> as they provided both reliable way for the point set comparison with a reduced cost leveraging sliced formulation. In <ref> we compared performance of Sliced Wasserstein-1 and Sliced Wasserstein-2 kernels, with the latter yielding significantly lower performance. Such a behavior is consistent with theoretical analysis stating that Sliced Wasserstein-1 has favoriable properties when compared to kernel based on Wasserstein-2 distance.



  
Kernel Mean Embeddidngs. <cit.> proposed to leverage  kernel mean embedding of measures in RKHS, so that the distributional regression could now be casted to a regression the corresponding Hilbert space. While the original work leveraged kernel between RKHS embeddings to train SVM, <cit.> provided theoretical guarantees for the  for learning Ridge regressor from distribution embeddings in the Hilbert space to the outputs. <cit.> applied the same approach in Bayesian regression settings, and <cit.> used it for Gaussian Process Regression. The setup gives a great deal of flexibility by choosing the kernel in RKHS, with a few being tested in practice. Gaussian kernel is a common choice, due to it's universality <cit.>. Among others, inverse multiquadric (IMQ) is a popular choice, typically paired with random features to improve runtime complexity of the algorithm. Linear kernel in RKHS, despite not being universal, was reported to produce competitive results in multiple practical setups. In <ref> we compared performance for 4 different kernels in RKHS, namely linear, Gaussian, Cauchy, and IMQ, -  all defined as functions of Maximum Mean Discrepancy (MMD), which gives as a way to compute the value of the kernel from finite number of samples with high accuracy. While all kernels demonstrated different performance in terms of the test accuracy, the difference reported is not that substantial.

Distribution kernels were successfully applied in different kernel-based methods, such as , as well as in different modalities, like distribution to distribution regression <cit.>, distribution regression for sequential data <cit.>, and more. In cases where input data is naturally represented as a probability distribution but not immediately applicable for existing distribution regression solutions, pre-processing or encoding of inputs is required. <cit.> proposed learning a latent space representation for input data to apply distribution regression in the text modality, with the encoder being trained jointly with the classifier. To our best knowledge, methods for learning data-dependent kernel for solving distributional regressions were not previously reported.



 §.§ Hyperspherical Uniformity Gap



The link between entropy maximization in the space of distribution, the gap between average norm and norm of the mixture distribution marginalized over the dataset (i.e. 'average' distribution), and properties of distributions that minimize and maximize kernel mean embedding norm (as described in <Ref>) unveils a subtle yet significant connection to the concept of the Hyperspherical Uniformity Gap (HUG), introduced in <cit.>. HUG has been developed to generalize the phenomenon of neural collapse observed in supervised classification settings. In our approach, working directly with samples from input distributions grants us explicit access to the 'grouping' of points in the input space. Even though the HUG framework setup does not have a notion of 'grouping', we can close the gap by noting that class labels provided with the dataset implicitly create 'groupings' of points, which can be interpreted as empirical samples drawn from latent probability distributions (one for each class).

However, a notable distinction lies in the dimensional aspect: HUG focuses on the distribution of points on the surface of a finite-dimensional hypersphere, whereas our work encompasses an infinite-dimensional hyperball setting, given the use of points in kernel induced RKHSs. Furthermore, the loss function introduced in our study presents a unified optimization objective that weaves together both the inner-group and inter-group dynamics while being derived from first principles. In the HUG framework these two aspects are addressed separately.

Establishing a more formal connection between these frameworks emerges as a promising direction for future research. Such an endeavor could offer a novel perspective on supervised classification, particularly by conceptualizing class prototypes as probability distributions rather than mere vectors. This exploration might bridge the gap between these distinct approaches, enriching our understanding of classification paradigms in high-dimensional spaces.




§ ADDITIONAL EXPERIMENTS



In the following section, we present additional experiments on learning data-dependent distribution kernels in the image and text domains. While representing text as empirical samples from the finite space of tokens (i.e., words from the dictionary) is quite common, the choice to model images as histograms over pixel positions is more subtle. We demonstrate that, in both scenarios, unsupervised pre-training of the encoder yields distribution kernel that achieves strong performance on downstream classification tasks, showcasing the versatility of the proposed learning framework in scenarios where distribution regression formulations are uncommon.



 §.§ Image Classification Tasks


MNIST <cit.> and Fashion-MNIST <cit.> consist of 28 × 28 pixel grayscale images divided into 10 classes. We considered each individual image to be a probability distribution (via rescaling pixel intensities so that l_1(image)=1) over the discrete space of pixel positions (i.e., histogram). Given this support space, the encoder f_θ is a discrete map which we implemented as a table lookup (i.e., embeddings) from pixel indices to the points on the hypersphere 𝒮^31. Embeddings were initialized by sampling points uniformly. Gradients of the MDKE objective with respect to the embeddings parameters were computed via automatic differentiation using projected gradient steps to ensure that the embeddings remain on the hypersphere. The small size of the support space enables the exact computation of the inner product between kernel mean embeddings of input distributions <Ref> and, subsequently, the distribution kernel <Ref> during both training and evaluation. This ensured a lower variance of the accuracy for the downstream classification.

Performing MNIST classification upon pre-training with our unsupervised encoder significantly improves the baseline (random initialization of latent embeddings) accuracy of 85.0% by reaching a plateau at 92.15%.

To understand the data-dependency of our encoding procedure, we analyzed the latent spaces of MNIST and Fashion-MNIST datasets.  The visualization of pixel-level interaction, computed using Gaussian kernel Gram matrix, reveals complex, dataset-specific interactions (<Ref>, <ref>). Spectral clustering using the kernel Gram matrix provides deeper insight into the pixel interaction landscape. The chart shows clusters of pixels with correlated intensities (<Ref>, <ref>), with the number of clusters set to 10 empirically.






 §.§ Text Classification Tasks


To assess our method's performance in a larger discrete support space, we utilized the "20 Newsgroups" <cit.>, a multi-class text classification dataset. We reduced the size of the dataset to 5 classes (resulting in 2,628 sentences and 38,969 unique words) by subsampling both training and test subsets. We treated sentences as empirical distributions over words, assuming word sets to be enough for topic classification, despite no positional info.

The encoder f_θ mirrored the setup used in the MNIST case (<Ref>), with l_2 normalized word embeddings on 𝕊^31.

However, while in the MNIST case embeddings computations were performed exactly, here considering the entire embedding kernel Gram matrix is impractical due to its large size. Instead, we optimized embeddings by randomly sampling 20 words per sentence, making the inner product between embeddings a stochastic approximation. This setup is meant to confirm that the optimization of the proposed MDKE objective yields a solution that is robust w.r.t. the excessive risk induced by first-level subsampling. 

During the unsupervised pre-training phase, we observed a steady decrease in the training loss (<Ref>) despite the small batch size (50 sentences with total of 1,000 words). Importantly, for this experiment we employed a regularized version of the objective (MDKE-R). We discuss the rationale and empirical evidence supporting the use of this regularization scheme in <Ref>. During evaluation, to reduce computational complexity, we sample 2,000 sentences for the train and 1,000 for the test split, keeping classes balanced. The maximum classification accuracy achieved was approximately 89.3%, while the random initialization performance averaged at 37. 5% (<Ref>). The framework’s high accuracy in downstream classification showcases its prowess in learning potent latent representations, even when dealing with large, finite input distributions.





§ IMPLEMENTATION DETAILS



In this section, we provide an example illustrating the implementation of the proposed method using the PyTorch framework. All experiments were performed on a single machine with 1 GPU and 6 CPUs.

Functions to compute distribution kernel Gram matrix:


def pairwise_kernel(x, gamma1):
    B, T = x.size(0), x.size(1)
    X_unroll = x.reshape(B*T, -1)
    dist = (X_unroll[:, None, :] - X_unroll[None, :, :])**2
    dist = torch.sum(dist, dim=2)
    G = F.avg_pool2d(
        dist[None, : , :],
        kernel_size=(T, T),
        stride=(T, T)
    )
    return torch.exp(-(gamma1/2.) * G.squeeze(0))

def distribution_kernel_gram(x, gamma1, gamma2):
    Gxy = pairwise_kernel(x, gamma1=gamma1)
    Gx = Gy = torch.diag(Gxy)
    G = Gx[:, None] + Gy[None, :] - 2*Gxy
    K = torch.exp(-(gamma2/2.) * G)
    return Gxy, K


Distribution kernel entropy estimator and the MKDE loss:


def distribution_kernel_entropy(K):
    B = K.size(0)
    Cov = (1/B) * K
    return -(Cov ** 2).sum().log2()

def mkde_loss(encoder, X, gamma1, gamma2):
    # X.shape is (n_distributions, n_samples, d_input)
    Z = encoder(X)
    # Z.shape is (n_distributions, n_samples, d_latent)
    _, K = distribution_kernel_gram(Z, gamma1, gamma2)
    # K.shape is (n_distributions, n_distributions)
    loss = -distribution_kernel_entropy(K)
    return loss


The code presented here follows the setup presented in <Ref> using Gaussian kernel both as embedding and as a distribution kernel. Other kernels could be used by adjusting implementation of both helper functions accordingly. When training on large datasets, <cit.> might be used to avoid memory overflow in average pooling.




 Paper Title: Generalized Out-of-Distribution Detection and Beyond in Vision Language Model Era: A Survey.












Miyai et al.: 
Generalized Out-of-Distribution Detection and Beyond in Vision Language Model Era: A Survey


§ ABSTRACT 
Detecting out-of-distribution (OOD) samples is crucial for ensuring the safety of machine learning systems and has shaped the field of OOD detection. Meanwhile, several other problems are closely related to OOD detection, including anomaly detection (AD), novelty detection (ND), open set recognition (OSR), and outlier detection (OD). To unify these problems, a generalized OOD detection framework was proposed, taxonomically categorizing these five problems. However, Vision Language Models (VLMs) such as CLIP have significantly changed the paradigm and blurred the boundaries between these fields, again confusing researchers. In this survey, we first present a generalized OOD detection v2, encapsulating the evolution of AD, ND, OSR, OOD detection, and OD in the VLM era. Our framework reveals that, with some field inactivity and integration, the demanding challenges have become OOD detection and AD. In addition, we also highlight the significant shift in the definition, problem settings, and benchmarks; we thus feature a comprehensive review of the methodology for OOD detection, including the discussion over other related tasks to clarify their relationship to OOD detection. Finally, we explore the advancements in the emerging Large Vision Language Model (LVLM) era, such as GPT-4V. We conclude this survey with open challenges and future directions.



Anomaly Detection, Novelty Detection, Open Set Recognition, Out-of-Distribution Detection, Outlier Detection, Vision Language Model, CLIP, Large Vision Language Model, Large Multi-modal Model


Generalized Out-of-Distribution Detection and Beyond in Vision Language Model Era: A Survey
    Atsuyuki Miyai, Student Member, IEEE, Jingkang Yang, Jingyang Zhang, Yifei Ming, Yueqian Lin, 
 Qing Yu, Member, IEEE, Go Irie, Member, IEEE, Shafiq Joty, Yixuan Li, Member, IEEE, Hai Li, Fellow, IEEE, Ziwei Liu, Member, IEEE, Toshihiko Yamasaki, Member, IEEE,  Kiyoharu Aizawa, Fellow, IEEE

A. Miyai, Q. Yu, T. Yamasaki, and K. Aizawa are with the Department of Information and Communication Engineering, The University of Tokyo, Japan, 1138656. Q. Yu is also with LY Corporation, Japan, 1028282.


E-mail: {miyai,yamasaki}@cvm.t.u-tokyo.ac.jp, {yu,aizawa}@hal.t.u-tokyo.ac.jp
J. Yang and Z. Liu are with 
S-Lab, School of Computer Science and Engineering, 
Nanyang Technological University, Singapore, 639798.


E-mail: {jingkang001,ziwei.liu}@ntu.edu.sg
J. Zhang, Y. Lin, and Hai Li are with the Department of Electrical and Computer Engineering, Duke University, Durham, NC, United States, 27708.


E-mail: {jz288,yueqian.lin,hai.li}@duke.edu
Y. Ming and S. Joty are with Salesforce AI Research, Palo Alto, CA, United States, 94301. S. Joty is also with NTU, Singapore, 639798 (on leave).


E-mail: {yifei.ming, sjoty}@salesforce.com
G. Irie is with the Department of Information and Computer Technology, Tokyo University of Science, Japan, 1258585.


E-mail: goirie@ieee.org

Y. Li is with the Department of Computer Sciences, University of Wisconsin-Madison, Madison, WI, United States, 53706.


E-mail: sharonli@cs.wisc.edu

Resources: <https://github.com/AtsuMiyai/Awesome-OOD-VLM>.


    Received July 31, 2024; accepted XX XX, XXXX
=========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================





§ INTRODUCTION

A reliable visual recognition system should not only accurately predict known contexts, but also identify and reject unknown examples <cit.>. In critical applications like autonomous driving, the system must alert and cede control to the driver upon encountering unfamiliar scenes or objects not seen during training. However, most existing machine learning models are trained based on the closed-world assumption <cit.>, where the test data is assumed to be drawn i.i.d. from the same distribution as the training data, known as in-distribution (ID). Therefore, the development of classifiers capable of detecting out-of-distribution (OOD) samples is a crucial challenge for real-world applications. This challenge is precisely the focus of research in the field of OOD detection.

While OOD detection primarily focuses on semantic distribution shift, several other tasks share similar goals and motivations, including outlier detection (OD) <cit.>, anomaly detection (AD)  <cit.>, novelty detection (ND)  <cit.>, and open set recognition (OSR) <cit.>. Subtle differences in the specific definitions among these sub-topics have caused confusion in the field, leading to similar approaches being proposed across them. To address this issue, the generalized OOD detection framework was introduced <cit.>. The taxonomy of the generalized OOD detection framework is shown in Fig. <ref>. The generalized OOD detection framework classifies these tasks as special cases or sub-tasks under a unified taxonomy. This framework provides clear definitions and fosters a deeper understanding of each field.



In recent years, the emergence of Vision Language Models (VLMs), represented by CLIP <cit.>, has rapidly accelerated research in the field of Computer Vision. This has changed the paradigm of the recognition field, allowing for zero-shot <cit.> or few-shot learning <cit.> in various domains. VLMs have significantly influenced the aforementioned five problems (OD, AD, ND, OSR, and OOD detection), and the application of VLMs has become a highly notable research field <cit.>. However, alongside this remarkable progress, the paradigm shift with the advent of the VLMs has blurred the boundaries between the five problems. Due to the difficulty of a clear understanding of the distinctions and interrelations between these tasks, each community within the fields is facing significant challenges in identifying the optimal direction to pursue in this VLM era.

In this survey, we introduce a novel unified framework termed generalized OOD detection v2, which extends the previous generalized OOD detection framework and summarizes the evolution of these five problems in the VLM era. To create it, we systematically review the use of VLMs across these five problem areas, tracing their development from the start to the present, and summarize the evolutionary trajectory of each problem. Importantly, our framework reveals that a paradigm shift has caused some fields to become inactive or integrate with others, and the demanding challenges in the VLM era become AD and OOD detection, which is a remarkable finding for each community. In addition to the inter-field evolution, we elaborate on the important shifts in the definition of OOD detection as well as the problem settings and benchmarks, with the contrast of those for related tasks. Then, we conduct a thorough review of the methodology for OOD detection and related tasks in the VLM era, intending to clarify their similarities and differences and inspire future research in OOD detection.

Finally, we introduce the evolution of these problems in the emerging Large Vision Language Model (LVLM) era, such as GPT-4V <cit.> or LLaVA <cit.> (also referred to as Large Multi-modal Models or Multi-modal Large Language Models <cit.>). 
We summarize the definition of each evolving problem, the findings so far, and future challenges.





To sum up, in this survey paper, we make three contributions to the research community:


  * Unified Framework in the VLM era:
We systematically review the evolution of five closely related topics of OD, AD, ND, OSR, and OOD detection in the Vision Language Model era and provide an updated unified framework termed generalized OOD detection v2.
Our framework reveals that the paradigm shift has led to some field inactivity or integration, and the demanding challenges are AD and OOD detection.
We hope that these observations highlight the demanding challenges in the VLM era and foster collaborative efforts among each community.


  * A Comprehensive Survey for OOD Detection in the VLM era:
While comprehensive surveys on OD, AD, ND, OSR, and OOD detection methodologies have been published in recent years <cit.>, this survey is the first to comprehensively overview OOD detection methods specifically in the VLM era.
By connecting with other related tasks, we aim to provide readers with a holistic understanding of the developments and interconnections within each problem, particularly in the context of OOD detection.


  * An Introduction to the Evolution in the LVLM Era:
We further introduce the evolution of each problem in the Large Vision Language Model era. Despite the infant stage of these fields, this survey offers an in-depth introduction to each problem, aiming to facilitate future advancements in this area.


  * Future Research Directions:
We draw readers' attention to the future work necessary for advancing the field in the VLM and LVLM era. We conclude this survey with discussions on open challenges and opportunities for future research.






The paper content is organized as follows. 
In Sec. <ref>, we introduce the new version of generalized OOD detection by summarizing the evolution of the five related fields in the VLM era.
We then overview the two key problems (OOD detection and AD) that have evolved and remain active in Sec. <ref>, with a detailed breakdown of existing methodologies being presented in Sec. <ref> (CLIP-based OOD detection) and Sec. <ref> (CLIP-based AD).
In Sec. <ref>, we introduce early advancements of OOD detection and AD in the LVLM era.
Sec. <ref> and Sec. <ref> feature discussions on potential challenges and future directions.
Finally, we conclude with Sec. <ref>.


§ GENERALIZED OOD DETECTION V2


In this section, we introduce a novel unified framework termed generalized OOD detection v2, which summarizes the evolution of the five related fields in the VLM era. We first revisit the previous generalized OOD detection framework in Sec. <ref>. Next, we introduce the evolution of each problem.



 §.§ Background: Generalized OOD Detection V1


We first briefly revisit a previous generalized OOD detection, which encapsulates five related sub-topics: anomaly detection (AD), novelty detection (ND), open set recognition (OSR), out-of-distribution (OOD) detection, and outlier detection (OD).
These sub-topics can be similar in the sense that they all define a certain in-distribution, with the common goal of detecting out-of-distribution samples under the open-world assumption.
Previously, subtle differences existed among the sub-topics in terms of the specific definition and properties of in-distribution (ID) and OOD data. 

To provide a clear definition, a generalized OOD detection framework was proposed <cit.>. The taxonomy for generalized OOD detection is shown in Fig. <ref>. It is based on the following four bases: (1) Distribution shift to detect: The task focuses on detecting either covariate shift (OOD samples from a different domain) or semantic shift (OOD samples from a different semantic). (2) ID data type: The in-distribution (ID) data contains either a single class or multiple classes. (3) Whether the task requires ID classification: Some tasks require classification of the ID data, while others do not. (4) Transductive vs. inductive learning: Transductive tasks require all observations (both ID and OOD), while inductive tasks follow the common train-test scheme.
According to the above taxonomy, these five problems can be clearly categorized as shown in Fig. <ref>: Anomaly detection is categorized into sensory anomaly detection, which deals with covariate shift, and semantic anomaly detection, which deals with semantic shift. Novelty detection falls under the same category as semantic anomaly detection.
When addressing a multi-class scenario that necessitates ID classification, both open-set recognition and out-of-distribution detection are encompassed within this category. The main difference between OSR and OOD detection was the benchmark setup <cit.> (Sec. <ref> (c)).
Outlier detection belongs to a different category from the other tasks, as this problem is transductive (it has access to all observations).

For the detailed definition of each task, we refer the readers to the previous generalized OOD detection survey paper <cit.>.



 §.§ Evolution of Each Problem in VLM Era


We review how each problem has evolved in the VLM era. To make a fair judgment, we comprehensively investigated papers that use VLMs from top venues and summarized them in Table <ref>. 
Our survey revealed that CLIP <cit.> is predominantly used as the VLM for OOD detection and other sub-tasks, and other VLMs <cit.> are rarely utilized. Therefore, we focus on CLIP as the target VLM in this survey and refer to OOD detection using CLIP as CLIP-based OOD detection. Similarly, we will prefix other tasks with “CLIP-based" (CLIP-based AD). As OOD detection research is primarily focused on the image domain, we conduct a survey of other tasks within the image domain that are common and have strong connections to OOD detection research. For instance, our survey does not cover video domain tasks <cit.> due to their limited connection to OOD detection.



(a) Sensory AD → CLIP-based AD
Sensory AD has continued to develop as a common problem setting for CLIP-based AD,
inheriting the challenges of traditional sensory AD <cit.>. As shown in Table <ref>, the first appearance in a top venue was at CVPR 2023, and since then, a total of six papers have been published in top venues. In addition, there are numerous other papers <cit.>. Moreover, in terms of benchmarks, in addition to the commonly used MVTec-AD <cit.>, the largest industrial anomaly detection dataset VisA <cit.> has also become a standard benchmark in the field. Therefore, it is evident that sensory AD has become a highly active and noteworthy field in the VLM era.



(b) Semantic AD/ND  → Inactive
Research on semantic AD/ND appears to become inactive in the VLM era. 
As shown in Table <ref>, there are only two papers, TMLR 2022 <cit.> and CVPR 2024 <cit.>. However, the CVPR 2024 work <cit.> aims to build a generalist anomaly detector that solves many AD tasks, including sensory AD and semantic AD, and is not primarily focused on semantic AD. 
The reasons for the inactivity include saturation of performance for one-class semantic AD/ND, and incompatibility of methods with CLIP for multi-class semantic AD/ND. As for one-class semantic AD/ND, TMLR <cit.> exists, but the performances with common CIFAR and ImageNet-30 datasets have already achieved around 99%. As for multi-class semantic AD/ND, a common approach is to treat ID classes as a single class, but treating ID classes as a single class is less compatible with CLIP's class-wise discriminative capability.



(c) OSR  → CLIP-based OOD Detection 
We consider that OSR has been integrated into CLIP-based hard OOD detection. According to Table <ref>, there are no top venue publications on OSR research in the VLM era. Originally, the main difference between OSR and OOD detection was the benchmark setup <cit.>. OSR typically divides the classes in the one dataset into some known (ID) classes and unknown (OOD) classes, as seen in MNIST-4/6 <cit.> CIFAR-4/6 <cit.>, CIFAR-50/50 <cit.>, and TinyImageNet-20/180 <cit.>. However, in recent years, some works on CLIP-based OOD detection incorporate the benchmark setup of OSR and create new benchmarks such as ImageNet-10/ImageNet-20 <cit.> and ImageNet-protocol <cit.> for hard OOD detection. Therefore, the boundary between OOD detection and OSR has effectively disappeared, and all research in the VLM era has been integrated into OOD detection. 

Nevertheless, while pure OSR research is declining, some studies have used the term “open-set" in the context of domain generalization <cit.>. 
These studies deviate from the original scope of OSR research and are rather closely aligned with the field of domain generalization <cit.>. Therefore, within our generalized OOD detection v2, we do not classify these studies as falling under OSR research. We will discuss them in the context of full-spectrum OOD detection, a research field combining generalization and detection, in Sec. <ref>. 




(d) OOD Detection  → CLIP-based OOD Detection
OOD detection is a highly active research area in the VLM era.
As shown in Table <ref>, there are many papers in top venues, indicating a high interest from the community. Additionally, as mentioned above, OSR has been integrated with OOD detection as a field of hard OOD detection <cit.>. Therefore, it is expected that OOD detection will continue to grow and develop further.



(e) OD  → Inactive
OD has become less active in the VLM era. Previously, OD was used for open-set semi-supervised learning <cit.>, learning with open-set noisy labels <cit.>, and novelty discovery <cit.>.
The reason for the inactivity is that the use of CLIP led to a reduction in training costs and only a small amount of data needs to be collected, eliminating the need for large amounts of unlabeled data and reducing the need to consider noisy data.
However, recently, Liang  <cit.> proposed Unsupervised Universal Fine-Tuning, a new problem setting for CLIP-based OD in ICML2024. Unsupervised Universal Fine-Tuning assumes a more realistic problem setting for unsupervised tuning of the downstream task with CLIP where some OOD samples are included in the unlabeled samples.
With this new problem setting, there is still a possibility that OD will become active in the future.
However, as OD is not currently an active area, we exclude OD from the main discussion of this survey. Unsupervised Universal Fine-Tuning is deeply related to OOD detection and will be discussed in detail in Sec. <ref>.





 §.§ Discussion

Through Sec. <ref>, we found that previously mixed fields have been correctly organized in the VLM era, and that the focus should be on OOD detection and sensory AD. These fields are still developing, with an increasing number of methodologies and benchmarks, and are expected to become more active in the future. Note here that this does not mean that other fields have come to an end. For example, one reason why one-class semantic AD/ND has not been studied is the saturation of performance <cit.>. If more fine-grained and challenging datasets could be constructed, the field could be reactive. We put this in out-of-scope for this survey paper, but this is an important future challenge.



§ OVERVIEW OF EACH PROBLEM IN VLM ERA


In addition to the above inter-field evolution, we emphasize that the advent of VLMs has significantly changed the field of OOD detection itself. In this section, we present an overview of CLIP-based OOD detection, highlighting the key changes in the problem definition, the problem setting, and benchmarks. In addition, we also present an overview of CLIP-based AD in the hope that the understanding of each field will lead to a deeper understanding of CLIP-based OOD detection. For items that remain unchanged from traditional problems, such as background, applications, and evaluation, we refer the readers to the original generalized OOD detection paper <cit.>.



 §.§ CLIP-based Out-of-Distribution Detection


Definition
The definition of CLIP-based OOD detection differs significantly from that of conventional OOD detection.
Conventional OOD detection aims to detect test samples drawn from a distribution that is different from the training distribution. As another definition, OOD detection is defined as a task to detect test samples that the model cannot or does not want to generalize <cit.>. 
However, for CLIP-based OOD detection, CLIP has a vast amount of knowledge, so the OOD sample is completely unrelated to the distribution of the CLIP's pretraining data or the CLIP's own generalization ability. Therefore, traditional definitions cannot adequately describe the definition of CLIP-based OOD detection.

Unlike the previous definition, CLIP-based OOD detection is defined as follows <cit.>: CLIP-based OOD detection aims to detect samples that do not belong to any ID class text provided by the user. Given a pre-trained model, a classification task of interest is defined by a set of class labels , 
which we refer to as the ID classes. The semantic distribution is represented by the distribution P(). CLIP-based OOD detection aims to detect test samples that come from the distribution with the semantic shift from the ID classes, P()≠ P(). 
Following the definition of the generalized OOD detection framework <cit.>, ideal OOD detectors should keep the classification performance on test samples from ID class space , and reject OOD test samples with semantics outside the support of .



Problem Setting
CLIP-based OOD detection focuses on solving the image classification task in a computationally efficient way. Unlike traditional OOD detection settings, which primarily involve training an ID classifier with whole ID data, CLIP-based OOD detection primarily focuses on a zero-shot <cit.> (without utilizing ID images) or few-shot <cit.> (utilizing only a few ID images) setting. Each detailed definition of both settings are described later in Sec. <ref>. The field is advancing towards greater computational efficiency, requiring minimal or no training data.



Benchmark
Most recent works in CLIP-based OOD detection use high-resolution and large-scale datasets such as ImageNet <cit.>. 
The common ImageNet OOD benchmark uses ImageNet as ID and other datasets <cit.> as OOD. However, in this common benchmark, the semantics between ID and OOD are far, which may allow easy distinction between the ID and OOD. Therefore, recent works use more challenging OOD benchmarks where they split ImageNet classes into ID and OOD categories for hard OOD detection <cit.>. The representative datasets are ImageNet-20 <cit.>, ImageNet-10 <cit.>, and the recently proposed ImageNet-protocol <cit.> created by dividing into multiple variations of ID/OOD pairs from ImageNet-1K. This creation strategy initially focused on OSR but has recently been repurposed for OOD detection. These changes in the datasets shift OOD detection closer to the real world and make it a more challenging and practical task.





 §.§ CLIP-based Anomaly Detection


Definition
Unlike OOD detection, the definition of anomaly detection (AD) has not changed between conventional AD and CLIP-based AD. AD is intended for use in specific circumstances (industrial inspection), where samples that deviate from predefined normality are considered an anomaly <cit.>. Whether a model can generalize is irrelevant to the definition of “Anomaly". Therefore, even with the emergence of CLIP, the definition has not changed. 



Problem Setting
CLIP-based AD focuses on solving anomaly classification and segmentation in a computationally efficient way. 
Anomaly classification, like conventional AD, is a binary classification task that distinguishes between normality and abnormality. Anomaly segmentation, also following conventional AD, involves segmenting the location of anomalies. Like CLIP-based OOD detection, CLIP-based AD also primarily focuses on a zero-shot <cit.> (without utilizing images in the target dataset) or few-shot <cit.> (using only a few normal images in the target dataset) setting. Each detailed definition of zero-shot and few-shot settings is described later in Sec. <ref>. As another shift, conventional AD created separate models for each category  <cit.>, while CLIP-based AD creates a single unified model across multiple categories <cit.>, which leads to a more computationally efficient approach.

One key difference from CLIP-based OOD detection is that CLIP-based OOD detection does not involve localization tasks, while these are mainstream in CLIP-based AD. This will be discussed in detail in Sec. <ref>. 



Benchmark
Most works on CLIP-based AD tackle industrial inspection <cit.>. 
As for the benchmarks, besides the common MVTec-AD dataset <cit.>, the more challenging dataset VisA <cit.> has been newly employed <cit.>. The VisA benchmark includes objects with complex structures such as printed circuit boards and multiple instances with different locations within a single view, making it one of the most challenging datasets currently available in the open datasets. Since the pioneering work in CLIP-based AD (WinCLIP <cit.>) used MVTec-AD and VisA, many subsequent works have also used these datasets <cit.>.








§ CLIP-BASED OOD DETECTION: METHODOLOGY


In this section, we introduce the methodologies for CLIP-based out-of-distribution (OOD) detection. Fig. <ref> presents the timeline for representative methodologies for CLIP-based OOD detection. Table <ref> presents representative methods. We introduce methods for zero-shot OOD detection in Sec. <ref>, few-shot OOD detection in Sec. <ref>, and other research directions in Sec. <ref>. For each methodology, we categorize them by the type of training and whether additional OOD prompts were employed. 



 §.§ Zero-shot Out-of-Distribution Detection


Zero-shot OOD detection was proposed in 2021 by Fort  <cit.>. Since then, a growing number of methods have been proposed year by year.



Definition of Zero-shot OOD Detection
In zero-shot OOD detection, the term “Zero-shot" refers to the non-use of ID images during both training and inference phases. For instance, the method with additional training with auxiliary datasets (non-use of ID images) can be regarded as a zero-shot method <cit.>. The method with the pre-processing of the ID class texts can also be regarded as a zero-shot method <cit.>.





  §.§.§ Training-free Methods


a. With OOD Prompts
CLIP-based OOD detection started in this setting. 
The earliest work is ZeroOE <cit.>. ZeroOE feeds the potential OOD labels to the textual encoder of CLIP. However, the method of using known OOD labels is infeasible for real-world applications.
To solve this problem, ZOC <cit.> proposed to train an OOD label generator based on the visual encoder of CLIP and use the generated pseudo-OOD labels for OOD detection.
However, when dealing with large-scale datasets encompassing a multitude
of ID classes, the label generator may not generate effective candidate OOD labels, resulting in poor performance.
Building on these early works <cit.>, recent works focus on how to obtain high-quality OOD labels through either (i) OOD label retrieval <cit.> or (2) OOD label generation <cit.>. (i) One of the representative retrieval-based methods is NegLabel <cit.>. NegLabel selects high-quality OOD labels from extensive corpus databases by calculating the distance between an extracted OOD label and ID label. (ii) One of the representative generation-based methods is EOE <cit.>.
EOE utilizes Large Language Models (LLMs) to produce high-quality OOD labels. 
By modifying the prompts given to the LLM, EOE can be generalized to a variety of tasks, including far and near OOD detection.



b. Without OOD Prompts
In zero-shot OOD detection, many methods utilize OOD labels, but the difficulty and cost of creating these labels pose challenges. To address these issues, Ming  <cit.> proposed MCM, which uses only ID labels to detect OOD. MCM is a simple approach that devises softmax scaling to align visual features with textual concepts for OOD detection. Despite its simplicity, MCM has high effectiveness and scalability, and it serves as a crucial baseline in CLIP-based OOD detection.
Building on the concept of MCM, Miyai  <cit.> proposed GL-MCM, which extends MCM by just adding a local MCM score to enhance the fine-grained detection capability in local regions. SeTAR <cit.> enhances MCM and GL-MCM by changing the model's weight matrices using a simple greedy search algorithm.
We consider these methods to be post-hoc methods for CLIP-based OOD detection in that they directly employ an ID classifier for OOD detection. Due to their simplicity and high scalability, these post-hoc methods can bring fundamental performance improvements for many subsequent methods <cit.>. Therefore, we expect that this field should be developed further in the future, reflecting the trajectory of the field before CLIP emerged <cit.>.





  §.§.§ Auxiliary Training-based Methods


CLIPN <cit.> is the only auxiliary training-based method for zero-shot OOD detection. CLIPN aims to empower the logic of saying “no" within CLIP, and it designs a novel learnable “no" prompt and an additional “no" text encoder to capture negation semantics within images. To create an additional text encoder, CLIPN needs to be pre-trained on the CC-3M dataset <cit.>.
While the extensive pre-training of CLIPN may indeed lead to intensive computations and lower scalability, once pre-trained, it performs zero-shot OOD detection across a wide range of domains with comparable performance to few-shot OOD detection methods <cit.>.
In the future, within this field, there is potential for zero-shot open-vocabulary OOD detection to further advance the realm of zero-shot OOD detection, which will be discussed later in Sec. <ref>.



 §.§ Few-shot Out-of-Distribution Detection


Few-shot OOD detection was concurrently proposed by Miyai <cit.> and Ming <cit.> in June 2023. Since then, it has become the most active research area in CLIP-based OOD detection.



Definition of Few-shot OOD Detection
CLIP-based few-shot OOD detection aims to detect OOD images using only a few labeled ID images.
In few-shot OOD detection, the term “Few-shot" refers to the use of a few ID images during training or inference phases. For instance, the method with additional training with a few ID images can be regarded as a few-shot method <cit.>. Even without training, if a method uses a few ID images as a reference, we regard it as a few-shot method <cit.>. 
Regarding the number of shots, it is common to experiment with 1-shot to 16-shot <cit.>, following the closed-set setting <cit.>.



  §.§.§ ID Training-based Methods


a. Without OOD Prompts
Few-shot OOD detection began in this setting. Ming  <cit.> proposed PEFT-MCM for CLIP-based OOD detection, which demonstrates the effectiveness of combining parameter-efficient tuning methods (prompt learning <cit.> or adapter <cit.>) and MCM <cit.>. Concurrently, Miyai <cit.> proposed LoCoOp, a pioneer prompt learning approach for few-shot OOD detection. 
LoCoOp enhances CoOp's <cit.> OOD detection capabilities by performing OOD regularization with local OOD features. LoCoOp is the simplest prompt learning method and serves as a crucial baseline in few-shot OOD detection. 
Unlike LoCoOp, which utilizes non-ID local regions for OOD regularization, GalLoP <cit.> proposes an approach that utilizes local ID regions to enable a more fine-grained distinction between ID and OOD samples. GalLoP learns a diverse set of prompts by utilizing both global and local visual representations, thereby enhancing the detection capabilities.



b. With OOD Prompts
Similar to zero-shot OOD detection, recent works in few-shot OOD detection utilize additional OOD prompts <cit.>.
As representative methods, LSN <cit.> and NegPrompt <cit.> were proposed concurrently. They state that the simple negative prompts added “not” (“not a photo of a [cls]”) fail to capture the dissimilarity for identifying OOD samples. 
Therefore, by preparing negative prompts and training with them, LSN and NegPrompt can learn suitable negative prompts, enabling more accurate detection of OOD samples.
The difference between LSN and NegPrompt lies in their approach to the use of negative prompts. LSN prepares unique negative prompts for each class and learns suitable negative prompts for each class. In contrast, NegPrompt prepares multiple negative prompts common to all ID classes and trains them to learn generic templates representing the negative semantics of any given class labels. Also, NegPrompt tested the performances in the hard OOD detection setting with ImageNet-protocol <cit.>, outperforming LoCoOp and CoOp. 
Alternatively, IDPrompt <cit.> takes a different approach by introducing ID-like prompts, which are designed for OOD features that are close to the ID features. It extracts ID-like OOD regions in ID training images and trains ID-like prompts with these extracted OOD data. In a unique direction, LAPT <cit.> proposes an automatic sample collection strategy that retrieves or generates training ID images only with ID class names, which achieves high performance without image collection and annotation costs. LAPT then performs distribution-aware prompt learning, which distinguishes between ID class and OOD class tokens. LAPT is positioned within the context of more efficient few-shot OOD detection in this survey paper since it requires generating or retrieving “ID images" for the data collection.

In the context of few-shot OOD detection, recently, Li  <cit.> proposed a new problem setting called open-vocabulary OOD (OV-OOD) detection. 
While common few-shot OOD detection involves training on images from all ID classes during training, OV-OOD detection involves training on images from just a small subset of ID classes and performing OOD detection using all ID classes at inference time. Formally, we define a subset of semantic labels ⊂, where  represents all ID labels. Based on this subset of labels, we define a corresponding subset dataset ⊂. During training, only  is used. Then, at inference time, all ID classes  are used, and the goal is to detect OOD from a combination of all ID test data  with  and OOD test data  with . For this setting, existing few-shot OOD detection methods <cit.> can be easily applied by simply combining the rest of the ID classes. In particular, NegPrompt <cit.> learns general negative prompts that are not specific to the training ID classes, so it achieves high performance in OV-OOD detection.




  §.§.§ Training-free Methods


Training-free few-shot OOD detection is a novel research field, and only Dual-Adapter <cit.> falls under this category. Dual-Adapter adopts a prior-based method Tip-Adapter <cit.>, which leverages both textual and
visual features with a cache model and enhances performance without training. To adapt this to few-shot OOD detection, Dual-Adapter employs the concept of dual cache modeling and constructs Positive-Adapter and Negative-Adapter, and identifies OOD samples with the prediction difference with both adapters. 






 §.§ Other Important Research Directions




  §.§.§ CLIP-based Full-spectrum OOD Detection

CLIP-based full-spectrum OOD (FS-OOD) detection is a crucial challenge <cit.>. FS-OOD detection was proposed by Yang  <cit.> in 2022 as an important setting that considers both OOD generalization <cit.> and OOD detection simultaneously. 
Unlike standard OOD detection, which only focuses on semantic shifts between training and test distributions, FS-OOD detection further considers non-semantic covariate shift by including covariate-shifted ID images. As for the benchmarks, OpenOOD v1.5 <cit.> provides two large-scale benchmarks based on ImageNet-200 and ImageNet-1K, incorporating ImageNet-C <cit.> with image corruptions, ImageNet-R <cit.> with style changes, and ImageNet-V2 <cit.> with resampling bias as ID. As for CLIP-based methods, LSA <cit.> uses a bidirectional prompt customization mechanism, which adjusts discriminative ID and OOD boundary. 




  §.§.§ Other Tasks with CLIP-based OOD Detection

Unsupervised Universal Fine-Tuning
CLIP-based OOD detection is useful for a new task called Unsupervised Universal Fine-Tuning (UUFT) <cit.>.
UUFT is a problem of unsupervised learning for outlier detection (OD). Existing studies for unsupervised learning assumed that all unlabeled images belong to one of the ID classes <cit.>, but they require prior knowledge of exact class names linked to ground truth labels, which restricts their usefulness in various real-world situations. For a more realistic setting, UUFT assumes that OOD images are included in the unlabeled images. To detect OOD images during training, they developed MCM <cit.> and proposed UEO, which leverages sample-level confidence to approximately minimize the conditional entropy of confident instances and maximize the marginal entropy of less confident instances.



Open-world Prompt Tuning
CLIP-based OOD detection is useful for a new task called Open-world Prompt Tuning <cit.>. 
Open-world Prompt Tuning is a task that evaluates the classification accuracy on a mix of known and novel ID classes while training the model with known classes. To solve this problem, Zhou  <cit.> proposed DeCoOp which incorporates OOD detection into the inference pipelines and improves the base-to-new separability, preventing performance degradation on new classes. 


§ CLIP-BASED AD: METHODOLOGY


In this section, we introduce methodologies for
CLIP-based anomaly detection (AD) in the hope that the contrast with OOD detection clarifies the similarities and differences between each task and facilitates a deeper understanding of CLIP-based OOD detection. 



 §.§ Zero-shot Anomaly Detection


CLIP-based zero-shot AD was proposed in 2023 by Jeong  <cit.>. Although it started about two years later than OOD detection, many methods have been proposed up to the present.



Definition of Zero-shot AD
The meaning of the term “Zero-shot" for zero-shot AD is similar to that for zero-shot OOD detection.
In zero-shot AD, the term “Zero-shot" refers to the non-use of the images in the target domain during both training and inference phases. For instance, the method with additional training with auxiliary datasets can be regarded as a zero-shot method <cit.>. The method with the pre-processing of the target class texts can also be regarded as a zero-shot method <cit.>.





  §.§.§ Training-free Methods


With Anomaly Prompts
In zero-shot AD, a common approach is to utilize anomaly prompts to detect anomalies. This hypothesis is supported by several observations from existing work <cit.>. Firstly, the concepts of normality and anomalies are context-dependent states <cit.> of an object, with language playing a crucial role in defining these states. Secondly, language provides additional insights that help differentiate defects from acceptable variations in normality. 

The simplest zero-shot AD methods are (i) to perform anomaly classification with CLIP using text prompts for normality and anomalies as classes (“” vs. “”) and (ii) to calculate the similarity to the normal prompt (“”) as the score. These methods are called CLIP-AC <cit.>. Jeong  <cit.> reported that CLIP-AC with both normal and anomaly prompts outperforms that with only normal text prompts, which indicates the importance of the use of anomaly prompts.
However, the performances for this naive method are not yet satisfactory due to the wide range of variations of anomalies.
To solve this issue, Jeong  <cit.> proposed WinCLIP.
WinCLIP performs a compositional ensemble on a large number of pre-defined normal and anomaly templates and efficient extraction and aggregation of window/patch/image-level features aligned with the text. WinCLIP outperforms CLIP-AC by a large margin. Because of its simplicity and pioneering work, WinCLIP has become an important baseline for CLIP-based AD.
AnoCLIP <cit.> follows WinCLIP's approach of using a large number of pre-defined normal and anomaly templates but modifies the templates to be domain-aware (industrial photo) and contrastive state for normal and anomaly (perfect and imperfect).
However, it is noteworthy that the performance of the ensemble strategies of previous methods heavily depends on the text descriptions <cit.>. 
Also, it is observed that more descriptions are not always better <cit.>, which makes the previous approaches <cit.> using a naive ensemble of large templates somewhat uncontrollable and random in their applications. Therefore, SDP <cit.> proposes RVS, a representative vector selection paradigm, which makes the mechanism of extracting representative vectors from large templates controllable, allowing for a more diverse selection of representative vectors.




  §.§.§ Auxiliary Training-based Methods


Unlike CLIP-based OOD detection, all auxiliary training-based methods in zero-shot AD are open-vocabulary AD methods that are trained with auxiliary AD datasets and tested on unseen target datasets by simply changing the category prompt. Existing methods perform supervised training on the test set of one dataset and perform zero-shot testing on the other dataset <cit.> (training with MVTec-AD and evaluation with VisA.)

In recent years, the development of auxiliary training-based methods has received more attention than training-free zero-shot methods. There are two main reasons why training is necessary for AD: (i) The first is the domain gap between semantics and anomalies. CLIP is pre-trained to understand the semantics of images, so when applied in a zero-shot manner, it captures the semantics of the image. However, actual anomalies are not semantics, but rather represent the state of an object and appear only in local areas of the image. Therefore, without training, this domain gap between semantics and anomalies cannot be bridged. (ii) The second reason is that there are limitations to relying on a large set of manually crafted anomaly prompts. This incurs prompt creation costs and also makes it difficult to respond to unknown anomalies. Therefore, by replacing the anomaly prompts with learnable parameters, they aim to solve the high costs and limited adaptability to new anomalies.

To address the above issue (i), APRIL-GAN (also known as VAND) <cit.> was proposed. APRIL-GAN tackles the domain gap between semantics and anomaly by adding additional linear layers in vision encoders. These linear layers project image features at each scale into the text space, creating and aggregating anomaly maps at each stage. 
Similarly, SDP+ <cit.> also incorporates additional linear layers into SDP <cit.> to effectively project image features into the text feature space, addressing the misalignment between image and text.
To solve both the issue (i) and (ii), AnomalyCLIP <cit.> was proposed. AnomalyCLIP is a prompt learning-based method similar to CoOp. By replacing anomaly prompts with learnable parameters, it eliminates the need to prepare a large number of manually pre-defined prompts such as those in WinCLIP <cit.>.
Furthermore, unlike CoOp which learns object semantics, AnomalyCLIP learns object-agnostic text prompts that capture generic normality and abnormality in an image regardless of its semantics. To achieve this, AnomalyCLIP introduces object-agnostic text prompt templates for both normal and anomaly and performs global and local context optimization. A more recent approach, FiLo <cit.> leverages Large Language Models (LLMs) to generate fine-grained anomaly descriptions for each object category. This method replaces generic abnormal descriptions with LLM-generated specific anomaly content for each sample. By adding learnable prompts before the generated anomaly prompts, FiLo performs global and local context optimization, enhancing the ability to detect anomalies. 
As a unique direction from these methods, RWDA <cit.> proposes a data augmentation approach by utilizing CLIP's text embeddings as training data. RWDA adds randomly generated words into normal and anomaly prompts to generate a diverse set of normal and anomaly training samples and trains a regular feed-forward neural network with diverse text embeddings. 





 §.§ Few-shot Anomaly Detection


CLIP-based few-shot AD was proposed in 2023 by Jeong  <cit.>, concurrently with the development of zero-shot AD <cit.>.
Traditional few-shot AD research focuses on modeling the normal distribution from a limited number of normal samples to detect anomalies <cit.>. However, these methods often struggle to generalize to new domains, as they typically require retraining with the target datasets. With the advent of CLIP, the field of few-shot AD is shifting towards using a few target images only for inference at test time without training.



Definition of Few-shot AD
CLIP-based few-shot AD aims to detect anomaly images using only a few images in the target domain. The meaning of the term “Few-shot" is similar to that of few-shot OOD detection.
In few-shot AD, the term “Few-shot" refers to the use of a few normal images in the target domain during training or inference phases. For instance, the method with additional training with a few normal images in the target domain can be regarded as a few-shot method <cit.>. Even without training, if a method uses a few normal images in the target domain as a reference, we regard it as a few-shot method <cit.>. 



  §.§.§ Training-free Methods


The earliest approach in CLIP-based few-shot AD is WinCLIP+ <cit.>, an improved method of WinCLIP. WinCLIP, a base zero-shot AD method, cannot identify certain defects that can only be defined visually rather than textually. For example, the “Metal-nut" category in MVTecAD has an anomaly type labeled “flipped upside down," which can only be identified relative to a normal image. To address this, WinCLIP+ incorporates a few normal reference images into a memory bank <cit.> and calculates the anomaly score with the cosine similarity between the query image and its most similar image in the memory bank.



  §.§.§ ID Training-based Methods


There are few training-based methods for CLIP-based few-shot AD research, although CLIP-based few-shot OOD detection actively explores the training-based methods. 
This is likely because detecting anomalies in unknown classes is highly valuable in practical applications, and training on target data may oversimplify the task of few-shot anomaly detection. This oversimplification concern arises because the anomaly space for known categories is much more limited compared to that for OOD detection. Training with the target data could therefore make the task oversimplified, reducing its difficulty. The only existing work in this area is PromptAD <cit.>, a prompt learning method for one-class AD (where the normal class consists of one class). In one-class AD, traditional prompt learning methods for multi-class classification (CoOp <cit.>) do not work well. To address this, PromptAD creates a large number of anomaly prompts by adding a learnable anomaly suffix to the normal prompt. It then learns to bring the visual features closer to the normal prompt and further away from the anomaly prompts, enabling prompt learning for one-class AD.




  §.§.§ Auxiliary Training- and Reference-based Methods


We explore the methods trained on auxiliary datasets and utilize the normal images in the target domain as references during inference.
An early work in this category is APRIL-GAN (few-shot) <cit.>, which uses a linear layer trained with auxiliary datasets. Similar to WinCLIP+ <cit.>, APRIL-GAN (few-shot) utilizes a few ID reference images with a memory bank-based approach <cit.>.
More recently, Zhu  <cit.> proposed an in-context-learning-based method called InCTRL. InCTRL trains a model to discriminate anomalies from normal samples by learning to identify residuals or discrepancies between query images and a set of few-shot normal images (in-context sample prompts) from auxiliary data. During inference, InCTRL identifies anomalies by measuring the discrepancy between the features of the query image and a few in-context normal samples from the target dataset.




 §.§ Other Research Direction



  §.§.§ Anomaly Detection with Localization Models

Some works <cit.> tackle AD using SAM <cit.> or DINO <cit.>, foundation models for localization. The representative work is SAA and SAA+ <cit.>. SAA is a simple baseline approach and utilizes Grounding-DINO <cit.> for the anomaly region generator and SAM <cit.> for the anomaly region refiner. SAA+ is an improved method of SAA and it incorporates domain expert knowledge and target image context into SAA. 
Given that AD necessitates localization, it is expected that the number of works employing localization foundation models such as SAM will continue to increase.



  §.§.§ Medical Anomaly Detection

While most works on CLIP-based AD focus on industrial AD, recent studies have begun to challenge medical anomaly detection (medical AD) <cit.>.
CLIP-based medical AD is a more challenging area than industrial AD due to the larger gap between different data modalities. A representative work on medical AD is MVFA <cit.>. MVFA is a method specifically tailored for medical AD. It incorporates multiple residual adapters into the CLIP's visual encoder to reduce the domain gap, enabling a stepwise enhancement of visual features across different levels. 
The future progression of medical AD and industrial AD offers an intriguing perspective, exploring whether these fields will develop independently or influence each other.
However, when considering practical applications, it should be noted that medical AD faces the challenge that anomalies are not always describable in the language. Therefore, the development of medical AD methods that do not use CLIP is also important.






 §.§ Discussion


We discuss the similarities and differences between CLIP-based OOD detection and CLIP-based AD to deepen our understanding of CLIP-based OOD detection.



  §.§.§ Difference between Each Methodology

Differing Scopes of OOD
OOD detection and AD differ significantly in the scope of OOD (anomaly) they cover, which leads to differences in methodologies, particularly in the use of OOD prompts. As explained in Sec. <ref>, sensory AD is intended for specific use cases like industrial inspection, where samples deviating from predefined normality (defective products) are considered anomalies <cit.>. In other words, in sensory AD, the anomaly space is limited to damaged objects with shared semantics, and anomalies like images of dogs are not expected. This limited anomaly space allows even simple prompts to achieve decent performance. Therefore, as shown in Table <ref>, all AD methods utilize anomaly prompts.
Conversely, in OOD detection, as explained in Sec. <ref>, anything semantically different from the ID class is considered OOD. Thus, utilizing naive manual OOD prompts is forbidden (even if it improves benchmark performance). This vastness of the OOD space is the key factor differentiating the methodologies between the two fields.



Essential Features Learned for Detecting OOD
There is a significant difference in the features that need to be captured between CLIP-based OOD detection and CLIP-based AD. In CLIP-based OOD detection, 
it is ideal to learn a more compact ID decision boundary that produces low uncertainty for the ID data, with high uncertainty for OOD data <cit.>.
On the other hand, CLIP-based AD aims to learn the anomalies, instead of ID decision boundary for detecting anomalies <cit.>. 
The learned features are entirely different, which is helpful in developing each training-based method.



Difficulty of Localization Task
CLIP-based OOD detection and CLIP-based AD differ significantly in the difficulty of OOD (anomaly) localization tasks. In CLIP-based AD, anomaly segmentation is a mainstream task, often performed alongside classification in many papers. However, in CLIP-based OOD detection, there has been no research on object-level OOD detection/segmentation. Object-level OOD detection aims to detect OOD objects <cit.>. The inactivity is related to the size of the OOD space, and the too-vast space of OOD makes it difficult to identify OOD objects with prompts effectively. To pave the way for future development, the foundation models for localization such as SAM <cit.>, which can segment individual objects, have the potential to address object-level OOD detection/segmentation. Object-level OOD detection/segmentation using SAM is a promising future research direction.





  §.§.§ Similarity between Each Methodology

Each Problem Setting
The existing problem settings for CLIP-based AD and CLIP-based OOD detection are similar. Both primarily focus on zero-shot and few-shot settings and can be categorized into training-free, auxiliary training-based, and ID training-based methods. By examining each problem setting more closely, we can observe, for instance, that while open-vocabulary AD is predominant in CLIP-based AD, it has not been deeply explored in CLIP-based OOD detection. This provides valuable insights into future directions that are crucial for OOD detection.



History of Approaches
The history of the progress of methods for CLIP-based AD and CLIP-based OOD detection is similar. For instance, both problems initially started with naive methods with manual OOD prompts (ZeroOE <cit.> for OOD detection, WinCLIP <cit.> for AD). To address the issues with these initial approaches, subsequent methods emerged that replaced OOD prompts with learnable parameters (LSN <cit.> and NegPrompt <cit.> for OOD detection, and AnomalyCLIP <cit.> for AD). Therefore, by carefully examining each other's fields, there is potential for mutual enhancement and interaction in the future.


§ EVOLUTION IN LVLM ERA


In this section, we introduce the early advances in OOD detection and AD in the Large Vision Language Models (LVLM) era. While previous sections focused on VLMs such as CLIP, this section shifts our focus to the more emerging topic of “Large" VLMs.
Recent advancements in computer vision have led to the emergence of LVLMs such as GPT-4V <cit.> and LLaVA <cit.>. 
Although these fields are still in their early stages with limited papers, this survey provides a deep introduction to each problem in the hope that our detailed review can help foster further advancements in this area.







 §.§ Change of Each Problem

i. Sensory AD → Sensory AD
Sensory AD has continued to develop in the LVLM era <cit.>. The use of LVLMs has made AD applicable in many domains and modalities <cit.>. 



ii. OOD Detection → Unsolvable Problem Detection
In the LVLM era, OOD detection has evolved into a new task termed Unsolvable Problem Detection (UPD) <cit.>. UPD evaluates the LVLMs' ability to recognize and abstain from answering unexpected or unsolvable input questions, effectively expanding the scope of OOD detection into the context of Visual Question Answering (VQA) tasks. This shift to the VQA task has significantly broadened the concept of OOD detection to a wider range of AI tasks involving LVLMs.




 §.§ Unsolvable Problem Detection



  §.§.§ Summary of Problem

Background
Following the recent revolutionary development of LLMs <cit.>, LVLMs <cit.> have demonstrated remarkable capabilities in diverse applications <cit.>. However, a significant concern has arisen regarding the reliability of these models, specifically their ability to generate accurate and trustworthy information. These models frequently produce incorrect or misleading information, a phenomenon referred to as “hallucination” <cit.>.
Among the various hallucination issues <cit.>, the challenge of identifying out-of-place questions is crucial for deploying LVLMs in safety-critical applications. This challenge extends the concept of OOD detection to the VQA tasks for LVLMs and represents a specific aspect of LVLMs' trustworthiness.



Definition
Unsolvable Problem Detection (UPD) is a task to measure the trustworthiness of LVLMs, which is designed to evaluate models' capacity to withhold answers when faced with unsolvable problems. The UPD task can be categorized into three distinct problem types: Absent Answer Detection (AAD), Incompatible Answer Set Detection (IASD), and Incompatible Visual Question Detection (IVQD).
The details of each setting are as follows:

      
  * Absent Answer Detection (AAD): AAD evaluates the model's capacity to determine when the correct answer is absent from the provided options.
      
  * Incompatible Answer Set Detection (IASD): IASD assesses the model's ability to discern answer choices that are completely irrelevant to the given question and image.
      
  * Incompatible Visual Question Detection (IVQD): IVQD evaluates the model's capacity to discern whether a question and image are unrelated or mismatched.




Benchmark
Miyai  <cit.> created MM-UPD Bench for the UPD challenge. MM-UPD encompasses MM-AAD, MM-IASD, and MM-IVQD benchmarks for each UPD problem. Each benchmarks are created on the top of MMBench (dev) <cit.>, which is a systematically designed objective benchmark for evaluating various abilities of LVLMs. Following the definition of each ability in MMBench (“Coarse Perception: Image Scene” and “Logic Reasoning: Future Prediction”), MM-UPD evaluates the trustworthiness of LVLMs from various abilities.

Although MM-UPD is the main benchmark, the adaptation cost of creating UPD problems is not high, making it highly applicable to other benchmarks. For instance, the recently proposed MuirBench <cit.>, a benchmark for multi-image understanding, has incorporated the concept of UPD by adding unsolvable problems. 



Application
UPD has a wide range of applications, from everyday use of LVLMs to robot manipulation. Especially when incorporating LVLMs into safety-critical domains such as robot manipulation <cit.> and autonomous driving <cit.>, there is a risk of significant problems if the LVLM fails to identify erroneous user questions and makes incorrect predictions. UPD serves as a task to ensure safety in such safety-critical scenarios.




Evaluation
UPD introduces new evaluation metrics that incorporate the concept of the evaluation protocols for OOD detection, taking into account the prediction distribution for both standard (ID) and UPD (OOD) samples. 
The rationale is that ideal LVLMs should not only give correct answers for the standard problems but need to withhold answering in the UPD scenario where the problem is unsolvable. 
To better reflect the ideal behavior of LVLMs, UPD measures several metrics:
(i) Standard Accuracy: 
The accuracy on standard problems where the image, question, and answer sets are all aligned, and the ground-truth answer is always present within the provided options. (ii) UPD Accuracy: The accuracy of AAD/IASD/IVQD problems. (iii) Dual Accuracy: Accuracy of standard and UPD pairs. We count success only if the model is correct for both standard and UPD problems.




  §.§.§ Findings

In the following, we briefly summarized the findings of the UPD challenge <cit.>.



1. Most LVLMs Hardly Hesitate to Answer.
Most LVLMs, especially open-source LVLMs, have significantly low UPD accuracies, which indicates the difficulty of the UPD challenge. For example, LLaVA-1.5 <cit.> and CogVLM <cit.>, which are state-of-the-art LVLMs, completely fail to withhold answering. GPT-4V achieves higher performances than other LVLMs due to its safety training process <cit.>. However, there is still a performance gap from the upper bound scores.



2. Performance Tendency Differs a lot by Each Ability in the Benchmark.
The performance of LVLMs differs in each ability in the MM-UPD Bench. For instance, GPT-4V has its limitation in attribute comparison and LLaVA-NeXT-34B has its limitation in object localization.



3. Effective Prompt Strategies Vary Across Different LVLMs 
Effective prompt strategies vary across different LVLMs. In the original paper, they experimented with an option-based prompt approach that adds an option of “None of the above" and an instruction-based approach that adds an instruction “If all options are incorrect, answer None of the above". As a result, the effectiveness of each approach differs significantly depending on the type of LVLMs. This highlights the difficulty of finding an effective prompt strategy for all LVLMs. 




 §.§ Anomaly Detection in LVLM Era



  §.§.§ Summary of Problem

Background
Anomaly detection is a crucial task in a variety of domains and data types. However, existing anomaly detection models are often designed for specific domains or modalities <cit.>. Also, current AD methods only provide an anomaly score for the test sample and require a manual threshold to distinguish between normal and anomalous instances for each sample <cit.>. To facilitate real-world applications, developing a system capable of expressing anomalies in natural language across various modalities and domains is crucial for ensuring accessibility to a wider range of users.



Definition
The definition of AD remains consistent with conventional and CLIP-based AD, aiming to identify samples that deviate from predefined normality. The key distinction lies in the output. While previous methods produced an anomaly score requiring a manual threshold, AD with LVLMs aims to recognize and describe anomalies using text, eliminating the need for manual thresholds and enhancing human interpretability.



Benchmark
Since the field of AD with LVLMs is in its infant stage, there are still no unified benchmarks. AnomalyGPT <cit.> focuses on industrial image anomaly detection/localization and uses the standard benchmarks MVTec-AD <cit.> and VisA <cit.>. 
More recently, Cao  <cit.> extend the domain and modality and demonstrate the applications in industrial image anomaly detection/localization (MVTec-AD <cit.>), point cloud anomaly detection (MVTec 3D <cit.>), medical image anomaly detection/localization (Chest X-ray <cit.>, Head CT <cit.>), logical anomaly detection (MVTec LOCO <cit.>), pedestrian anomaly detection (UCF-Crime Dataset <cit.>), traffic anomaly detection (Kaggle Accident Detection <cit.>), and time series anomaly detection (Outlier Detection Dataset <cit.>).



Evaluation
Evaluation in anomaly detection with LVLMs is an open challenge.  AnomalyGPT <cit.> asks LVLMs the question “Is there an anomaly in this image?" and determines anomaly or normal based on the simple rule-based approach of whether the response contains a “yes" or “no". However, this rule-based approach is not robust, as a response is considered correct even if the explanation following “yes" is completely incorrect. On the other hand, Cao  <cit.> conducted only qualitative evaluations and left quantitative evaluations as an open challenge. Therefore, the evaluation of anomaly detection by LVLM is a future challenge.



  §.§.§ Findings

Cao  <cit.> described the observations of GPT-4V in the paper, so we briefly summarized them here.



1. GPT-4V Excels in Zero/One-shot Settings across Various Modalities and Fields.
GPT-4V shows proficiency in identifying anomalies in multi-modality (images, point clouds, X-rays) and multi-field (industrial, medical, pedestrian, traffic, and time series anomaly detection). In addition, GPT-4V demonstrates strong performance in both zero-shot and one-shot settings. 



2. GPT-4V can Understand Both Global and Fine-grained Anomalies.
GPT-4V can recognize both global and local abnormal patterns or behaviors, which indicates the ability to understand global and fine-grained semantics.



3. GPT-4V can be Enhanced with Increasing Prompts.
By giving more context and information, the model significantly improves its ability to detect anomalies accurately. 



§ POTENTIAL CHALLENGES


This section discusses potential challenges for CLIP-based OOD detection that may be highlighted by the widespread adoption of our framework. Since similar or ambiguous problem settings currently exist, we present these challenges to avoid future confusion for readers.





 §.§ Open-vocabulary OOD Detection
 

The existing setting for open-vocabulary OOD (OV-OOD) detection <cit.> involves training with images from a small subset of ID classes and evaluating with all ID class names, as explained in Sec. <ref>. However, this problem definition has several issues: (i) The OOD detection performance on ID classes not included in the training subset is unclear, (ii) This contradicts the settings of the existing open-vocabulary settings, including CLIP-based AD (Sec. <ref>), which assume no explicit class overlap between training and test ID data <cit.>, leading to confusion in the fields.
Therefore, for clear evaluation and common understanding in the field, we redefine the OV-OOD detection as in-class open-vocabulary OOD detection (in-class OV-OOD detection) and zero-shot open-vocabulary OOD detection (zero-shot OV-OOD detection). The illustration for each setting is shown in Fig. <ref>. Detailed explanations for each setting are as follows:



In-class OV-OOD Detection
In-class OV-OOD detection is the same as the original setting proposed by  <cit.>, involving training with images from a small subset of ID classes and evaluating on all ID class names including training classes.
In-class OV-OOD detection should be evaluated in the few-shot OOD detection setting to assess how well these methods retain their performance when trained with limited class data. In this setting, it is preferable to employ the existing benchmark setup that randomly selects subset ID classes from all ID classes <cit.>.



Zero-shot OV-OOD Detection
Zero-shot OV-OOD detection shares the training setting with in-class evaluation, using a small subset of ID classes. However, it differs in the testing phase, where it exclusively utilizes classes not included in the training subset. 
This problem setting is consistent with the common open-vocabulary setting <cit.> and the open-vocabulary setting in CLIP-AD <cit.>.
Zero-shot OV-OOD detection can be positioned within the area of zero-shot OOD detection, which would facilitate the research of zero-shot OOD detection. When selecting the subset of classes for training, we should ensure semantic dissimilarity between the classes in the evaluation and training sets to guarantee the validity of the zero-shot problem setting. For instance, when splitting ImageNet classes into training and evaluation subsets for zero-shot OV-OOD detection, it would be preferable to consider the hierarchical classes of ImageNet to avoid similar semantics between training and evaluation subsets.




 §.§ CLIP-based Full-spectrum OOD Detection






CLIP-based full-spectrum OOD detection is a well-established task with existing research <cit.>. However, with the future prevalence of hard OOD detection, which incorporates the OSR setup, the distinction between this task and open-set domain generalization (OSDG)  <cit.> may become ambiguous. OSDG shares a similar motivation of realizing models capable of both generalization and detection. The illustration of both problem settings is shown in Fig. <ref> (a)(b). OSDG assumes that only data with covariate shifts will be input during testing, and aims to classify input samples into one of ID classes if their semantics align with ID classes and detect OOD that exhibit semantic shifts.
OSDG is a research area with a much smaller number of works compared to the main topics of this survey (OD, AD, ND, OSR, and OOD detection), but, in recent years, some works have tackled OSDG using CLIP <cit.>.

To eliminate potential ambiguity, this paper defines a hard full-spectrum OOD (FS-OOD) detection, following our generalized OOD detection v2. The illustration of the problem setting for hard FS-OOD detection is shown in Fig. <ref> (c).
Hard FS-OOD detection expands the scope of existing FS-OOD detection by utilizing the conventional OSR setup. 
It introduces OOD samples from different categories within the training domain and OOD samples that share the same covariate shift as the target domain but exhibit semantic shifts. This task can be regarded as an extensive version of both existing FS-OOD detection and OSDG tasks. Hard FS-OOD detection represents a developing challenge with promising potential for future research.




§ FUTURE DIRECTIONS


In this section, we discuss the future directions of OOD detection and UPD.
For OOD detection, we explore not only OOD detection for VLMs but also single-modal OOD detection, with a specific focus on emerging challenges as VLMs evolve. 
For a discussion of the long-standing challenges in OOD detection, we can refer the readers to the previous generalized OOD detection paper <cit.>.



 §.§ OOD Detectin for Vision Language Models

a. Hard OOD Detection
Hard OOD detection will become increasingly important in the future due to its high practicality and the challenging nature of the problem. Hard OOD detection utilizes the OSR benchmark setup, where some classes within a single dataset are designated as ID and others as OOD. In this field, not only small datasets such as ImageNet-10 and ImageNet-20 <cit.> but also datasets with a larger number of classes such as ImageNet-protocol <cit.> have been proposed. Many existing studies, such as LoCoOp <cit.> and LSN <cit.>, primarily use the common ImageNet OOD benchmark, so hard OOD detection has not yet been well studied. This field will develop further in the future.



b. Post-hoc Methods
To propose post-hoc methods is important for the fundamental performance improvement of CLIP-based OOD detection. The methods of directly employing an ID classifier such as MCM <cit.> are called post-hoc methods. Prior to CLIP, various approaches were proposed  <cit.>.
However, CLIP-based post-hoc methods often underperform methods with additional OOD prompts, so they are not extensively researched in zero-shot OOD detection. However, we should focus on the scalability of post-hoc methods. The post-hoc methods <cit.> can be easily applied to many subsequent methods <cit.>, bringing fundamental performance improvements. Furthermore, very recently, post-hoc methods specifically tailored for prompt learning methods have also emerged <cit.>.
Therefore, proposing post-hoc methods and demonstrating the improvements not only in zero-shot but also in subsequent few-shot settings <cit.> is crucial, even if they underperform methods using OOD prompts in the zero-shot setting. This field should continue evolving, mirroring its growth before the advent of CLIP.



c. Bridging the Gap with Closed-set Classifiers.
OOD detection ensures the safety of ID classifiers, so it is crucial to bridge the gap between the advancements in existing closed-set classifiers and OOD detection. 
Currently, the representative method for few-shot OOD detection is LoCoOp <cit.>, a text prompt learning method based on CoOp. However, in closed-set settings, few-shot learning methods based on text prompt learning have been proposed other than CoOp <cit.>. Furthermore, text-based prompt learning methods only train the text prompts, so they cannot handle differences in image domains. Therefore, adopting methods that can handle image domain differences <cit.> for OOD detection is essential for bridging the gap with closed-set ID classifiers.



d. Training-free Few-shot OOD Detection
The research direction of training-free few-shot OOD detection is still in its infant stage, with only one existing study <cit.>. Given the prevalence of training-based methods in few-shot OOD detection, proposing methods that do not require training is crucial. Considering the advancements of training-free methods in CLIP-based anomaly detection, we anticipate a similar trajectory for CLIP-based OOD detection.
Future directions include refining adapter-based methods or leveraging external knowledge such as retrieval augmentation <cit.>.
Addressing training-free few-shot OOD detection is a pivotal step towards realizing more computationally efficient OOD detection in the future.



e. Full-spectrum OOD Detection
CLIP-based full-spectrum OOD (FS-OOD) detection is a promising research area <cit.>. In practical applications, there is a strong motivation to create models that can not only detect semantically shifted OOD inputs but also generalize to covariate-shifted data <cit.>. Within CLIP-based methods, OOD detection and generalization are often discussed in separate contexts <cit.>, resulting in a trade-off between detection and generalization performance <cit.>. Furthermore, there exists potential ambiguity in the distinction between FS-OOD detection and open-set domain generalization.
To eliminate the potential ambiguity, we have formulated a potential problem setting called hard FS-OOD detection.
We hope this survey inspires further advancements and developments for FS-OOD detection.



f. Open-vocabulary OOD Detection
Open-vocabulary OOD (OV-OOD) detection has a high practical potential, but it is still in its infant stages <cit.>.
In particular, as explained in Sec. <ref>, zero-shot OV-OOD detection is a potential research field. 
We hope that this survey paper will inspire future efforts in OV-OOD detection.



g. Object-level OOD Detection
Object-level OOD detection remains an unexplored area. As discussed in Sec. <ref>, this is due to the vastness of the OOD space, which makes it difficult to identify OOD objects using texts. 
To pave the way for future advancements in object-level OOD detection/segmentation, foundation models for localization, such as SAM <cit.>, offer a promising solution. By integrating these models with methods such as MCM <cit.>, we can potentially achieve object-level OOD detection and segmentation, opening up a new frontier in OOD detection research.




 §.§ Single-modal OOD Detection

a. Leveraging Large Pre-trained Models
Leveraging large pre-trained models is important for single-modal OOD detection. Numerous methods for OOD detection conduct experiments using backbones trained from scratch and do not utilize pre-trained models <cit.>. 
In a recent study, Miyai  <cit.> systematically investigated the impact of pre-training on OOD detection from both the perspectives of the types of OOD data and pre-training algorithms <cit.>.
Dong  <cit.> explored parameter-efficient learning for single-modal OOD detection and proposed DSGF, which leverages both fine-tuned features and original pre-trained features. While leveraging large pre-trained models <cit.> with lightweight tuning is an active area of research in single-modal closed-set classification <cit.>, there have been limited studies for single-modal OOD detection, which presents a promising avenue for future research.



b. Real-world Benchmarks and Evaluations
Considering the future development of CLIP-based OOD detection, there should be increasing focus on expanding the scope of benchmarks to encompass real-world scenarios where CLIP is less applicable.
For instance, recently, Baek  <cit.> introduced ImageNet-ES, consisting of variations in environmental and camera sensor factors by directly capturing 202k images with an actual camera in a controlled testbed, which bridges the gap between the common benchmark and the real-world scenario. Besides, utilizing datasets such as WILDS <cit.>, which consider real-world data shifts, or datasets for medical OOD detection <cit.>, can provide valuable insights, especially in safety-critical applications like autonomous driving and medical image analysis.




 §.§ Unsolvable Problem Detection

a. Exploring Effective Solutions
It is important to propose effective solutions for UPD. One of the potential approaches is to adapt the methodologies of OOD detection to UPD. For example, the perplexity of the LVLM's response could be used as a score to identify unsolvable queries. 
Furthermore, proposing model-agnostic post-hoc methods is also important to enhance the reliability of many LVLMs.
Therefore, incorporating the concepts of OOD detection techniques for UPD is an important direction for future work.



b. Extension to Diverse Benchmarks
MM-UPD Bench consists of general QA datasets. However, UPD can be incorporated into more diverse benchmarks including domain-specific knowledge for advanced reasoning <cit.> and multi-image understanding <cit.>. For example, MuirBench <cit.> incorporated the concept of UPD as a metric for robust evaluation. 
Integrating the concept of UPD into benchmarks is essential for evaluating the robustness and trustworthiness of LVLMs in their target tasks.



c. Theoretical Understanding of UPD
Theoretically understanding the reasons behind the difficulty of UPD could provide the community with valuable insights. Theorizing the behavior of LVLMs poses a shared challenge in the field, highlighting the importance of collaborative efforts within the community.



§ CONCLUSION


In this survey, we comprehensively review the evolution of the five problems including AD, ND, OSR, OOD detection, and OD in the VLM era, and propose a framework of generalized OOD detection v2. Our framework identifies OOD detection and AD as the primary challenges in the VLM era, which highlights the demanding challenges and fosters collaborative efforts among each community. 
By articulating the shifts in the definitions, problem settings, and benchmarks, we encourage subsequent works to accurately understand their evolving target problems in the VLM era.
By sorting out the methodologies, we hope that readers can easily grasp the mainstream methods, identify important baselines and novel problem settings, and propose future solutions.
By shedding light on recent studies in the LVLM era, we hope that researchers within each community can identify promising research directions in this emerging era.
By providing future directions, we hope that our survey will clarify the tasks to be tackled by future works in the VLM and LVLM era, thereby facilitating future advances in the right direction.


§ ACKNOWLEDGMENT

This work was supported by JST BOOST, Japan Grant Number JPMJBS2418 and JST JPMJCR22U4. We thank Toyooka Mashiro (AYM Lab at UTokyo) for his valuable assistance in designing the figures, and Kazuki Egashira, Yuki Imajuku, Takubon Son, and Zaiying Zhao (AYM Lab at UTokyo) for valuable feedback on the paper. 

ieeetr






[
    < g r a p h i c s >
]Atsuyuki Miyai
(Student Member, IEEE)
received his B.E. degree in Information and Communication Engineering in 2022 and his M.E. degree in Interdisciplinary Studies in Information Science in 2024 from The University of Tokyo. He is currently a Ph.D. student with the Department of Information and Communication Engineering at The University of Tokyo. His research topic is the safety of vision language models and foundation models. He is an IEEE Student Branch Chair at The University of Tokyo.




[
    < g r a p h i c s >
]
    Jingkang Yang is currently a PhD student at the College of Computing and Data Science (CCDS) at Nanyang Technological University (NTU), Singapore, working in the MMLab@NTU under the supervision of Dr. Ziwei Liu. His research interests include visual generalist models and AI safety for foundation models. He has published over 20 papers in relevant fields at top-tier conferences and journals, including CVPR, ICCV, ECCV, NeurIPS, ICLR, and IJCV. He serves as a reviewer of CVPR, ICCV, ECCV, NeurIPS, ICLR, etc., and is an outstanding reviewer in ICCV 2021 and CVPR 2024.




[
    < g r a p h i c s >
]Jingyang Zhang received his B.E. degree in Electronic Engineering from Tsinghua University, Beijing, China, in 2019.
Since then, he has been pursuing the Ph.D. degree at Dept. of Electrical and Computer Engineering at Duke University, supervised by Dr. Yiran Chen and Dr. Hai (Helen) Li.
His research spans from robustness of deep learning-based vision systems to (more recently) generative AI and multi-modal LLMs.




[
    < g r a p h i c s >
]Yifei Ming received his Ph.D. in Computer Science from the University of Wisconsin-Madison in 2024, advised by Dr. Yixuan Li. He is currently a Research Scientist at Salesforce AI Research. His research interests encompass reliable machine learning, vision-language models, large language models, and responsible foundation models. He has a prolific publication record and serves on the program committees of leading machine learning conferences and journals, including NeurIPS, ICLR, ICML, EMNLP, and IJCV.




[
    < g r a p h i c s >
]Yueqian Lin
received a B.S. degree in Data Science from Duke Kunshan University in 2024, graduating summa cum laude with signature work distinction. He is currently pursuing a Ph.D. in Electrical and Computer Engineering at Duke University under the supervision of Dr. Yiran Chen and Dr. Hai (Helen) Li. His research focuses on robustness in generative models.




[
    < g r a p h i c s >
]Qing Yu
(Member, IEEE) 
received his B.E. degree in Information and Communication Engineering and the M.E. degree in Interdisciplinary Studies in Information Science from The University of Tokyo in 2018 and 2020, respectively. He completed his Ph.D. in Information Science and Technology from The University of Tokyo in 2023. He is currently a Research Scientist at LY Corporation, Japan. His research interests include multimodal recognition and generation.




[
    < g r a p h i c s >
]Go Irie
(Member, IEEE) received his B.E. and M.E. in System Engineering from Keio University, Japan, in 2004 and 2006, respectively. He completed his Ph.D. in Information Science and Technology from The University of Tokyo in 2011. 
He is currently an Associate Professor at the Department of Information and Computer Technology, Tokyo University of Science, Japan. He was a research scientist at NTT Corporation, Japan, from 2006 to 2022, and a Visiting Research Scholar at Columbia University from 2012 to 2013. His research interests include pattern recognition, machine learning and media understanding.




[
    < g r a p h i c s >
]Shafiq Joty is currently a Research Director at Salesforce Research (Palo Alto, USA), where he oversees the NLP group's efforts in large language modeling (LLM) and generative AI. He also holds the position of a tenured Associate Professor (currently on leave) in the School of Computer Science and Engineering (SCSE) at NTU, Singapore. He was a founding manager of the Salesforce Research Asia (Singapore) lab. His research has contributed to over 30+ patents and 145+ papers in top-tier NLP and ML conferences and journals. He has served as the Program Chair of SIGDIAL-2023, as a member of the best paper award committees for ICLR-23 and NAACL-22, and in the capacity of a (senior) area chair for many of the leading NLP and ML conferences.





[
    < g r a p h i c s >
]Yixuan Li (Member, IEEE) received the PhD degree from Cornell University, in 2017, advised by John E. Hopcroft. She is an assistant professor in the Department of Computer Sciences, the University of Wisconsin-Madison. Subsequently, she was a postdoctoral scholar in the computer science department, Stanford University. Her research focuses on the algorithmic and theoretical foundations of learning in open worlds. She has served as Area Chair for ICLR, NeurIPS, ICML, and Program Chair for Workshop on Uncertainty and Robustness in Deep Learning. She is the recipient of the AFOSR Young Investigator Program (YIP) award, NSF CAREER award, MIT Technology Review Innovator Under 35, Forbes 30 Under 30 in Science, and multiple faculty research awards from Google, Meta, and Amazon. Her works received a NeurIPS Outstanding Paper Award, and an ICLR Outstanding Paper Award Honorable Mention, in 2022.





[
    < g r a p h i c s >
]Hai (Helen) Li
(Fellow, IEEE) received the Ph.D. degree from Purdue University in 2004. 
Dr. Li is currently the Clare Boothe Luce Professor and Department Chair of the Electrical and Computer Engineering Department at Duke University.
Her current research interests include neuromorphic circuits and systems for brain-inspired computing, machine learning acceleration and trustworthy AI, conventional and emerging memory design and architecture, and software and hardware co-design.
Dr. Li is a recipient of the NSF Career Award (2012), DARPA Young Faculty Award (2013), TUM-IAS Hans Fischer Fellowship from Germany (2017), and ELATE Fellowship (2020). She received 9 best paper awards and additional 9 best paper nominations from international conferences. Dr. Li is a Distinguished Lecturer of the IEEE CAS Society (2018-2019) and a Distinguished Speaker of ACM (2017-2020). 





[
    < g r a p h i c s >
]Ziwei Liu
(Member, IEEE) is currently a nanyang assistant professor with Nanyang Technological University, Singapore. His research interests include computer vision machine learning and computer graphics. He has published extensively on top-tier conferences and journals in relevant fields, including CVPR, ICCV, ECCV, NeurlPS, ICLR, ICML, IEEE Transactions on Pattern Analysis and Machine Intelligence, ACM Transactions on Graphics and Nature - Machine Intelligence. He is the recipient of ICCV Young Researcher Award, HKSTP Best Paper Award, CVPR Best Paper Award Candidate, ICBS Frontiers of Science Award and MIT Technology Review Innovators under 35 Asia Pacific. He serves as an area chair of CVPR, ICCV, ECCV, NeurlPS and ICLR, as well as an associate editor of International Journal of Computer Vision.





[
    < g r a p h i c s >
]Toshihiko Yamasaki
(Member, IEEE) received the Ph.D. degree from The University of Tokyo. He is currently a Professor at the Department of Information and Communication Engineering, Graduate School of Information Science and Technology, The University of Tokyo. He was a JSPS Fellow for Research Abroad and a visiting scientist at Cornell University from 2011 to 2013. His current research interests are computer vision, multimedia, pattern recognition, machine learning. He is a pioneer of attractiveness computing.





[
    < g r a p h i c s >
]Kiyoharu Aizawa
(Fellow, IEEE) received the B.E., the M.E., and the Dr.Eng. degrees in Electrical Engineering all from the University of Tokyo, in 1983, 1985, 1988, respectively. He is a professor with the Department of Information and Communication Engineering, and director of VR center, University of Tokyo. He was a visiting assistant professor with the University of Illinois from 1990 to 1992. His research fields are multimedia, image processing, and computer vision, with a particular interest in interdisciplinary and cross-disciplinary issues. He received the 1990, 1998 Best Paper Awards, the 1991 Achievement Award, 1999 Electronics Society Award from IEICE Japan, and the 1998 Fujio Frontier Award, the 2002 and 2009 Best Paper Award, and 2013 Achievement award from ITE Japan. He received the IBM Japan Science Prize in 2002. He is on Editorial Board of ACM TOMM. He served as the Editor-in-Chief of Journal of ITE Japan, an Associate Editor of IEEE TIP, TCSVT, TMM, and MultiMedia. He has served a number of international and domestic conferences; he was a General co-Chair of ACM Multimedia 2012 and ICMR2018. He is a Fellow of IEEE, IEICE, ITE and a member of Science Council of Japan. 






 Paper Title: OmniParser for Pure Vision Based GUI Agent.



Extracting doubly-excited state lifetimes in helium directly in the time domain with attosecond noncollinear four-wave-mixing spectroscopy
    Stephen R. Leone
    August 5, 2024
==========================================================================================================================================




§ ABSTRACT 
   The recent success of large vision language models shows great potential in driving the agent system operating on user interfaces. However, we argue that the power multimodal models like GPT-4V as a general agent on multiple operating systems across different applications is largely underestimated due to the lack of a robust screen parsing technique capable of: 1) reliably identifying interactable icons within the user interface, and 2) understanding the semantics of various elements in a screenshot and accurately associate the intended action with the corresponding region on the screen. To fill these gaps, we introduce OmniParser, a comprehensive method for parsing user interface screenshots into structured elements, which significantly enhances the ability of GPT-4V to generate actions that can be accurately grounded in the corresponding regions of the interface. 
   We first curated an interactable icon detection dataset using popular webpages and an icon description dataset. These datasets were utilized to fine-tune specialized models: a detection model to parse interactable regions on the screen and a caption model to extract the functional semantics of the detected elements.
   
   OmniParser significantly improves GPT-4V's performance on ScreenSpot benchmark. And on Mind2Web and AITW benchmark, OmniParser with screenshot only input outperforms the GPT-4V baselines requiring additional information outside of screenshot. 




§ INTRODUCTION

Large language models have shown great success in their understanding and reasoning capabilities. More recent works have explored the use of large vision-language models (VLMs) as agents to perform complex tasks on the user interface (UI) with the aim of completing tedious tasks to replace human efforts <cit.>. Despite the promising results, there remains a significant gap between current state-of-the-arts and creating widely usable agents that can work across multiple platforms, e.g. Windows/MacOS, IOS/Android and multiple applications (Web broswer Office365, PhotoShop, Adobe), with most previous work focusing on limiting applications or platforms.

While large multimodal models like GPT-4V and other models trained on UI data <cit.> have demonstrated abilities to understand basic elements of the UI screenshot, action grounding remains one of the key challenges in converting the actions predicted by LLMs to the actual actions on screen in terms of keyboard/mouse movement or API call <cit.>. It has been noted that GPT-4V is unable to produce the exact x-y coordinate of the button location, Set-of-Mark prompting <cit.> proposes to overlay a group of bounding boxes each with unique numeric IDs on to the original image, as a visual prompt sent to the GPT-4V model. By applying set-of-marks prompting, GPT-4V is able to ground the action into a specific bounding box which has ground truth location instead of a specific xy coordinate value, which greatly improves the robustness of the action grounding <cit.>. However, the current solutions using SoM relies on parsed HTML information to extract bounding boxes for actionable elements such as buttons, which limits its usage to web browsing tasks. We aim to build a general approach that works on a variety of platforms and applications. 

In this work, we argue that previous pure vision-based screen parsing techniques are not satisfactory, which lead to significant underestimation of GPT-4V model's understanding capabilities. And a reliable vision-based screen parsing method that works well on general user interface is a key to improve the robustness of the agentic workflow on various operating systems and applications. We present OmniParser, a general screen parsing tool to extract information from UI screenshot into structured bounding box and labels which enhances GPT-4V's performance in action prediction in a variety of user tasks. 


We list our  contributions as follows: 

    
  * We curate a interactable region detection dataset using bounding boxes extracted from DOM tree of popular webpages. 
    
  * We propose OmniParser, a pure vision-based user interface screen parsing method that combines multiple finetuned models for better screen understanding and easier grounded action generation.
    
  * We evaluate our approach on ScreenSpot, Mind2Web and AITW benchmark, and demonstrated a significant improvement from the original GPT-4V baseline without requiring additional input other than screenshot. 
    




§ RELATED WORKS



 §.§ UI Screen Understanding


There has been a line of modeling works focusing on detailed understanding of UI screens, such as Screen2Words <cit.>, UI-BERT <cit.>, WidgetCaptioning <cit.>, Action-BERT <cit.>. These works demonstrated effective usage of multimodal models for extracting semantics of user screen. But these models rely on additional information such as view hierarchy, or are trained for visual question answering tasks or screen summary tasks. 


There are also a couple publicly available dataset that on UI screen understanding. Most notably the Rico dataset <cit.>, which contains more than 66k unique UI screens and its view hierarchies. Later <cit.>  auguments Rico by providing 500k human annotations on the original 66k RICO screens identifying various icons based on their shapes and semantics, and associations between selected general UI elements (like icons, form fields, radio buttons, text inputs) and their text labels. Same on mobile platform, PixelHelp <cit.> provides a dataset that contains UI elements of screen spanning across 88 common tasks. In the same paper they also released RicoSCA which is a cleaned version of Rico. For the web and general OS domain, there are several works such Mind2Web <cit.>, MiniWob++<cit.>, Visual-WebArena <cit.>, and OS-World <cit.> that provide simulated environment, but does not provide dataset explicitly for general screen understanding tasks such as interactable icon detection on real world websites. 

To address the absence of a large-scale, general web UI understanding dataset, and to keep pace with the rapid evolution of UI design, we curated an icon detection dataset using the DOM information from popular URLs avaialbe on the Web. This dataset features the up-to-date design of icons and buttons, with their bounding boxes retrieved from the DOM tree, providing ground truth locations. 



 §.§ Autonomous GUI Agent


Recently there has been a lot of works on designing autonomous GUI agent to perform tasks in place of human users. One line of work is to train an end-to-end model to directly predict the next action, representative works include: Pixel2Act <cit.>, WebGUM<cit.> in web domain, Ferret <cit.>, CogAgent <cit.>, and Fuyu <cit.> in Mobile domain. Another line of works involve leveraging existing multimodal models such as GPT-4V to perform user tasks. Representative works include MindAct agent <cit.>, SeeAct agent <cit.> in web domain and agents in <cit.> for mobile domain. 
These work often leverages the DOM information in web browser, or the view hierarchies in mobile apps to get the ground truth position of interactable elements of the screen, and use Set-Of-Marks<cit.> to overlay the bounding boxes on top of the screenshot then feed into the vision-language models. However, ground truth information of interactable elements may not always be available when the goal is to build a general agent for cross-platforms and cross-applications tasks. Therefore, we focus on providing a systematic approach for getting structured elements from general user screens. 






§ METHODS

A complex task can usually be broken down into several steps of actions. Each step requires the model's (e.g. GPT-4V) ability to: 1) understand the UI screen in the current step, i.e. analyzing what is the screen content overall, what are the functions of detected icons that are labeled with numeric ID, and 2) predict what is the next action on the current screen that is likely to help complete the whole task. Instead of trying to accomplish the two goals in one call, we found it beneficial to extract some of the information such as semantics in the screen parsing stage, to alleviate the burden of GPT-4V so that it can leverages more information from the parsed screen and focus more on the action prediction. 


Hence we propose OmniParser, which integrates the outputs from a finetuned interactable icon detection model, a finetuned icon description model, and an OCR module. This combination produces a structured, DOM-like representation of the UI and a screenshot overlaid with bounding boxes for potential interactable elements. We discuss each component of the OmniParser in more details for the rest of the section. 






 §.§ Interactable Region Detection


Identifying interactable regions from the UI screen is a crucial step to reason about what actions should be performed given a user tasks. Instead of directly prompting GPT-4V to predict the xy coordinate value of the screen that it should operate on, we follow previous works to use the Set-of-Marks approach <cit.> to overlay bounding boxes of interactable icons on top of UI screenshot, and ask GPT-4V to generate the bounding box ID to perform action on. However, different from <cit.> which uses the ground truth button location retrieved from DOM tree in web browswer, and <cit.> which uses labeled bounding boxes in the AITW dataset <cit.>, we finetune a detection model to extract interactable icons/buttons. 

Specifically, we curate a dataset of interactable icon detection dataset, containing 67k unique screenshot images, each labeled with bounding boxes of interactable icons derived from DOM tree. We first took a 100k uniform sample of popular publicly availabe urls on the web <cit.>, and collect bounding boxes of interactable regions of the webpage from the DOM tree of each urls. Some examples of the webpage and the interactable regions are shown in <ref>.  

Apart from interactable region detection, we also have a OCR  module to extract bounding boxes of texts. Then we merge the bounding boxes from OCR detection module and icon detection module while removing the boxes that have high overlap (we use over 90% as a threshold). For every bounding box, we label it with a unique ID next to it using a simple algorithm to minimizing the overlap between numeric labels and other bounding boxes. 




 §.§ Incorporating Local Semantics of Functionality

We found in a lot of cases where only inputting the UI screenshot overlayed with bounding boxes and associated IDs can be misleading to GPT-4V. We argue the limitation stems from GPT-4V's constrained ability to simultaneously perform the composite tasks of identifying each icon's semantic information and predicting the next action on a specific icon box. This has also been observed by several other works <cit.>. To address this issue, we incorporate the local semantics of functionality into the prompt, i.e. for each icons detected by the interactable region detection model, we use a finetuned model to generate description of functionality to the icons, and for each text boxes, we use the detected texts and its label.

We perform more detailed analysis for this topic in section <ref>. To the best of our knowledge, there is no public model that is specifically trained for up-to-date UI icon description, and is suitable for our purpose to provide fast and accurate local semantics for the UI screenshot. Therefore we curate a dataset of  7k icon-description pairs using GPT-4o, and finetune a BLIP-v2 model <cit.>  on this dataset. Details of dataset and training can be found in Appendix <ref>. After finetuning, we found the model is much more reliable in its description to common app icons. Examples can be seen in figure <ref>. And in figure <ref>, we show it is helpful to incorporate the semantics of local bounding boxes in the form of text prompt along with the UI screenshot visual prompt. 




















§ EXPERIMENTS AND RESULTS

We conduct experiments on several benchmarks to demonstrate the effectiveness of OmniParser. We start by a motivating experiments showing that current GPT-4V model with set of mark prompting <cit.> is prone to incorrectly assigning label ID to the referred bounding boxes. Then we evaluate on Seeclick benchmark and Mind2Web to further showcase OmniParser with local semantics can improve the GPT-4V's performance on real user tasks on different platforms and applications.   



 §.§ Evaluation on SeeAssign Task


To test the ability of correctly predicting the label ID given the description of the bounding boxes for GPT-4v models, We handcrafted a dataset SeeAssign that contains 112 tasks consisting of samples from 3 different platforms: Mobile, Desktop and Web Browser. Each task includes a concise task description and a screenshot image. The task descriptions are manually created and we make sure each task refers to one of the detected bounding boxes, e.g. 'click on 'settings”, 'click on the minimize button'. During evaluation, GPT-4V is prompted to predict the bounding box ID associated to it. Detailed prompt are specified in Appendix.  
The task screenshot images are sampled from the ScreenSpot <cit.> benchmark, where they are labeled with set of marks using OmniParser. The tasks are further divided into 3 sub-categories by difficulty: easy (less than 10 bounding boxes), medium (10-40 bounding boxes) and hard (more than 40 bounding boxes). 

From table <ref>, we see that GPT-4V often mistakenly assign the numeric ID to the table especially when there are a lot of bounding boxes over the screen. And by adding local semantics including texts within the boxes and short descriptions of the detected icons, GPT-4V's ability of correctly assigning the icon improves from 0.705 to 0.938.  

From figure <ref>, we see that without the description of the referred icon in the task, GPT-4V often fails to link the icon required in the task and the ground truth icon ID in the SoM labeled screenshot, which leads to hallucination in the response. With fine-grain local semantics added in the text prompt, it makes it much easier for GPT-4V to find the correct icon ID for the referred icon. 
























 §.§ Evaluation on ScreenSpot

ScreenSpot dataset <cit.>  is a benchmark dataset that includes over 600 interface
screenshots from mobile (iOS, Android), desktop
(macOS, Windows), and web platforms. The task instructions are manually created so that each instruction corresponds to an actionable
elements on the UI screen. We first evaluate the performance of omniparser using the this benchmark.  In table <ref>, we can see across the 3 different platforms: Mobile, Desktop and Web, omniparser significantly improves the GPT-4V baseline. Noticeably, omniparser's performance even surpasses models that are specifically finetuned on GUI dataset including SeeClick, CogAgent and Fuyu by a large margin. 
We also note that incorporating the local semantics (omniparser w. LS in the table) further improves the overall performance. This corroborates with the finds in section <ref> that incorporating local semantics of the UI screenshot in text format, i.e. adding OCR text and descriptions of the icon bounding boxes further helps GPT-4V to accurately identify the correct element to operate on.  
Furthermore, our findings indicate that the interactable region detection (ID) model we finetuned improves overall accuracy by an additional 4.3% compared to using the raw Grounding DINO model. This underscores the importance of accurately detecting interactable elements for the success of UI tasks. Overall, the results demonstrate that the UI screen understanding capability of GPT-4V is significantly underestimated and can be greatly enhanced with more accurate interactable elements detection and the incorporation of functional local semantics.

  



 §.§ Evaluation on Mind2Web

In order to test how OmniParser is helpful to the web navigation secnario, We evaluate on <cit.> benchmark. There are 3 different categories of task in the test set: Cross-Domain, Cross-Website, and Cross-Tasks.

We used a cleaned version of Mind2Web tests set processed from the raw HTML dump which eliminates a small number of samples that has incorrect bounding boxes. In total we have 867, 167, 242 tasks in the test set from Cross-Domain, Cross-Website, and Cross-Tasks category respectively. During evaluation, we feed both the parsed screen results and the action history as text prompt, and SOM labeled screenshot to GPT-4V similar to the prompting strategy in <cit.>. Following the original paper, we perform offline evaluation focusing on the element accuracy, Operation F1 and step success rate averaged across the task.

In the first section of the table (row 1-3), We report numbers from a set of open source VL models as it appears in <cit.>. Here CogAgent and Qwen-VL are not finetuned on the Mind2Web training set. More detailed information about model settings can be found in the Appendix<ref>. 

In the second section of the table (row 4-9) we report numbers from Mind2web paper <cit.> and SeeAct <cit.>
paper. In this section, all of the approaches use the HTML elements selected by a finetuned element proposal model on Mind2Web training set which produces top 50 relevant elements on the HTML page based on the user task. Additionally, GPT-4V+SOM and GPT-4V+textual choices corresponds to the SeeAct with image annotation, and textual choices grounding methods respectively. In GPT-4V+SOM, the set of mark (SOM) boxes are selected from the element proposal model, and are labeled with the ground truth location extracted from HTML. In contrast, GPT-4V+textual uses DOM information of the selected relevant elements directly in the text prompt, rather than overlaying bounding boxes on top of screenshot. The better performance of textual choice corroborates with the experiment results in <ref>. 

In the last section (row 10-11), we report numbers from OmniParser. We observe GPT-4V augumented with local semantics of icon functionality and the finetuned interactable region detection model (w. LS + ID) performs better than the model with raw grounding DINO model (w. LS + GD) in all of the categories. 


Further, without using parsed HTML information, OmniParser is able to outperform GPT-4's performance that uses HTML in every sub-category by a large margin, suggesting the substantial benefit of the screen parsing results provided by OmniParser. 
Additionally, OmniParser outperforms the GPT-4V+SOM by a large margin. Compared to GPT-4V+textual choices, OmniParser significantly outperforms in Cross-Website and Cross-Domain category (+4.1% and +5.2%), while underperforming (-0.8%) slightly in the Cross-Task category, which indicates that OmniParser provides higher quality information compared ground truth element information from DOM and top-k relevant elemnt proposal used by the GPT-4V+textual choices set-up, and make the GPT-4V easier to make a accurate action prediction. 
Lastly, OmniParser with GPT-4V significantly outperform all the other trained models using only UI screenshot such as SeeClick and Qwen-VL.  








 §.§ Evaluation on AITW

In additional to evaluation on multi-step web browsing tasks, we assess OmniParser on the mobile navigating benchmark AITW <cit.>, which contains 30k instructions and 715k trajectories. We use the same train/test split as in <cit.> based on instructions, which retain only one trajectory for each instructions and no intersection between train and test. For a fair comparison, we only use their test split for evaluation and discard the train set as our method does not require finetuing.  

In table <ref>, we report the GPT-4V baseline in <cit.> paper, which corresponds to the best performing set up (GPT-4V ZS+history) that uses UI elements detected by IconNet <cit.> through set-of-marks prompting <cit.> for each screenshot at every step of the evaluation. The detected UI elements consist of either OCR-detected text or an icon class label, which is one of the 96 possible icon types identified by IconNet. Additionally, action history is also incorporated at each step's prompt as well.  We used the exact same prompt format in <cit.> except the results from the IconNet model is replaced with the output of the finetuned interactable region detection (ID) model.
Interestingly, we found that the ID model can generalize well to mobile screen. By replacing the IconNet with the interactable region detection (ID) model we finetuned on the collected webpages, and incorporating local semantics of icon functionality (LS), we find OmniParser delivers significantly improved performance across most sub-categories, and a 4.7% increase in the overall score compared to the best performing GPT-4V + history baseline. 










§ DISCUSSIONS


In this section, we discuss a couple of common failure cases of OmniParser with examples and potential approach to improve. 



Repeated Icons/Texts From analysis of the the GPT-4V's response log, we found that GPT-4V often fails to make the correct prediction when the results of the OmniParser contains multiple repeated icons/texts, which will lead to failure if the user task requires clicking on one of the buttons. This is illustrated by the figure <ref> (Left) in the Appendix. A potential solution to this is to add finer grain descriptions to the repeated elements in the UI screenshot, so that the GPT-4V is aware of the existence of repeated elements and take it into account when predicting next action. 



Corase Prediction of Bounding Boxes One common failure case of OmniParser is that it fails to detect the bounding boxes with correct granularity. In figure <ref> (Right), the task is to click on the text 'MORE'. The OCR module of OmniParser detects text bounding box 8 which encompass the desired text. But since it uses center of the box as predicted click point, it falls outside of the ground truth bounding box. This is essentially due to the fact that the OCR module we use does not have a notion of which text region are hyperlink and clickable. Hence we plan to train a model that combines OCR and interactable region detection into one module so that it can better detect the clickable text/hyperlinks. 


Icon Misinterpretation We found that in some cases the icon with similar shape can have different meanings depending on the UI screenshot. For example, in figure <ref>, the task is to find button related to 'More information', where the ground truth is to click the three dots icon in the upper right part of the screenshot.  OmniParser successfully detects all the relevant bounding boxes, but the icon description model interpret it as: "a loading or buffering indicator". We think this is due to the fact that the icon description model is only able to see each icon cropped from image, while not able to see the whole picture during both training and inference. So without knowing the full context of the image, a symbol of three dots can indeed mean loading buffer in other scenarios. A potential fix to this is to train an icon description model that is aware of the full context of the image. 






§ CONCLUSION

In this report, We propose OmniParser, a general vision only approach that parse UI screenshots into structured elements.  OmniParser encompasses two finetuned models: an icon detection model and a functional description models. To train them, we curated an interactable region detection dataset using popular webpages, and an icon functional description dataset. We demonstrate that with the parsed results, the performance of GPT-4V is greatly improved on ScreenSpot benchmarks. It achieves better performance compared to GPT-4V agent that uses HTML extracted information on Mind2Web, and outperforms GPT-4V augmented with specialized Android icon detection model on AITW benchmark. We hope OmniParser can serve as a general and easy-to-use tool that has the capability to parse general user screen across both PC and mobile platforms without any dependency on extra information such as HTML and view hierarchy in Android. 




§ ACKNOWLEDGEMENT

We would like to thank Corby Rosset and authors of ClueWeb22 for providing the seed urls for which we use to collect data to finetune the interactable region detection model. The data collection pipeline adapted AutoGen's multimodal websurfer code for extracting interatable elements in DOM, for which we thank Adam Fourney. We also thank Dillon DuPont for providing the processed version of mind2web benchmark.



alpha





§ APPENDIX



 §.§ Details of Icon-Description Dataset


In figure <ref>, we see that the original BLIP-2 model tend to focus on describing shapes and colors of app icons, while struggling to recognize the semantics of the icon. This motivates us to finetune this model on an icon description dataset. For the dataset, we use the result of parsed icon bounding boxes inferenced by the interactable icon detection model on the ScreenSpot dataset since it contains screenshots on both mobile and PC. For the description, we ask GPT-4o whether the object presented in the parsed bounding box is an app icon. If GPT-4o decides the image is an icon, it outputs one-sentence description of the icon about the potential functionality. And if not, GPT-4o will output 'this is not an icon', while still including this in the dataset. In the end, we collected 7185 icon-description pairs for finetuning. 

We finetune BLIP-2 model for 1 epoch on the generated dataset with constant learning rate of 1e^-5, no weight decay and Adam optimizer. We show a few of the qualitative examples of finetuned model vs the original model in figure <ref>. 





 §.§ Training details of Interactable Icon Region Detection Model

As introduced in <ref>, we train a YOLOv8 model on the interactable icon region detection dataset. We collect in total of 66990 samples where we split 95% (63641) for training, and 5% (3349) for validation. We train for 20 epochs with batch size of 256, learning rate of 1e^-3, and the Adam optimizer on 4 GPUs. We show the training curve in figure <ref>. 






 §.§ Details of SeeAssign Evaluation



  §.§.§ Prompt Used for GPT-4V

GPT-4V without local semantics: 

Here is a UI screenshot image with bounding boxes and corresponding labeled ID overlayed on top of it, your task is task. Which icon box label you should operate on? Give a brief analysis, then put your answer in the format of “`Box with label ID: [xx]“`

GPT-4V with local semantics: 

Here is a UI screenshot image with bounding boxes and corresponding labeled ID overlayed on top of it, and here is a list of icon/text box description: parsed_local_semantics. Your task is task. Which bounding box label you should operate on? Give a brief analysis, then put your answer in the format of “`Box with label ID: [xx]“`



 §.§ Details of Mind2Web Evaluation

Here we list more details of each baseline in table <ref>.


SeeClick, QWen-VL SeeClick is a finetuned version of Qwen-VL on the Mind2Web training set and we report both of their numbers in their paper. 

CogAgent CogAgent number is taken from the SEEAct paper <cit.>, where they report cogagent-chat-hf checkpoint that is not fine-tuned on Mind2Web for experiments. 

MindAct(Gen), MindAct, GPT-3.5-Turbo, GPT-4 The numbers for these baseline are taken from the Mind2Web <cit.> paper, where they use HTML information to augument the corresponding web agent.

GPT-4V+som This model corresponds to the image annotation grounding method in SeeAct paper, where the som boxes extracted from the selelcted HTML elements are provided to GPT-4V to make action prediction.

GPT-4V+textual choice This corresponds to the best performing scenario in SeeAct paper (except the Oracle), that uses the selected HTML elments information in a multi-choice question format as input to the GPT-4V agent. 



  §.§.§ Qualitative Examples

We list a few more examples to demonstrate local semantics of icon function description helps GPT-4V make better action prediction in figure <ref>.


























 Paper Title: MSMA: Multi-agent Trajectory Prediction in Connected and Autonomous Vehicle Environment with Multi-source Data Integration.


On-the-fly Point Feature Representation for Point Clouds Analysis
    Yang Xulei
    August 5, 2024
=================================================================



§ ABSTRACT 
The prediction of surrounding vehicle trajectories is crucial for collision-free path planning. In this study, we focus on a scenario where a connected and autonomous vehicle (CAV) serves as the central agent, utilizing both sensors and communication technologies to perceive its surrounding traffics consisting of autonomous vehicles (AVs), connected vehicles (CVs), and human-driven vehicles (HDVs). Our trajectory prediction task is aimed at all the detected surrounding vehicles. To effectively integrate the multi-source data from both sensor and communication technologies, we propose a deep learning framework called MSMA utilizing a cross-attention module for multi-source data fusion. Vector map data is utilized to provide contextual information. The trajectory dataset is collected in CARLA simulator with synthesized data errors introduced. Numerical experiments demonstrate that in a mixed traffic flow scenario, the integration of data from different sources enhances our understanding of the environment. This notably improves trajectory prediction accuracy, particularly in situations with a high CV market penetration rate. The code is available at: <https://github.com/xichennn/MSMA>.









§ INTRODUCTION



Predicting the trajectories of surrounding vehicles is a critical component of ensuring collision-free path planning. Autonomous vehicles (AVs) provide promising solutions to achieving safer road environment <cit.> facilitated by the technological advancement in sensors (camera, LiDAR, radar, etc) and artificial intelligence. Connected vehicle (CVs) technologies are also being deployed to improve the road safety by enhancing vehicular situational awareness through various CV options such as vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) communications. With the same objective of improving road safety, CVs and AVs really should be developed in a cooperative manner. As argued in <cit.>, CV technology provides information that AV sensors can not see immediately (e.g. maneuver commands issued to other vehicles) as well as information that may be occluded by adjacent vehicles. Two examples are shown in Figure <ref>. In both scenarios, the AV failed to detect the potential hazard in advance due to its limited field of view. However, if the green vehicle could communicate its trajectory information to the AV, the accidents could be effectively avoided. We believe that the integration of AV and CV technologies can provide complementary benefits, significantly enhancing the performance and safety of the transportation system. Hereinafter, we refer to a vehicle with attributes from both sides as connected and autonomous vehicle (CAV), whereas vehicles without these attributes are termed Human-Driven Vehicles (HDVs). 



While extensive research has focused on trajectory prediction within the context of AV, trajectory prediction studies within the CAV environment have garnered comparatively less attention, primarily due to the low market penetration rate (MPR) of CVs. Gathering real-world data in the CAV environment is inherently more complex than in purely autonomous settings. Given that CAVs are still in the early stages of deployment, obtaining large-scale datasets with comprehensive trajectory information from actual CAVs navigating intricate traffic scenarios remains a limitation. Most CAV trajectory prediction studies rely on datasets like the Next Generation Simulation (NGSIM), which, regrettably, lacks CAV-specific data. Consequently, it may not fully encapsulate the distinct characteristics and behaviors exhibited by CAVs. 

Before the full-scale deployment of either connected or autonomous vehicular technology, it is anticipated that a prolonged transition period will ensue, characterized by a mixed traffic environment consisting of CVs, AVs, CAVs and HDVs. With the gradual deployment of CAV technology in real-world setting, studying the trajectory prediction in a CAV environment can help identify the limitations and robustness for trajectory prediction models that need to operate in complex and dynamic traffic environment. 

 To address this gap, we investigate a mixed traffic scenario in which a CAV serves as the central agent, as illustrated in  Figure <ref>. The CAV collects information from its surroundings using both sensors and communication technology. Typically, the communication range extends farther than the sensing range, covering approximately 300 meters for the former and 200 meters for the latter <cit.>. As a result, connected vehicles within the sensing zone provide dual observations. It is worth noting that we categorize both AVs and HDVs as non-connected vehicles (NCVs) since information regarding these two categories is solely acquired through sensors. It is crucial to recognize that data obtained from different sources inherently carry distinct types of errors, with sensor noise and communication latency being the most common ones. Leveraging the complementary nature of information from these sources holds the potential to enhance scene understanding and, importantly, mitigate errors associated with each individual source. To facilitate our study, we collected a trajectory dataset in the CARLA simulator <cit.>, introducing synthesized data errors tailored to each source to accurately reflect their inherent characteristics. 
 
 A novel deep learning framework was proposed comprising a multi-source encoder, an agent-agent and agent-lane interaction module and a multi-agent decoder. The multi-source encoder efficiently processes historical vehicle trajectories by employing source-specific temporal encoders to capture the temporal dependencies and a cross-attention fusion module for multi-source data integration. By representing each agent and lane as graph nodes, we leverage Graph attention models (GAT) to capture agent-agent and agent-lane interactions. The multi-agent decoder simultaneously predicts the trajectories of all surrounding vehicles. An Ablation study is conducted to assess the impact of multi-source data and the cross-attention fusion module on the model’s performance. Furthermore, we compare model's performance across different MPRs to unveil the synergistic benefits of sensor and communication technologies. Additionally, we illustrate the prediction results on individual scenes. Finally, a comparative study with existing research work is presented. 



The main contributions of this work lie in three-folds:

  
  * We are the first to study the impact of multi-source data for trakectory prediction in a mixed traffic flow.
  
  * A customized motion dataset with map geometry was collected in CARLA and utilized for the validation of the problem. 
  
  * A novel deep learning framework is proposed to handle the multi-source data fusion, agent-agent and agent-lane interactions, and multi-agent predictions.




§ RELATED WORK



 §.§ Publicly Available Motion Dataset

There is a plethora of open motion datasets collected by autonomous vehicle on-board sensors in urban scenarios such as Waymo  <cit.>, ArgoVerse <cit.>, ArgoVerse2 <cit.>, NuScenes <cit.>, etc., which have significantly advanced trajectory prediction research within the context of AVs. Research that utilizes these datasets often assumes a flawless data source and does not account for the inherent data errors they may encompass. The NGSIM dataset, commonly employed in the context of CAV, was collected by video cameras mounted on buildings and lacks CAV specific scenarios. Primarily, it encompasses highway driving scenarios. The V2X-Seq trajectory dataset <cit.> provides valuable multi-view trajectory information, which is closely related to our study. However, our work differs as we focus on the overlapping range of on-board sensors and communication technologies, where complementary information can be utilized. To the best of our knowledge, there are currently no publicly available motion datasets dedicated to CAV research. Therefore, we will leverage a simulated dataset generated by CARLA, affording us the flexibility to construct CAV scenarios and explore data sources featuring diverse characteristics.



 §.§ Trajectory Prediction in Connected and Autonomous Vehicle Environment



  §.§.§ Autonomous Environment

Due to the temporal-spatial characteristics of trajectory data, recurrent neural networks (RNN) with long short-term memory (LSTM) or gated recurrent unit (GRU) architectures are extensively employed to predict future trajectories taking their historic trajectory as input  <cit.>. Social-LSTM was proposed considering interactions among neighboring vehicles <cit.>. The spatial information was encoded by convolution neural networks (CNN) or graph neural networks (GNN) <cit.>. As vehicles' decisions are largely associated with the map topology, an emerging avenue of research is proposed to leverage the high-definition maps (HD maps) to assist the prediction. <cit.> rasterized the traffic scenes into birds' eye view images where map vector layers (e.g. polygons, polylines) are encoded into RGB color space. Conventional CNN are then used to model the scene. As the image-based representation is dense and computationally expensive, subsequent work proposed to use more compact and efficient vectorized representation (polygons, polylines) directly <cit.> and then model by graph convolutional networks (GCN) <cit.>, transformers <cit.>, or point cloud networks <cit.>. 



  §.§.§ Connected Environment

Trajectory prediction models proposed in CV lags behind the ones developed in AV as autonomous driving has a much higher requirement on the trajectory prediction accuracy. <cit.> first clustered the training data then employed Gaussian process regression to learn the motion pattern. <cit.> recognized three driving styles using Gaussian mixture model, then proposed a personalized joint time series model based on LSTM to predict the front vehicle trajectories. <cit.> proposed a CNN-LSTM framework utilizing 8 surrounding vehicles to predict the future trajectory of an ego-vehicle. <cit.> developed a heterogeneous context-aware graph convolutional networks which combines the information encoding of historical vehicle trajectories, scene image stream and inter-vehicle interaction patterns to predict the future trajectory of a target vehicle. 



  §.§.§ Connected and Autonomous Environment

There is limited literature in the CAV context. <cit.> proposed to utilize all the historic trajectories of surrounding CAVs to predict the longitudinal trajectory of a HDV in front of the ego-vehicle. Promising results showed that information from CAV improved the model performance even when the market penetration rate is as low as 20%. <cit.> proposed an interactive network model utilizing LSTM and CNN with a correction mechanism. The model enables a CAV to predict the trajectories of surrounding vehicles, and the trajectories can be corrected during the prediction process to improve the prediction accuracy. Both models are tested on the NGSIM dataset which contains only limited driving scenarios: highway driving (including ramp merging and double lane change) and signalized intersection scenarios. 

We will essentially follow a similar approach for modeling temporal, spatial, and contextual information, with the distinction of operating within a CAV environment where we will account for multi-source data integration.








 §.§ Multi-source Data Fusion in Cooperative Driving

There is a growing trend in leveraging multi-source data through Vehicle-to-Everything (V2X) communications to enhance model performance. However, existing research primarily focuses on cooperative perception, where perceptions from different viewpoints are fused <cit.>. Some studies have considered joint perception and motion forecasting, but the emphasis on data fusion persists mainly in the perception stage <cit.>. With the recent release of a V2X trajectory dataset <cit.>, we anticipate an increase in research directly addressing trajectory data fusion.



§ DATA PREPARATION AND SCENE REPRESENTATION



 §.§ Data Preparation


The traffic was simulated in CARLA Town03 using the Traffic Manager (TM) module, which controls vehicles in autopilot mode. The simulation interface is shown in Figure <ref>(a). The simulation lasted for 270 seconds with a sampling rate of 10Hz. The positions of the 265 populated vehicles were logged at each timestamp. 



To construct the scenario depicted in Figure <ref>, we employed the following process on the collected dataset: First, we iterated through all the vehicles designated as CAVs, one at a time. For each CAV at time t, we identified all the neighboring vehicle IDs within a 50-meter radius which is the communication range we employed. Among these IDs, we randomly assigned CV IDs according to the MPR. We also identified all the IDs that fall within a 30-meter sensing range. The trajectory data for these identified IDs were processed within a time window spanning from t-5 seconds to t+5 seconds, constituting a 10s time window. Within this window, we allocated 3s for motion history and 5s for the prediction horizon. The initial 2s were reserved for simulating latency errors. 

To simulate latency, we introduce delays ranging from 1 to 15 frames to the historical trajectories of CV IDs, shifting the ground-truth trajectories. Additionally, Gaussian noises with variances ranging from 0 to 0.5 are introduced to the historical trajectories of sensor IDs. It's important to note that for IDs found in both CV IDs and sensor IDs, we maintain dual sets of historical trajectories — one representing the delayed data and the other the noisy data. This approach allows us to account for both latency and sensor noise in the dataset.

The map of Town03 is publicly available in OpenStreetMap (OSM) XML format (shown in Figure <ref>(b)) and is processed following the Argoverse map API (Chang, 2019). We utilize the lane centerlines within a 75-meter radius of the CAV. 



 §.§ Scene Representation

Vectorized representation was employed for its efficiency <cit.>. The scenes were first transformed such that the CAVs are positioned at the origin with its heading aligned along the positive x-axis.

Specifically, each trajectory can be represented as a sequence of displacements  {Δ𝐩_t}^0_t=-(T_h-1), where Δ𝐩_t∈ℝ^2 is the 2D displacement from time step t-1 to t and T_h is the total historical time steps. Similarly, for a lane centerline composed of lane segments, each segment can be represented as Δ𝐩_l^k ∈ℝ^2, which captures the 2D displacement from the starting coordinate to the end coordinate of lane segment k.



§ METHODOLOGY



 §.§ Overview

An overview of our proposed model is illustrated in Figure <ref>. It consists of a multi-source temporal encoder, a Graph Attention Network (GAT) based agent-agent encoder, an agent-lane encoder and a multi-agent decoder. 





 §.§ Multi-source Temporal Encoder

To differentiate between data obtained from sensors and communication technologies, we employ source-specific temporal encoders for capturing temporal dependencies. While both RNNs and Transformers are suitable for handling temporal data, we use Transformer as the base model due to its efficiency in capturing long-range dependencies. Specifically, a self-attention mechanism is employed which allows the agent to focus on relevant temporal information from its own historical trajectory. 

We denote the input from agent i and source s as {Δ𝐩_t^i_s}^0_t=-(T_h-1), where s ∈{1,2,3}. Source 1 represents the input directly from CAV itself, for which we utilize ground-truth data in this context. Source 2 corresponds to input from communication technologies, where latency has been intentionally introduced. Source 3 captures input from sensor technologies which contains noise. 

A multilayer perceptron (MLP) is applied to the input to get its embedding 𝐙^i_s∈ℝ^T_h × d_h, where d_h is the embedding dimension. Following BERT <cit.> and HiVT <cit.>, we then append a learnable token 𝐳^i_s∈ℝ^d_h to the end of the embedding. Learnable positional encodings are then added to the embedding. The updated embedding 𝐙^i_s∈ℝ^(T_h+1) × d_h is then fed into the self-attention block:

     
           𝐐^i_s=𝐙^i_s𝐖^Q_s, 𝐊^i_s=𝐙^i_s𝐖^K_s, 𝐕^i_s=𝐙^i_s𝐖^V_s

           𝐙̃^i_s = softmax(𝐐^i_s𝐊^i_sT)/√(d_k)𝐕^i_s
    

where 𝐖^Q_s, 𝐖^K_s, 𝐖^V_s∈ℝ^d_h × d_k are learnable matrices for linear projection and d_k is the dimension of the transformed vectors. 1/√(d_k) implements the scaled dot product term for numerical stability of attention. Multiple heads can be used to attend to different parts of the sequence simultaneously. In addition, we apply Dropout and Layer Normalization to stabilize the training process. The updated appended token 𝐳̃^i_s∈ℝ^d_k summarizes of the temporal features. For agents with a single data source, this is the input to the subsequent agent-agent interaction module.


For agents with dual data sources, we apply across-attention to align and fuse the temporal features across the two sources. The temporal feature from sensor source is converted to the query vector and the temporal feature from communication source is used to compute the key and value vectors:


     
           𝐪̃^i_3=𝐳̃^i_3𝐖^Q_f, 𝐤̃^i_2=𝐳̃^i_2𝐖^K_f, ṽ^i_2=𝐳̃^i_2𝐖^V_f

           𝐅^i_3 = softmax(𝐪̃^i_3𝐤̃^i_2T)/√(d_k)ṽ^i_2

           α^i = sigmoid(𝐖^α[𝐳̃^i_3, 𝐅^i_3])

           Ẑ^i = α^i ⊙𝐖^self𝐳̃^i_3+(1-α^i)⊙𝐅^i_3
    


where 𝐖^Q_f, 𝐖^K_f, 𝐖^V_f, 𝐖^α, 𝐖^self∈ R^d_k × d_k are learnable matrices and ⊙ denotes element-wise product. 𝐅^i_3 represents the aligned features from the communication source in the sensor source. We then apply sigmoid to obtain a weight vector which computes the final fused representation 𝐳̂^i∈ R^d_k from the two sources. 𝐳̂^i is then the input for agents with dual sources to the next module as illustrated in Figure <ref>.



 §.§ Agent-agent Interaction

To capture agent-agent interactions, we employ the GAT architecture <cit.> to attend to neighboring features. Each agent is represented as a node within the graph. The temporal features 𝐳̃^i (𝐳̂^i for agents with dual source) we have just obtained serve as node features. The relative positions 𝐩^i_0-𝐩^j_0 between two agents i and j at the last observed step t=0 are modeled as edge attributes. GAT dynamically adjusts the weight of each edge during the message-passing process. We utilize multiple stacked graph attention layers to compute and update the edge weight e_ij. The resulting weighted information aggregation from surrounding agents of agent i can be expressed as follows:

    
       e_ij = a_G(𝐳^i𝐖^G, 𝐳^j𝐖^G, (𝐩^i_0-𝐩^j_0)𝐖^G) 

       α_ij = Softmax(e_ij) 

       𝐒^l(i) = ∑_j∈N_i α_ij𝐳^j𝐖^G
    

where W^G ∈ℝ^d_k × d_g is the shared linear transformation, d_g is the new feature dimension, the shared attentional mechanism a_G is a single-layer feedforward neural network: ℝ^d_kℝ that computes the edge weight e_ij indicating the importance of vehicle j's feature to vehicle i, and S^l is the output of the lth layer by summing up the weighted features from all the neighboring vehicles. Output from the last layer l̃ is the updated node feature, denoting as 𝐒̃^i. 



 §.§ Agent-lane Interaction

Now we consider the context information provided by lane centerlines. Each lane segment Δ𝐩_l^k ∈ℝ^2 is represented as a graph node, where the node feature is obtained by an MLP embedding ℝ^2 ℝ^d_g:

    
    𝐋 = MLP_L(Δ𝐩_l^k)


We have ϵ total segments queried within a 75m radius of the CAV. To efficiently assess relevant lane information while mitigating the computational load associated with distant or non-essential segments, we employ a 30m local radius for each traffic agent. Consequently, only lane segments falling within this local radius will be taken into account when evaluating interactions.

The edge attribute between a lane segment k and an agent i is determined by their relative position 𝐩_l^k-𝐩^i_0. If lane node k and agent node i are connected, we calculate the edge weight e_ik using graph attention layers:

     
           e_ik = a_L(𝐋^k𝐖^L,  𝐒^i𝐖^L, (𝐩_l^k-𝐩^i_0)𝐖^L)

           α_ik = Softmax(e_ik) 

           𝐌^l(i) = ∑_k∈N_i α_ik𝐋^k𝐖^L
    

where 𝐖^L ∈ℝ^d_g × d_g is the shared linear transformation, a_L is a feedforward neural network as in Eq. <ref> and 𝐌^l represents the aggregated lane information from the l-th layer. We output the node features from the last layer l̃ node feature, denoting as 𝐌̃^i, which is then concatenated with 𝐒̃^i as the final temporal-spatial embedding for agent i. 



 §.§ Multi-agent Decoder

We parameterize the distribution of future trajectories as a mixture model where each mixture component is a Gaussian distribution. The predictions are made for all agents in a single shot. For each agent, the model predicts D possible future trajectories (mixture components) and their confidence scores (mixing coefficients). The decoder has two headers: a regression head to predict the trajectory of each mode and a classification head to predict the confidence score of each mode. For agent i and its mode d, in the regression head, an MLP receives the final embedding as inputs and outputs the location μ^t_i,d∈ℝ^2 and its associated covariance σ^2^t_i,d∈ℝ^2*2 of the agent per future time step. In the classification head, we use another MLP followed by a softmax function to produce the confidence scores for each mode.


 §.§ Model Training

Since all the modules are differentiable, we can train the model end-to-end. The Loss function is defined as

    
    J = J_reg + α J_cls


Here, J_reg represents the loss from the regression head, and J_cls represents the classification loss. After experimentation, we have set α to 0.5. Given a set of D predicted trajectories, we identify the trajectory d̃ with the minimum Average Displacement Error (ADE), which is defined as the average Euclidean distance between ground-truth locations and predicted locations across all future time steps. We utilize the negative log-likelihood to calculate the regression loss:

    
    J_reg = -1/nT_f∑^n_i=1∑^T_f_t=1 logP(𝐩_t^i,pred - 𝐩_t^i,gt | μ̂^i,d̃_t,σ̂^̂2̂^i,d̃_t)


where T_f is the prediction horizon, n is the number of agents, P(|) is the probability density function of Gaussian distribution and μ̂_t^i,d̃, σ̂^̂2̂_t^i,d̃ are the mean and covariance from predicted trajectory d̃.

For J_cls, cross-entropy loss is applied:

    
    J_cls = -1/n∑^n_i=1∑^D_d=11_d logp_d


where 1_d is a binary indicator if d is the best mode, p_d is the confidence score of mode d.

The model was trained for 100 epochs using AdamW optimizer with initial learning rate 1e-3 for 10 warm-up epochs. The learning rate was then decayed every 10 epochs by a factor of 0.9. Our model consists of 4 layers for temporal encoding, 8 layers for both agent-agent and agent-lane interactions, 8 heads for all attention modules. We used 16-dimensional input embedding size and 64 for all hidden dimensions. The dropout rate was 0.1 throughout the architecture.


§ RESULTS ANALYSIS

The complete dataset was split into train, validation and test set with a ratio of 8:1:1. The standard metrics in motion predictions are adopted, including Average Displacement Error (ADE), Final Displacement Error (FDE), and Miss Rate (MR), where errors between the best predicted trajectory among the D=5 modes and the ground truth trajectory are calculated. The best here refers to the trajectory that has the minimum endpoint error. The ADE metric calculates the L2 distance across all future time steps and averages over all scored vehicles within a scenario, while FDE measures the L2 distance only at the final future time step and summarizes across all scored vehicles. MR refers to the ratio of actors in a scenario where FDE are above 2 meters.


 §.§ Ablation Study

To demonstrate the impact of cross-attention fusion, which aligns input from sensor and communication sources, we conducted two ablation studies under MPR=0.8. 



In the first study, our objective was to assess the fusion module's resilience to communication latency. This involved introducing a time delay to all historical CV trajectories, varying from 1 to 15 frames, i.e., 0.1 to 1.5 seconds. Gaussian noise was held constant at 0 for sensor trajectories. We then compared the ADE, FDE and MR metrics for all CVs under the model architectures with and without the cross-attention fusion module. The results, presented in Figure <ref>(a), clearly indicate that the inclusion of fused information from error-free sensors enhances the robustness of CVs against communication latency.

In the second study, we aim to evaluate the fusion module's ability to withstand sensor noise. We introduced Gaussian noises with varying variances, ranging from 0 to 0.5, to all historical trajectories of vehicles within the sensing range, while no latency was introduced to CV trajectories in this evaluation. The metrics comparison is shown in Figure <ref>(b), It is evident that the incorporation of fused information from communication technologies effectively mitigated the impact of sensor noises.


 §.§ Performance under Different CV Market Penetration Rates

We are also interested in assessing how varying numbers of CVs in the scenario influence the model's performance. To conduct this evaluation, models are trained under different MPRs with a fixed 1-frame delay and 0.1 noise variance. Since scenarios corresponding to different MPRs consist of varying numbers of CVs, to ensure a fair comparison, we evaluate the prediction performance using ADE exclusively for vehicles within the sensor range across different MPRs. The evaluation results are presented in Figure <ref>. We can observe a clear trend of performance improvement as the MPR increases especially at longer prediction horizons. 




To highlight the extent of improvement, we selected two scenarios for a comparative analysis of multi-agent predictions, specifically under MPR=0 and MPR=0.8. The first scenario includes vehicles navigating a roundabout, as depicted in Figure <ref>. Notably, when MPR=0, the predictions exhibit a slight deviation, as indicated by the arrows. However, a marked improvement is observed in both vehicles' predictions when they benefit from dual observations under MPR=0.8. Similar improvements are observed in the second scenario shown in Figure <ref>, where vehicles are driving in opposite directional turning lanes.








 §.§ Comparison with Existing Work

To evaluate the effectiveness of the proposed architecture MSMA, we compare with the following baselines on the same CARLA dataset:

  
  * LSTM Encoder Decoder: We use the vanilla setting with the decoder modified to accommodate 5 modes as described in this work. It predicts the future trajectory for a single agent at a time and only historical trajectory of the agent itself is utilized for prediction. 
  
  * VectorNet: We consider the first work using vectorized scene representation described in <cit.>. The default settings are employed without auxiliary graph completion. Only the central CAV’s future trajectory gets predicted thus reported. 
  
  * HiVT: We use the default setting with 64 hidden dimensions <cit.>. All the surrounding agents’ future trajectories were predicted. 


Ground truth historical trajectories are used for the baseline models as they do not handle multi-source data. MSMA with 1 frame delay and 0.1 gaussian variance under different MPRs are evaluated. ADEs across all scored agents are calculated. Table <ref> shows the displacement errors at each second and the ADEs for the models being compared. 

We note that LSTM produce the highest ADE as it lacks the information from neighboring agents and map data. While VectorNet shows slightly better performance, it struggles to capture both temporal and spatial relations effectively using a single GNN. On the other hand, HiVT significantly improves trajectory prediction accuracy by explicitly modeling both temporal information and spatial interactions using attention mechanisms. In comparison, MSMA performs even better, particularly when there is more complementary information available from higher CV MPRs.





§ CONCLUSION

In this work, we proposed a multi-source multi-agent (MSMA) encoder-decoder model for vehicle trajectory prediction in a connected and autonomous vehicle environment. Compared to existing models that rely solely on a single data source, our model incorporates the complementary information provided by both sensors and communication technologies to improve prediction accuracy. A customized CARLA data set was collected, data errors including noise and latency were introduced to accurately reflect the real-world scenarios. A cross-attention fusion module was proposed to align and integrate the perturbed data sources. Numerical experiments showcase that integrating complementary data sources helps improve model prediction performance, particularly when the market penetration rate of connected vehicles is high. 

While our method demonstrates efficacy, it has certain limitations. First, it assumes that CVs broadcast only their own trajectories, overlooking the potential benefits of cooperative driving. Enhanced collaboration could be achieved if each CV possesses information about surrounding vehicles within its communication range, and broadcasting this comprehensive information could augment the surrounding coverage. Second, the model assumes homogeneity among vehicles, and there is room for improvement by considering heterogeneous interactions through graph edges that reflect distinct vehicle characteristics. Last, despite the high-fidelity real-world scenarios provided by CARLA simulation, a real-world dataset would offer deeper insights into vehicle behaviors.




unsrt  
  




 Paper Title: Contrastive Graph Representation Learning with Adversarial Cross-view Reconstruction and Information Bottleneck.

















































address2]Yuntao Shou^a ,
shouyuntao@stu.xjtu.edu.cn

address3]Haozhi Lan
haozhilan1@gmail.com

address2]Xiangyong Cao^a ,cor1
caoxiangyong@mail.xjtu.edu.cn
[cor1]Corresponding author

[address1]School of Computer Science and Technology, Xi’an Jiaotong University, Xi’an, China

[address2]Ministry of Education Key Laboratory for Intelligent Networks and Network Security, Xi’an Jiaotong University, Xi’an, China

[address2]School of Artificial Intelligence, Xi'an Jiaotong University





§ ABSTRACT 
Graph Neural Networks (GNNs) have received extensive research attention due to their powerful information aggregation capabilities. Despite the success of GNNs, most of them suffer from the popularity bias issue in a graph caused by a small number of popular categories. Additionally, real graph datasets always contain incorrect node labels, which hinders GNNs from learning effective node representations. Graph contrastive learning (GCL) has been shown to be effective in solving the above problems for node classification tasks. Most existing GCL methods are implemented by randomly removing edges and nodes to create multiple contrasting views, and then maximizing the mutual information (MI) between these contrasting views to improve the node feature representation. 
However, maximizing the mutual information between multiple contrasting views may lead the model to learn some redundant information irrelevant to the node classification task. To tackle this issue, we propose an effective Contrastive Graph Representation Learning with Adversarial Cross-view
Reconstruction and Information Bottleneck (CGRL) for node classification, which can adaptively learn to mask the nodes and edges in the graph to obtain the optimal graph structure representation. Furthermore, we innovatively introduce the information bottleneck theory into GCLs to remove redundant information in multiple contrasting views while retaining as much information as possible about node classification. Moreover, we add noise perturbations to the original views and reconstruct the augmented views by constructing adversarial views to improve the robustness of node feature representation. Extensive experiments on real-world public datasets demonstrate that our method significantly outperforms existing state-of-the-art algorithms.




Graph Neural Networks, Contrastive Learning, Mutual Information, Information Bottleneck, Adversarial Learning







§ INTRODUTION

Graph Neural Networks (GNNs) have attracted extensive attention from researchers due to the increase in large amounts of real-world graph-structured data <cit.>. Meanwhile, GNNs are widely used in intelligent recommender systems, and social media fields because they provide a practical way to aggregate high-order neighbor information <cit.>.




Although GNNs have achieved reliable performance on node classification tasks, we argue that most of the node classification models based on GNNs suffer from the following two problems. i) Popularity Bias. As shown in Fig. <ref>, different categories of papers have different numbers, and the number of citations of papers also varies, and this unbalanced learning can lead to the popularity bias problem in GNNs learning. In most node classification tasks, the categories and degrees of nodes follow a long-tail distribution, which means that popular categories have many papers, while most papers have few citations. In other words, most of the nodes have less interaction, which hinders the information update of the nodes. The above-mentioned popularity bias problem cause GNNs to tend to learn node representations that are popular and have many interactions, which hinders the representation learning of GNNs. ii) Noise Interference. There may be miscitations in the citation process of papers (i.e., there is a citation relationship between two unrelated papers in different fields), which leads to noise in the information contained in the data. Studies have shown that the feature extraction ability of GCN is closely related to the quality of the input graph, which means that the input image with noise may cause the model to learn poor solutions. In response to the above problems, existing graph contrastive learning (GCL) methods <cit.>, <cit.>, <cit.>, <cit.>, <cit.> propose an effective solution mechanism to alleviate popular bias and improve the robustness of the GCN model.

Nonetheless, the above-mentioned methods suffer from two limitations. 1) Most GCL methods perform data augmentation to optimize the graph structure by randomly masking nodes or perturbing edges. However, the strategy of randomly masking nodes and edge perturbations is too random, which may cause serious damage to the semantic information of the graph structure. For example, in the functional prediction of molecular structural properties, if edges are randomly perturbed, the structural properties of the molecule will change greatly. In addition, the interpretability that can alleviate popular paranoia and improve model robustness through the above methods is relatively poor. 2) The purpose of existing GCL methods <cit.>, <cit.>, <cit.> to generate multiple views through data augmentation is to maximize the mutual information between views, which may cause the model to capture task-irrelevant feature information. Inspired by the information bottleneck theory <cit.>, we believe that a good GCL method should reduce as much redundant information as possible while retaining as much task-related information as possible.

To tackle the aforementioned issues, we propose a novel method called Contrastive Graph Representation Learning with Adversarial Cross-view
Reconstruction and Information Bottleneck (CGRL) for node classification. CGRL consists of two key components: i.e., adaptive automatic generation of graph-augmented views and graph contrastive learning via information bottlenecks.

First, this paper designs an automatic graph augmentation that adaptively learns node masks and edge perturbations to optimize the original graph into relevant views. In addition, SCGCL employs a joint training strategy to train an adaptive learnable view generator and node classifier in an end-to-end manner, thereby generating augmented views with structural heterogeneity but semantic similarity. As a result, the generated augmented view can undersample the popular nodes in the original graph while retaining the majority of isolated nodes to alleviate the model's popularity bias problem. Intuitively, random masking nodes or perturbed edges do not consider the distribution probabilities and neighborhood information of different types of nodes in the original graph, but dropout them randomly. However, GCN based on message passing is difficult to reconstruct the information of isolated nodes and it is easier to optimize the semantic information of popular nodes. Therefore, the model may achieve better classification results on popular nodes and poor classification results on isolated nodes. The method CGRL proposed in this paper takes the augmented views of debias information and inputs them into GCN for node classification, which improves the model's ability to resist popular bias.

Second, we integrate multiple views that are semantically similar and contain complementary information into a shared feature space for compact representation, which can improve the robustness of CGRL. The intuition behind is that when different views contain complementary semantic information, the model can obtain more prior knowledge to improve the performance of node classification tasks <cit.>. However, we argue that maximizing mutual information between different views forces the model to learn redundant information that is irrelevant to downstream tasks. Inspired by the information bottleneck (IB) theory <cit.>, it obtains optimal solutions by maximizing label information relevant to downstream tasks and minimizing mutual information between different views. Based on the IB strategy, an automatic graph augmenters learns to generate augmented views that remove noise information and contain semantically similar and complementary views. In addition, when calculating the contrastive loss, we not only use the node feature representations of the two augmented views, but also introduce the node feature representations of the original view perturbed in an adversarial manner as a third view. This additional adversarial view introduces perturbations that force the model to not only accurately distinguish the semantic features of the augmented views, but also to maintain an understanding of the semantic integrity of the original graph in the face of perturbations. Through multi-view adversarial reconstruction, we further improve the robustness of the feature representations.

Compared with the previous methods, the contributions of this paper method are summarized as:

Firstly, we propose a Contrastive Graph Representation Learning with Adversarial Cross-view Reconstruction and Information Bottleneck (CGRL) for node classification. The CGRL method can alleviate the popularity bias and interaction noises problem of the existing GNNs in aggregating neighbor node information.

Secondly, The CGRL approach provides a learnable approach to adaptively mask nodes and edges for multiple graph contrastive views in an end-to-end manner. In addition, redundant information irrelevant to node classification is discarded by innovatively introducing information bottleneck theory into multi-view graph contrastive learning.

Thirdly, we introduce a cross-view adversarial reconstruction strategy to further improve the robustness of node feature representation. 

Finally, extensive experiments also show that the proposed CGRL method outperforms the state-of-the-art methods on seven real-world publicly available datasets. 



§ RELATED WORK


Graph Representation Learning   Early work <cit.> have made great progress in representation learning tasks (e.g., node classification, entity alignment, and link prediction, etc). Specifically, GNNs on graph representation learning usually follow the information aggregation mechanism to update the feature representation of nodes, i.e., stimulating connected neighbor nodes to have similar semantic information. Inspired by GNNs, several works (e.g., AROPE <cit.>, DeepWalk <cit.>, and GraphSAGE <cit.>, etc) on graph representation learning usually follows the label propagation mechanism to update the feature representation of nodes, i.e., stimulating connected neighbor nodes to have similar labels. In recent years, GNNs (e.g., GraphSMOTE <cit.>, Nodeformer <cit.>, DisGNN <cit.>, and GraphFL <cit.>, etc) have applied GNN algorithms to learn discriminative latent representations on node classification tasks <cit.>.


Contrastive Learning   Contrastive learning (CL) <cit.>, which is originally widely used in computer vision to obtain better image feature representations, has received extensive research attention in graph learning. Specifically, graph contrastive learning (GCL) learns discriminative node representations by maximizing mutual information (MI) between multiple graph views. For instance, DGI <cit.> learns more discriminative node representations by contrasting node embeddings of local and global graph views. GIC <cit.> maximizes the MI between node clusters with high similarity to make full use of coarse-grained and fine-grained information between nodes. CMC <cit.> maximizes MI by contrasting feature representations from different views. GMI <cit.> estimates and maximizes the MI between the input graph and the feature representation from two aspects of node features and network topology. Some recent self-supervised graph learning (SGLs) methods (e.g., MGAE <cit.>, GraphMAE <cit.>, and GAE <cit.>) generate multiple graph views of nodes and edges and force the consistency between different graph views. On the one hand, all these methods generate multiple contrasting graph views by randomly masking nodes or edges, which may cause some important structural and semantic information to be lost. On the other hand, these methods maximize mutual information between different graph views, which may force the model to learn some task-independent semantic information. To sum up, existing self-supervised graph learning for node classification suffer from insufficient information utilization, i.e., information associated with the label.



Information-Bottleneck Representation Learning   The information bottleneck (IB) theory <cit.> argues that if the feature representation learned by the model from the input data discards information that is not useful for the given task while retaining as much as possible the information relevant to the given task, it will increase the generalization of downstream tasks. Formally, IB needs to construct multiple views for feature representation learning. Motivated by the effectiveness of the IB, several works have considered transferring information bottleneck theory to graph representation learning tasks. For instance, MIB <cit.> designs an unsupervised multi-view method and uses the information bottleneck theory to minimize the representation of redundant information.  DeepIB <cit.> reduces redundant information irrelevant to a given task by minimizing mutual information between multiple views and input data. CMIB <cit.> combines information bottleneck theory to capture the complementarity of intrinsic information between different views and balance the consistency of multi-view latent representations. Nevertheless, almost all the above-mentioned methods aim to obtain a discriminative graph view to replace the original input graph, which may cause some semantic information and topological information of the original graph to be lost. In conclusion, existing information bottleneck methods for node classification suffer from insufficient information utilization, i.e., input graph information and multi-view information.






§ PRELIMINARIES


Notations.
Suppose 𝒢={𝒱, ℰ,ℛ,𝒲} represents a graph where 𝒱={v_1, v_2, …, v_M} is the nodes set, M is the number of nodes, ℰ⊆𝒱×𝒱 represent the edges set, r_ij(r_ij∈ℰ) represents the connection relationship between node i and node j, and ω_ij (ω_ij∈𝒲,0≤ω_ij≤ 1) the weight of the edge ℰ_ij. The feature matrix and degree matrix of nodes are expressed as X= {x_i}_i=1^M amd A={a_ij}∈{0, 1}^M × M, respectively, where x_i represent the features of the node v_i, and if (v_i, v_j) ∈ℰ then a_ij=1 otherwise a_ij=0. 𝒢 = G(x) is regarded as the process of graph processing.

GCN Processing. For input features X ∈ℝ^M × D, a graph 𝒢 = F=G(X) is constructed based on the features X. A GCN layer is utilized to update features representations between nodes by aggregating information from their neighbor nodes. Specially, GCN operates as follows:

    𝒢^'    =F(𝒢, 𝒲) 
        = Update(Aggregate(𝒢, W_agg), W_update) 
        = W_update(∑_r ∈ℛ∑_j ∈ N_i^r1/|𝒩_i^r |(ω_ij W_agg𝒢+ω_ii W_agg𝒢))

where 𝒩_i^r is the set of neighbor nodes of node i under the edge relationship r ∈ℛ, and W_agg and W_update represent the learnable weights of the nodes in aggregating surrounding neighbor nodes information and updating the aggregated features, respectively.

In addition, we also introduce a multi-head attention mechanism in the GNN layer to capture node feature information and topology information in a more fine-grained manner. The feature vectors x_i^” after being aggregated and updated is divided into N heads, i.e., h^1, h^2, …, h^N, and each head is assigned a learnable parameter. Therefore, the feature vectors x is finally updated as follows:

    𝐱_i^'=[h^1 W_update^1, h^2 W_update^2, …, h^N W_update^N],



Information Bottleneck (IB). IB is an information theory-based strategy that describes the information in the data that is relevant to downstream tasks. IB argues that if the obtained feature representation excludes semantic information in the original input that is irrelevant to a given downstream task, it improves the robustness of the model. Specifically, for a given input data x, with associated label information is y, using the IB strategy for model optimization can obtain a compact feature representation z. The optimization goals of IB are as follows:

    max _𝐙 I(𝐘, 𝐙, θ)-β I(𝐗, 𝐙, θ),

where β is an adjustment factor, θ is a learnable parameter.

Combining I(Y,Z) and mutual information theory in Eq. <ref>, we can get:

    I(𝐘, 𝐙)     =∫ d 𝐲 d 𝐳 p(𝐲, 𝐳) logp(𝐲, 𝐳)/p(𝐲) p(𝐳)
        =∫ d 𝐲 d 𝐳 p(𝐲, 𝐳) logp(𝐲|𝐳)/p(𝐲).


However, directly calculating p(y|z) is quite difficult. Inspired by <cit.>, we use q(y|z) as the variational approximation of p(y|z). 

Since the KL divergence is always greater than or equal to 0, we can get:

    𝐊 𝐋[p(𝐲|𝐳), q(𝐲|𝐳)] ≥ 0 ⇒∫ d 𝐲 p(𝐲|𝐳) logp(𝐲|𝐳)/q(𝐲|𝐳)≥ 0 
    ⇒   ∫ d 𝐲 p(𝐲|𝐳) log p(𝐲|𝐳) ≥∫ d 𝐲 p(𝐲|𝐳) log q(𝐲|𝐳).


Therefore we can know:

    I(𝐘, 𝐙)    ≥∫ d 𝐲 d 𝐳 p(𝐲, 𝐳) logq(𝐲|𝐳)/p(𝐲)
        =∫ d 𝐲 d 𝐳 p(𝐲, 𝐳) log q(𝐲|𝐳)+H(𝐲) 
       ≥∫ d 𝐲 d 𝐳 p(𝐲, 𝐳) log q(𝐲|𝐳) 
        =∫ d 𝐲 p(𝐲) ∫ d 𝐳 p(𝐳|𝐲) log q(𝐲|𝐳) .


For I(X,Z) in Eq. <ref>, we can get:

    I(𝐙, 𝐗)   = 	∫ d 𝐳 d 𝐱 p(𝐱, 𝐳) logp(𝐳 , 𝐱)/p(𝐳), p(𝐱)
       = ∫ d 𝐳 d 𝐱 p(𝐱, 𝐳) logp(𝐳|𝐱)/p(𝐳).


Similarly, it is quite difficult to calculate p(z). We use r(z) as the variational approximation to estimate p(z). Since 𝐊 𝐋[p(𝐳), r(𝐳)] ≥ 0 ⇒∫ d 𝐳 p(𝐳)log p(𝐳) ≥∫ d 𝐳 p(𝐳)log r(𝐳), we can get an upper bound:

    [ I(𝐙, 𝐗) ≤∫ d 𝐱 d 𝐳 p(𝐱) p(𝐳|𝐱) logp(𝐳|𝐱)/r(𝐳);     =∫ d 𝐱 p(𝐱) ∫ d 𝐳 p(𝐳|𝐱) logp(𝐳|𝐱)/r(𝐳) . ]


Combining the above analysis and inequalities, we can obtain the lower bound of the information bottleneck theory:

    I(𝐘, 𝐙)-∑_v=1^V β I(𝐙, 𝐗) 
       ≥∫ d 𝐲 p(𝐲) ∫ d 𝐳 p(𝐳|𝐲) log q(𝐲|𝐲) 
        - β∫ d 𝐱 p(𝐱) ∫ d 𝐳 p(𝐳|𝐱) logp(𝐳|𝐱)/r(𝐳).




§ METHODOLOGY

In this section, we introduce our proposed Contrastive Graph Representation Learning with Adversarial Cross-view Reconstruction and Information Bottleneck (CGRL) method in detail.



 §.§ Automatically Generated Multi-view Augmentation


Adversarial View. By adding perturbed adversarial examples to the original image views, the robustness of the model is significantly enhanced. This improvement can be attributed to a possible explanation that there is still non-predictive redundant information in the information shared between the two augmented views. When adversarial examples are introduced, this redundant information is weakened or eliminated, forcing the model to focus on more important and discriminative features. We define adversarial view as follows:

    𝒢_adv^(l)=𝒢 + δ^*


    δ^*   =δ_∞⩽ϵargmaxℒ_adv(𝒢_E D,𝒢_N D,𝒢+δ) 
       =δ_∞⩽ϵargmaxmax_δ^*[ℒ_CL(𝒢_N D,G+δ^*) .
       .+ℒ_CL(𝒢_E D,G+δ^*)]

where 𝒢_adv^(l) = {𝒱', ℰ',ℛ',𝒲'}, 𝒱={v_1, v_2, …, v_M}, δ is the randomly initialized Gaussian noise, ℒ_CL is the infoNCE loss, ϵ is the radius. Inspired by recent work <cit.>, we add a perturbation δ to the output of the first hidden layer. It has been empirically shown that it can more effectively perturb the intermediate representation of the model than adding perturbations to the initial node features, allowing the model to learn and predict in more complex environments.

Node-Masking View.
As shown in Fig. 1, both the category of nodes and the degree of nodes in citation data show data imbalance, which hinders GCN from learning the feature representation of minority class nodes. Therefore, we perform automatic learnable node masking before each information aggregation and feature update of GCN to alleviate the shielding effect of influential nodes on minority class nodes. The node-masking view we created is formulated as follows:

    𝒢_N D^(l)={{v_i' ⊙η_i^(l)| v_i' ∈𝒱'}, ℰ', ℛ', 𝒲'}

where η_i^(l)∈{0,1} is sampled from a parameterized Bernoulli distribution Bern(ω_i^l), and η_i^(l) = 0 represents masking node v_i, η_i^(l) = 1 represents keeping node v_i.

Randomly removing some nodes and their connections in the graph may result in a large loss of minority class node information, thereby affecting the information aggregation of minority class nodes and leading to unsatisfactory classification results. Therefore, instead of directly removing the selected nodes from the graph, we replace the selected nodes by sampling the representation of the local subgraph using a random walk strategy to obtain a local representation of the node.

Edge Perturbation View.
The goal of perturbing edges is to generate an optimized graph structure that filters noisy edges and alleviates the problem of popularity bias. The edge perturbation view is formulated as follows:

    𝒢_E D^(l)={𝒱',{e_i j' ⊙η_i j^(l)| e_i j' ∈ℰ', ℛ', 𝒲'}}

where η_
ij^(l)∈{0,1} is also sampled from a parameterized Bernoulli distribution Bern(ω_ij^l), and η_ij^(l) = 0 represents perturbating edges e_ij, η_i^(l) = 1 represents keeping edge e_ij.

To enable the model to automatically learn whether to mask nodes and perturb edges, we formally define the learnable parameter ω^l_i and ω^l_ij as follows:

    ω_i^(l)=Linear(𝐞_i'^(l)) ;   ω_i j^(l)=Linear([𝐞_i'^(l) ; 𝐞_j'^(l)])


To ensure that the model can automatically optimize and generate augmented multi-views in an end-to-end learning method, we use reparameterization technology <cit.> to convert the discretized η into a continuous function. The formula is defined as follows:

    η_i=exp((log(π_i)+g_i) / τ)/∑_j=1^m exp((log(π_j)+g_j) / τ),    for  i=1, …, m

where g_i=-log(-log(ϵ_i)), ϵ_i ∼ Uniform(0, 1), τ∈ℝ^+ means annealing temperature, τ represents the class probability, and m represents number of categories.

After obtaining the masked node and edge perturbed views, we input them into GCN for feature representation to obtain optimized multi-views. The formula is defined as follows:

    𝐄_N D^(l)=GraphConv(𝐄_N D^(l-1), 𝒢_N D^(l)) 
       𝐄_E D^(l)=GraphConv(𝐄_E D^(l-1), 𝒢_E D^(l))

where GraphConv represents the graph convolution operation, and we choose GAT as our graph encoder. 𝐄_N D and 𝐄_E D represent the node feature representations of node-masking view and edge perturbation view respectively, 𝐆_N D and 𝐆_E D represent node-masking view and edge perturbation view respectively.



 §.§ Contrastive Learning via IB

Although CGRL combines an automatic learnable view augmentation and a node classification process for model optimization, we argue that relying solely on the classification objective does not well guide the node masking and edge perturbation process to create optimal multi-views. Therefore, we follow the information bottleneck strategy <cit.> to retain sufficient semantic information relevant to downstream tasks in the augmented node mask and edge perturbation views. Specifically, unlike traditional CL strategies, we encourage maintaining topological heterogeneity between the augmented view and the original graph while maximizing the information relevant to the node classification task. Based on the above strategy, we can obtain topologically heterogeneous but semantically similar enhanced multi-view representations and effectively remove noise information in the graph. Therefore, the objective in Eq. <ref> is summarized as:

    min_(E, Ẽ)ℒ_CLS + I(𝐄, 𝐄̃) 
        = -∑_i=1^ny_ilog(ŷ) + I(𝐄, 𝐄̃)

where L_CLS represents the cross-entropy loss, I(𝐄, 𝐄̃) represents the mutual information between the original view and the augmented view.

Following <cit.>, minimizing the lower bound of mutual information (i.e., Eq. 9) is equivalent to maximizing the InfoNCE loss <cit.>. Therefore, we adopt negative InfoNCE to optimize the feature representation between the augmented view and the original graph. Specifically, we treat the same node representations in the original graph and the automatically generated view as positive pairs (i.e., {(e_i', ẽ_i')|v_i' ∈𝒱'}), and different node representations as negative pairs (i.e., {(e_i', ẽ_j')|v_i', v_j' ∈𝒱'}, i ≠ j).

    I(𝐄, 𝐄̃)     = ∫ d 𝐲 p(𝐲) ∫ d 𝐳 p(𝐳|𝐲) log q(𝐲|𝐲) 
        - β∫ d 𝐱 p(𝐱) ∫ d 𝐳 p(𝐳|𝐱) logp(𝐳|𝐱)/r(𝐳)
        = ∑_v_i' ∈𝒱'logexp(sim(𝐞_i', 𝐞̃_i') / τ)/∑_v_j ∈𝒱exp(sim(𝐞_i', 𝐞̃_j') / τ).

where s(·) is used to measure the similarity between positive and negative sample pairs.



 §.§ Adversarial Cross-view Reconstruction

To further achieve feature disentanglement, we propose a cross-view reconstruction mechanism. Specifically, we hope that the representation pairs within and across enhanced views can recover the original data. More specifically, we define (𝐄_ND, 𝐄_ED), (𝐄, 𝐄_ED), (𝐄, 𝐄_ED) as a cross-view representation pair, and repeat the reconstruction process on it to predict the original view, aiming to ensure that 𝐄_ND, 𝐄_ED, 𝐄 are optimized to approximately disentangle each other. Intuitively, the reconstruction process is able to separate the information of the shared feature set from the information in the unique feature set between the two enhanced views. We formally define the reconstruction process as:

    ℒ_recon=   1/2N[𝐄-𝐄_ND_2^2+𝐄-𝐄_ED_2^2]

wherer N represents the number of nodes.



 §.§ Model Training

We train the model with the goal of optimizing the node mask view, edge perturbation view, and node classification:

    ℒ   =ℒ_CLS+αℒ_CLS ^N D
    		+(1-αℒ_CLS ^E D)
       +β(I(𝐄, 𝐄_N D)+I(𝐄, 𝐄_E D))
    		+ℒ_recon+λΘ_2^2

where λ and β represent the learning weights of L2 regularization and information bottleneck contrast learning respectively.




§ THEORETIC ANALYSIS
 
Theorem 1. Specifically, we regard the augmented view as 𝒢̃, the noisy view as 𝒢^', and the node label information as Y_CLS. We assume that 𝒢̂ is not related to the classification information Y_CLS. Therefore, the upper bound of I(𝒢̂ ; 𝒢̃) is defined as follows:

    I(𝒢̂ , 𝒢̃) ≤ I(𝒢 , 𝒢̃)-I(Y_CLS , 𝒢̃)


Proof. We assume that the clean graph 𝒢 is defined by 𝒢^' and label information Y, and we can get (𝒢^', Y_CLS) →𝒢→𝒢̃ according to the Markov chain <cit.>. Therefore, we can get:

    I(𝒢 , 𝒢̃)    ≥ I((Y_CLS, 𝒢̂) , 𝒢̃) 
        =I(𝒢̂ , 𝒢̃)+I(Y_CLS , 𝒢̃|𝒢̂) 
        =I(𝒢̂, 𝒢̃)+H(Y_CLS|𝒢̂)-H(Y_CLS|𝒢̂, 𝒢̃)


Because there is no correlation between 𝒢̃ and Y_CLS, H(Y_CLS|𝒢̃) = H(Y_CLS). Furthermore, H(Y_CLS|𝒢̂, 𝒢̃) ≤ H(Y_CLS). Therefore, we simplify the Eq. <ref> as follows:

    I(𝒢 , 𝒢̃)    ≥  I(𝒢̂ , 𝒢̃)+H(Y_CLS)-H(Y_CLS|𝒢̃) 
       =   I(𝒢̂, 𝒢̃)+I(Y_CLS , 𝒢̃)

Therefore, we prove that Eq. <ref> holds. In summary, we provide a theoretical basis to ensure that graph contrastive learning via information bottlenecks can achieve noise invariance by reducing redundant and interfering information in augmented views.

Theorem 2. We optimize the reconstruction by minimizing the entropy H(𝐄|𝐄_ND, 𝐄_ED). Ideally, when H(𝐄|𝐄_ND, 𝐄_ED) - 𝔼_𝐄, 𝐄_ND, 𝐄_ED[log p (𝐄|𝐄_ND, 𝐄_ED)]=0, we achieve the best feature disentanglement. However, in practice, the estimation of conditional probability p(𝐄|𝐄_ND, 𝐄_ED) is very tricky and complicated. Therefore, we use an approximate variational distribution q(𝐄|𝐄_ND, 𝐄_ED) to simplify the calculation and optimization process. We provide a theoretical upper bound on H(𝐄|𝐄_ND, 𝐄_ED) as follows:

    H(𝐄|𝐄_ND, 𝐄_ED)⩽max{𝐄-𝐄_ND_2^2, 𝐄-𝐄_ED_2^2}


Proof. For a given original view 𝐄 and two augmented views 𝐄_ND and 𝐄_ED, we have:

    p(𝐄_ND,𝐄_ED)   =p(𝐄_ND)p(𝐄_ED)
    p(𝐄_ND,𝐄_ED|𝐄)   =p(𝐄_ND|𝐄)p(𝐄_ED|𝐄)


Lemma 1. For three given random variables a, b, c, if p(b, c) = p(b)p(c) and p(b, c| a) = p(b| a)p(c| a), then I(a, b| c) = I(a, b). Based on the definition of mutual information, we deduce:

    I(a;b| c)= 
       =∑_a∑_b∑_cp(a,b,c)logp(a,b,c)p(c)/p(a,c)p(b,c)
       =∑_a∑_b∑_cp(a)p(b,c| a)logp(b,c| a)p(c)/p(c| a)p(b)p(c)
       =∑_a∑_b∑_cp(a)p(b| a)p(c| a)logp(b| a)p(c| a)/p(c| a)p(b)
       =∑_a∑_bp(a)p(b| a)logp(b| a)/p(b)
       =∑_a∑_bp(a,b)logp(b| a)/p(b)
       =I(a;b)


According to Lemma 1, we can derive the theoretical bound of I(𝐄;𝐄_ND,𝐄_ED) as follows:

    I(𝐄;𝐄_ND,𝐄_ED)
    		
        =I(𝐄;𝐄_ND|𝐄_ED)+I(𝐄;𝐄_ND)+I(𝐄;𝐄_ED) 
       ⩾ I (𝐄_ND, 𝐄_ED;𝐄) 
       =I(𝐄_ND;𝐄)+I (𝐄_ED;𝐄) 
       ⩾ I(𝐄_ND;𝐄_ED)


Assuming q is a Gaussian distribution, the reconstruction process can be equivalent to minimizing the information entropy and its theoretical upper bound is formally defined as follows:

    H(𝐄|𝐄_ND, 𝐄_ED)⩽max{𝐄-𝐄_ND_2^2, 𝐄-𝐄_ED_2^2}


Proof. The estimation of conditional probability p(𝐄|𝐄_ND, 𝐄_ED) is very tricky and complicated. Therefore, we use an approximate variational distribution q(𝐄|𝐄_ND, 𝐄_ED) to simplify the calculation and optimization process. Therefore, we have,

    H(𝐄|𝐄_ND, 𝐄_ED)
       =-𝔼_p(𝐄|𝐄_ND, 𝐄_ED)[log p(𝐄|𝐄_ND, 𝐄_ED)] 
       ≤-𝔼_p(𝐄|𝐄_ND, 𝐄_ED)[log q(𝐄|𝐄_ND, 𝐄_ED)] 
       - D_KL(p(𝐄|𝐄_ND, 𝐄_ED)q(𝐄|𝐄_ND, 𝐄_ED))


Assume q(𝐄|𝐄_ND, 𝐄_ED) is a Gaussian distribution 𝒩(𝐄|𝐄_ND, 𝐄_ED,σ^2𝐈):

    H(𝐄|𝐄_ND, 𝐄_ED)
       ⩽-𝔼_p(𝐄|𝐄_ND, 𝐄_ED)[log q(𝐄|𝐄_ND, 𝐄_ED)] 
       =-𝔼_p(𝐄|𝐄_ND, 𝐄_ED)[log(1/√(2π I)σe^-1/2(𝐄-𝐄_{ND,ED})^2/(σ^2𝐈))] 
       =-𝔼_p(𝐄|𝐄_ND, 𝐄_ED)[log(1/√(2π I)σ)-(𝐄-𝐄_{ND,ED})^2/2σ^2𝐈]

Therefore, we get the upper bound of H(𝐄|𝐄_ND, 𝐄_ED).



§ EXPERIMENTS





 §.§ Experimental Setup


Datasets Description   In our experiments, seven publicly available benchmark datasets are used including two Amazon items datasets <cit.> (i.e., Computers, and Photo), five citation network datasets <cit.> (i.e., Citeseer, Pubmed, DBLP, CoraFull, and Cora), three large-scale datasets <cit.> (i.e., Ogbn-arxiv, Ogbn-mag, and Ogbn-products), page network datasets <cit.> (i.e., Wiki-CS and Croco).







Evaluation Metrics   We use classification accuracy to evaluate the performance of our method CGRL and other comparative methods.

Comparison Methods   We compare our method CGRL with twelve state-of-the-art deep learning-based algorithms, including two traditional graph embedding algorithms (i.e., raw features <cit.>, and DeepWalk <cit.>), three semi-supervised algorithms (i.e., GCN <cit.>, NIGCN<cit.>, and GAT <cit.>), and nine self-supervised algorithms (i.e., GAE <cit.>, VGAE <cit.>, DGI <cit.>, GCA <cit.>, MVGRL <cit.>, GIC <cit.>, GRACE <cit.>, GMI <cit.>, and CRLC <cit.>).


Setting-up 
All the experiments in this paper are implemented on a server with 2 A100 (total 160GB memory). For each experiment, we run the code five times with a random seed to obtain the final mean and corresponding standard deviation to avoid experimental chance. Futhermore, for some parameter settings of the model, we set epochs to 1000/300, batch-size to full-batch/mini-batch, learning rate to 0.005, mask rate to 0.5, α to 0.5, β is 0.5, the activation function to GELU, dropout is 0.2, the weight decay to 1e-4, learning rate scheduling to cosine, warmup epochs to 100, and hidden_size to 128. We utilize the Adam optimization algorithm to update parameters.



 §.§ Results and Analysis






Node classification.
Tables 1 and 2 summarize the node classification accuracy of the baselines and the proposed method CGRL on twelve real graph-structured data sets. Specifically, our approach leverages traditional graph embedding algorithms (i.e., raw features and DeepWalk). For example, our method CGRL improves the average accuracy by 20.31% and 10.76% compared to raw features and DeepWalk methods respectively. Compared with self-supervised methods (e, g., DGI, and GCA, etc.), CGRL also achieves better performance. In addition, CGRL also outperforms semi-supervised algorithms (i.e., GCN , NIGCN, and GAT). The performance improvement may be attributed to the design of the multi-view contrast learning strategy for adaptive masking nodes and perturbed edges, which enables the model to automatically learn whether to mask nodes and perturbed edges. To ensure that our model can reconstruct nodes and edges, we use a random walk strategy to sample the node's subgraph structure to blur its representation. However, most other baselines perform multi-view contrastive learning by randomly dropping nodes and edges, which will seriously destroy the semantic information of the graph. In addition, we also introduce the information bottleneck theory to ensure that the augmented views are structurally heterogeneous but semantically similar. The intuition behind this is that maximizing the mutual information between multiple views will lead to a consistent representation of the augmented views learned by the model, which leads to overfitting of the model. 



Effect of noise. In order to verify the anti-interference ability of our model CGRL, we perform experiments to demonstrate the performance by varying the noise rate in {0, 0.1, 0.2, 0.3, 0.4, 0.5} and the β in {0.1, 0.2, 0.3, 0.4, 0.5
,0.6, 0.7, 0.8, 0.9}. The experimental results are shown in Fig. <ref>, We can find that when the parameter β is 0, CGRL has the worst effect on the Cora, Citeseer and PubMed data sets. When β is greater than 0, CGRL's anti-interference ability is significantly reflected, and the best effect is in the range of 0.2 to 0.6. The experimental results show that the introduction of information bottleneck theory can enhance the noise invariance ability of the model.


Hyper-parameter analysis.
We set two hyperparameters α and β during the model optimization process. Their settings will have a relatively large impact on the performance of the model. Therefore, we also investigated the impact of hyperparameters on CGRL. We performed the node classification task by varying α and β from 0.1 to 0.9 and visualized the accuracy in Fig. <ref>. When α and β are set to relatively large values, the node classification effect of the model is better, and when set to smaller values, the performance is poor. The difference in performance can be attributed to the inability of smaller parameter settings to eliminate redundant information in the graph structure and to obtain optimal augmented multi-view.




 §.§ Ablation Study

We performed three ablation experiments to verify the effectiveness of our proposed node-masking view, edge perturbation view, and information bottleneck theory.





  §.§.§ Effectiveness of adaptive graph contrastive learning via IB.


We perform ablation studies to analyze the impact of node-masking view, edge perturbation view, and IB criterion on experimental results. The experimental results are shown in Table <ref>. Firstly, when CGRL includes all modules, the accuracy of node classification on seven data sets is the highest. Secondly, We find that the node-masking view has a greater impact on the experimental results than the edge perturbation view. Experimental phenomena show that node classification mainly depends on the features of nodes. Thirdly, node-mask view and edge perturbation view combined with IB accuracy can further improve the accuracy of node classification. The performance improvement may be attributed that the IB criterion can promote multi-views to obtain structurally heterogeneous but semantically similar graph structures.



  §.§.§ Impact of GNN variants.
 
We explore the impact of different graph neural network variants on experimental results. As shown in Table <ref>, GAT has the best effect on seven graph structure data sets, with node classification accuracy rates of 86.3%, 75.4% and 84.5% on the Cora, Citeseer, and PubMed data sets, respectively, followed by GraphSAGE, with node classification accuracy rates of 83.6%, 72.8% and 82.0% , respectively, and GCN has the worst effect, with node classification accuracy rates of 82.7%, 70.7% and 81.4%, respectively. Therefore, we choose GAT as our encoder in our experiments.



  §.§.§ Effectiveness of multi-view optimization.

We perform ablation experiments to verify the impact of multi-view augmentation on the optimization process of model training. The experimental results are shown in Fig. <ref>. On Cora, Citeseer, and PubMed datasets, we found that in the absence of node-masking view or edge perturbation view, the loss value of the model cannot converge to the optimal value, and the edge perturbation view has the worst convergence effect. When CGRL combines the node-masking view, edge perturbation view and information bottleneck criterion, the loss value of the model can converge to close to 0. The experimental results show that the performance of the augmented node-masking view is better than the augmented edge perturbation view, but worse than the augmented multi-view that combines information bottlenecks, which shows that the information bottleneck theory can further enhance the utilization of information.



  §.§.§ Visualization.

We use the T-SNE method to project the high-dimensional node features into a 2-dimensional space and visualize them. The visualization results are shown in Figs. <ref> and <ref>. In the Cora data set, GCN and GAT have more overlaps between different node categories and the class boundaries are not clear enough, while CGRL has clearer class boundaries between different node categories. In the Citeseer data set, the feature embeddings of GCN and GAT are more scattered among the same node categories, and there is excessive overlap between different categories, while CGRL is more densely distributed for the same category of nodes and there is less overlap between different categories.










§ CONCLUSIONS

In this paper, we propose a Contrastive Graph Representation Learning with Adversarial Cross-view Reconstruction and Information Bottleneck for node classification to automatically generate structurally heterogeneous but semantically similar multi-views. Specifically, CGRL can adaptively learn to mask nodes and perturb edges in the graph to obtain optimal graph structure representation. Furthermore, we innovatively introduce the information bottleneck theory into GCL to eliminate redundant information in multiple contrasting views while retaining as much information about node classification as possible. Moreover, we add noise perturbations to the original views and reconstruct the augmented views by constructing adversarial views to improve the robustness of node feature representation. Extensive experiments on real-world public datasets show that our approach significantly outperforms existing the SOTA algorithms.






elsarticle-num


















 Paper Title: TrackSorter: A Transformer-based sorting algorithm for track finding in High Energy Physics.






^1University of California-Berkeley
^2Lawrence Berkeley National Laboratory
yashmelkani02@gmail.com, xju@lbl.gov

§ ABSTRACT 
Track finding in particle data is a challenging pattern recognition problem in High Energy Physics. It takes as inputs a point cloud of space points and labels them so that space points created by the same particle have the same label. The list of space points with the same label is a track candidate. We argue that this pattern recognition problem can be formulated as a sorting problem of which the inputs are a list of space points sorted by their distances away from the collision points and the outputs are the  space points sorted by their labels. In this paper, we propose the algorithm: a Transformer-based algorithm for  pattern recognition in particle data. uses a simple tokenization scheme to convert space points into discrete tokens. It then uses the tokenized space points as inputs and sorts the input tokens into track candidates. is a novel end-to-end track finding algorithm that leverages Transformer-based models to solve pattern recognition problems. It is evaluated on the TrackML dataset and has good track finding performance.





§ INTRODUCTION


The High Luminosity Large Hadronic Collider (HL-LHC) plans to collide two proton beams at the unprecedented center of mass energy of 14 TeV at an instantaneous luminosity of up to 7.5×10^34cm^-2s^-1. That corresponds to an average number of proton-proton collisions per beam crossing (i.e. pileup), ⟨μ⟩, of up to 200. HL-LHC brings opportunities and challenges. To cope with the challenges, the ATLAS <cit.> and CMS <cit.> experiments decided to build a new fully silicon-based inner tracker detector <cit.>. The new inner trackers will have better raditation tolerance, increased granularity, reduced material, and large readout bandwith to fulfill the requirement of the HL-LHC Runs. Take the ATLAS's new innder detector, ITk, as an example. ITk consists of a Pixel detector at a small radius and a large area Strip detector surrounding it. The Pixel detector consists of about five billion finely segmented silicon sensors, most of which have a pitch of 50×50 μm^2 and the rest 25×100 μm^2. The Strip detector comprises 23,000 long and skinny silicon sensors (75.5 μm× 24.1 or 48.2 mm). Each event with ⟨μ⟩=200 produces about 300,000 space points, out of them only about 10,000 space points come from particles of interest. Finding the tracks of interests from a point cloud of space points is a challenging pattern recognition problem. 

Our work is inspired by the remarkable capabilities of Large Language Models (LLMs) such as BERT <cit.>, GPT <cit.>, Llama <cit.>, and grok-1 <cit.>. The foundation of our work is to convert space points into discrete token ids (i.e. tokenization). Tokenization is a critical step in efficient learning and handling out-of-vocabulary words for LLMs learning natural language. For example, one can use letters as tokens for the English language. Doing so would result in a small vocabulary and can construct all out-of-vocabulary words. However, it is inefficient for learning because semantic relationships among letters are lost during tokenization. 
Tokenizing a point cloud of measurements in High Energy Physics (HEP) presents a unique challenge: converting variables from a continuous, multi-dimensional space into discrete spaces. Although some information from the continuous space will inevitably be lost during tokenization, this loss may be acceptable as long as it does not compromise the accuracy of the underlying physics. After all, all physics measurements inherently contain some level of uncertainties.
In the context of jet physics, the Omnijet framework  <cit.> explored three schemes for tokenizing jet constituents: physics-inspired binning of contitunents' kinematic variables (binning in short), conditional and unconditional tokenization via the vector-quantization variational autoencoder (i.e. VQ-VAE) <cit.> technique, which is also used in Ref <cit.> to build the codebook index. The binning scheme adjusts the bin width to match measurement uncertainties and the study shows a small bias in the reconstructed jet mass and poor resolution (Fig. 4 in Ref. <cit.>). The VQ-VAE schemes enjoy better performances with a larger number of tokens. In our previous research <cit.> on particle tracking data, we utilized the unique detector module IDs as the token identifiers for space points created on that detector module. While this tokenization process results in the loss of precise positional and other details of the space points, it enables us to directly train LLMs with the tracking data.

We argue that the track finding problem can be formulated as a sequence-to-sequence (seq2seq) problem, illustrated in Fig. <ref>. The input sequence is a list of space points ordered by their distances away from the collision point. And the output sequence is a list of the same space points ordered by their labels and their distances away from the collision point. Language models like Bart <cit.> are very good at solving seq2seq problems like machine translation, text summary, and question-answering. The Bart model consists of a bidirectional encoder (like BERT) and a left-to-right decoder (like GPT). A similar model is used in our work.






§ DATA


This study is based on the TrackML dataset <cit.>, which simulates the top quark pair production from proton-proton collisions at the HL-LHC. To simulate the effect of event pileup and produce realistic detector occupancy, a Poisson random number (with μ=200) of Quantum Chromodynamics "minimum bias" events are overlaid on top of the tt̅ collisions. The TrackML detector is a set of concentric cylindrical layers of pixelated sensors (the barrel) complemented by a set of circular disks (the endcaps) to ensure nearly 4π coverage in solid angle, as pictured in Fig. <ref>.

The detector is divided into nine volumes, each consisting of 2 to 7 layers. Each layer contains multiple silicon modules. There are 18,737 detector modules in the TrackML dataset. We use data from volume 8, 13, and 17, summing up to 14,000 modules. We introduce two custom tokens to indicate the start of the output sequence [SOS] and the end of each track . Therefore, as detector module IDs are treated as token identifiers, the vocabulary size in our work is the sum of the number of detector modules and the two special tokens; that's 14,002.  

 

Our study utilizes particles that have space points from at least 6 unique layers. Our training dataset uses tracks from 100 events that meet this condition, totaling 349k tracks. The validation dataset is similarly constructed from 10 events (35k tracks). A testing dataset for performance analysis is curated with an additional condition that each track has an average < 5 GeV, containing 67k tracks.

In natural language processing workflows, discrete tokens are first embedded into a continuous, dense vector representation, i.e. Word2Vec <cit.>. We used the continuous bag of words framework <cit.> to train a Word2Vec model by asking the model to predict a target word using all the words in a context window. To train the model, we randomly pair each track with another track and construct a target sentence following the ordering scheme shown in Fig. <ref> for each track pair. In total, our training data contains 349k sentences and 8M tokens. The “Generate Similar` (Gensim) library <cit.> is utilized for training. To achieve a reasonable embedding performance, the model uses an embedding dimension of 64, a context window of 20 tokens, and is trained for 100 epochs. In the future, we can use the TrackingBERT <cit.> method to embed detector modules.



§ MODEL AND TRAINING


The model utilizes the encoder-decoder structure of transformers <cit.>. Both encoder and decoder networks are composed of a stack of identical transformer modules, each having a single-head self-attention mechanism and a position-wise fully connected feed-forward network. We only use a single attention head because our embedding dimension is 64, which is relatively small. And the feed forward layers have a dimension of 256. The output of the decoder network is fed into a linear layer that spans the dimension of the vocabulary. Our model contains six bi-directional encoder layers followed by six left-to-right decoder layers, totaling 1.6M trainable parameters. We inject positional encoding into the input sequence to provide information on the order of the detector modules. 

The model is trained to autoregressively predict the correct sequence of tokens for 371 epochs using the Adam optimizer <cit.> in conjunction with the CosineAnnealingLR scheduler. Model weights corresponding to the lowest validation loss were saved.  




§ RESULTS AND DISCUSSIONS


During model inference, we utilize the greedy search algorithm to construct model predictions: given an input sequence, a count mask is created to store the number of instances of each token in the input sequence. The count value for the [SOS] and tokens are set to 0 and 100, respectively. The model is first fed with the [SOS] token and predicts the next token by calculating logits for each token in the vocabulary. The logits of tokens that have a value of zero in the count mask are set to zero. The token with the highest model logit is considered as the next token; thus, it is appended to the output sequence. Its corresponding count value is decremented by one so that it would not appear again in the output sequence. The updated output sequence is then fed back to the model for the next token prediction. In the case that the greedy algorithm predicts the token, the model logit of the token will be set to zero in the next step. 
This is to prevent the algorithm from predicting the token in consecutive steps. 
Note that after predicting the token, the model will decide what the next token will be. That means the model may predict track candidates in an arbitrary order.
Such predictions are repeated until all input tokens are in the output sequence and the last predicted token is the token. This termination condition is not ideal when noise space points [Noise space points are those created either from electronic noises or low-particles (i.e. < 200 MeV).] are presented as in real data. 

The model performance is evaluated by the tracking reconstruction efficiency, defined as the fraction of particles that are matched to at least one reconstructed track. Reconstructed track candidates are  matched to particles if (1) 75% of module hits in the reconstructed track are in the true particle track and (2) 75% of module hits in the true particle track are in the reconstructed track. To assess our model, each track in our testing dataset was randomly paired with another track before being passed to the model. Fig. <ref> shows the model's performance with respect to track length and track . 
The model performance is fairly stable against the track length, indicating the model can catch long-distance information. 
We note that tracking efficiency as a function of resembles the distribution of particle in our testing dataset, see figures on the top panel in Fig. <ref>. This suggests that an uniform sampling of particle during training may make model performant in all regions.  




We observed that a larger model size leads to better performance. We expect a larger dataset size to be beneficial to improve model performance. In our training dataset, we find several tokens represented in only a handful of samples. This adversely affects Word2Vec training of our initial embedding vectors as well as training of the model itself.

In smaller scale experiments focusing on the inner barrel detector region, we came up with physics-inspired embedding vectors for detector modules, such as the global coordinates, rotation matrix and pitch components, and geometric features of each detector module. The model performance resulting from this embedding scheme does not perform well compared to the more traditional Word2Vec implementation. 


The model works fairly well with two tracks per input. but it remains to be studied whether the can scale effectively in a dense environment like the HL-LHC, where each event contains 10k particles resulting in 100k detector hits. 
This would form the event-level context window for the language model. 
This large context window may not pose a problem, as the leading LLM models already have substantial capacities. 
For instance, GPT-4o supports a 128k context window <cit.>, 
Claude 3 Sonnet extends up to 200k tokens <cit.>, 
and the Gemini-1.5 model can handle up to 1 million tokens in production <cit.>. 



§ CONCLUSIONS


We reformulated the particle tracking problem as a sequence to sequence problem and proposed a Tranformer-based track sorting algorithm to address it. This algorithm achieves good tracking reconstruction efficiency, even for low-particles (< 1 GeV). Our work leverages a language model-style architecture to tackle high-energy problems. Our study trained the language model from scratch. It remains to be studied open large language models can be fine-tuned for HEP problem-solving.



§ DATA AND SOFTWARE AVAILABILITY

Data can be found at Kaggle [https://www.kaggle.com/c/trackml-particle-identification] and code is avaible at Github [https://github.com/YashMelkani/Track-Sorting-Tutorial].



§ ACKNOWLEDGEMENTS

This research used resources of the National Energy Research Scientific Computing Center (NERSC), a U.S. Department of Energy Office of Science User Facility located at Lawrence Berkeley National Laboratory, operated under Contract No. DE-AC02-05CH11231.



jhep



