# Knowledge-Aware Parsimony Learning: A Perspective from Relational Graphs

Quanming Yao  Yongqi Zhang Yaqing Wang Nan Yin James Kwok Qiang Yang Q.Yao is
an Assistant Professor in the Department of Electronic Engineering at the
Tsinghua University. Y.Zhang is an Assistant Professor in the Department of
Data Science and Analytics Thrust at the Hong Kong University of Science and
Technology (Guangzhou). Y.Wang is the staff researcher in Baidu Research.
N.Yin is a Research Associate in the Department of Computer Science and
Engineering at the Hong Kong University of Science and Technology. J.Kwok and
Q.Yang are Professors in the Department of Computer Science and Engineering at
the Hong Kong University of Science and Technology.  Correspondence is to
Q.Yao at<EMAIL_ADDRESS>
###### Abstract

[Abstract] The scaling law, a strategy that involves the brute-force scaling
of the training dataset and learnable parameters, has become a prevalent
approach for developing stronger learning models. In this paper, we examine
its rationale in terms of learning from relational graphs. We demonstrate that
directly adhering to such a scaling law does not necessarily yield stronger
models due to architectural incompatibility and representation bottlenecks. To
tackle this challenge, we propose a novel framework for learning from
relational graphs via knowledge-aware parsimony learning. Our method draws
inspiration from the duality between data and knowledge inherent in these
graphs. Specifically, we first extract knowledge (like symbolic logic and
physical laws) during the learning process, and then apply combinatorial
generalization to the task at hand. This extracted knowledge serves as the
“building blocks” for achieving parsimony learning. By applying this
philosophy to architecture, parameters, and inference, we can effectively
achieve versatile, sample-efficient, and interpretable learning. Experimental
results show that our proposed framework surpasses methods that strictly
follow the traditional scaling-up roadmap. This highlights the importance of
incorporating knowledge in the development of next-generation learning
technologies.

###### keywords:

parsimony learning, graph learning, few-shot learning, meta-learning,
automated machine learning, neural architecture search, drug development

##  1 Introduction

The learning techniques have progressed from manual feature engineering to
shallow models, then to deep networks, and now to foundation models (Bommasani
et al., 2021), achieving great success in the field of computer vision (He et
al., 2016), natural language understanding (Brown et al., 2020) and speech
processing (Rabiner and Juang, 1993). Specifically, large language model, like
ChatGPT (Ouyang et al., 2022), as representatives of foundation model, has
shown strong performance in versatile learning, which can adopted in many
different tasks. The belief is that larger models can be more expressive, thus
are likely to generalize better given sufficient training data (Jordan and
Mitchell, 2015; Donoho, 2017). This gives birth to the current roadmap, i.e.,
achieving stronger performance by aggressively scaling up the size of data and
models, which is also observed as scaling law (Kaplan et al., 2020).

Figure 1: The applications with relational graphs (left), and failure of
classical scaling law on tasks with relational graphs (right).

Different from the data form of images, natural language and speech,
relational graphs are a way of representing knowledge using nodes and edges,
while also capturing the meaning of that knowledge in a structured form that
can be used for machine learning. The application of graphs are widely spread
in real-life scenarios. As shown in the left of Figure 1, examples include
urban network design and planning (Porta et al., 2006), prediction of
molecular properties (Dai et al., 2016), reasoning from knowledge graphs
(Liang et al., 2018), and recommendation systems (Ying et al., 2018). To meet
the learning demands of diverse application mentioned above, we want to
develop the methods that can efficiently train on different tasks with a
unified architecture (versatile), fast adapt to new tasks with few samples
(sample-efficient), and exhibit inference process with evidence
(interpretable).

However, simply following the previous technical roadmap would not face the
requirement for relational graphs. The roadmap would first fail on versatile
learning, the reason can be attributed to the specialness of learning from
such a data type. The primary problem is the architectural incompatibility.
LLMs are implemented through Transformer architectures Vaswani et al. (2017),
achieving superior representation capabilities by scaling up model and data
size, thereby facilitating versatile learning across diverse tasks. In
contrast, Transformer architectures are inherently designed for sequential
structures and thus struggle with relational graphs. The limitations of
positional encoding (Vaswani et al., 2017) and scalability issues (Liu et al.,
2024) in handling graph structures hinder their effectiveness in versatile
learning. Besides, the learning methods for relational graphs would also
suffer from the representation bottleneck when layers are stacked up (Nt and
Maehara, 2019; Topping et al., 2021). Limited by the models’ learning ability,
simply stacking up the model layers would lead to oversmoothing, where node
representations become overly uniform, and oversquashing, where information is
compressed or distorted with increasing depth. These issues lead to
information loss, ultimately degrade the models’ representational
capabilities.

Besides, the roadmap is not yet applicable for sample-efficient and
interpretable learning. Sample-efficient learning typically has access to only
a few labeled samples, which cannot fundamentally satisfy the requirement of
scaling up. For sample-efficient learning, such as the molecular properties
prediction task, obtaining sample labels is costly and difficult. Simply
enlarging the model size will lead to overfitting on the labeled samples,
reducing its ability to generalize to test data. The other challenge is
interpretability learning, which focuses more on explaining how the features
or model structures map to the predictions, rather than relying on the
expressiveness of the models. For the interpretable learning, particularly in
the task of reliable reasoning on knowledge graphs, users require reliable
insights through model analysis. However, simply scaling up the model is
insufficient for interpreting data effectively.

Table 1: The duality of data and knowledge in relational graphs, which makes parsimony learning possible. Type of graph |  Data |  Knowledge
---|---|---
Urban network |  Nodes are locations, transportation hubs or functional zones, edges are roads, communication links or logistic routes, and the graphs are presented as spatial-temporal trajectory interactions |  Social force: Learning to simulate crowd trajectories with graph networks.
Molecule |  Nodes are atoms, edges are chemical bonds, graphs are molecular structure |  Physical and chemical properties of functional groups
Knowledge graph |  Nodes are entities, such as people, places, nodes are relationship between entities, and graphs are triplet data with head, tail and relation |  Logical rules, such as transitivity, symmetry and antisymmetry.
|  |

As a consequence, simply following the previous roadmap would degrade models’
generalization performance, and one can observe the point of inflection
(highlighted by the red circle) in the right of Figure 1. This indicates that
the path is not sustainable here, which motivates us to ask one question:

How to develop stronger models for learning from relational graphs if common
scaling law is not applicable?

Inspired by (Donoho, 2006; Candes and Recht, 2012; Liu et al., 2017), which
emphasize using the simplest model that explains the data effectively, we
attempt to find a parsimony way in machine learning, and use simple solutions
to achieve the effects of "making big gains with small things". Intuitively,
humans can can accumulate knowledge, such as symbols and concepts, as well as
rules and principles based on them. These knowledge enables us to learn a wide
range of subjects or skills and to apply them in various tasks (versatile),
fast adapt to new a task with a few or even without demonstrations (sample-
efficient), and reveal the underneath reasons that explain different phenomena
(interpretability). These properties are what we pursue on learning from
relational graphs. This approach is denoted as knowledge-aware parsimony
learning, uses knowledge as a bridge to employ the simplest possible models
and solutions, achieving effective and efficient results while ensuring
generalizability and interpretability.

Fortunately, distinct from the images and texts, which are passively
collected, relational graphs are typically actively constructed. Thus,
relational graphs can naturally contain knowledge delivered by humans, and we
can mine the knowledge from the data and use the knowledge combinatorial
generalize to different scenarios. As illustrated in Table 1, in the urban
network, the data consists of spatial-temporal trajectory interactions, and
the knowledge encompasses the traffic flow patterns, land use patterns and
socioeconomic activities. For the molecular 3D structure, typically the
sample-efficient learning task, the data comprises limited labeled 3D graph
structured molecular samples, and the knowledge includes the properties of
functional group. In knowledge graph, the data is the triplet data with head,
tail and relation, and the knowledge is the critical path for information
delivery. Therefore, the duality, i.e., the coexistence of data and knowledge,
is the key properties in relational graphs.

Figure 2: Illustration of knowledge learning in the relational graphs.

Based on such duality property, the key to achieve parsimony learning is to
develop a learning framework that can benefit from both data and knowledge.
The complex data is governed by relative simple symbolic logic and physical
laws (denote as knowledge), just like various colored can be determined by
only three basic one (i.e., red, blue and green). Thus, it is necessary to
design a framework to first mine the knowledge in the symbolic space by
searching for symbolic logic and physical laws during the learning process.
Then, by data fitting the combination of these knowledge in functional space,
it can be efficiently generalized to the task at hand. In this way, these
knowledge act as “building blocks” to achieve parsimony learning, as shown in
Figure 2.

Existing knowledge-based learning methods, such as inductive logic programming
(Lavrac and Dzeroski, 1994) and symbolic regression (Landajuela et al., 2022),
rely on strict logical operation search rules, and provide good
interpretability. The data-driven based learning methods, including GNNs (Kipf
and Welling, 2017) and LLM (Ouyang et al., 2022) methods, primarily utilize
deep learning techniques (Vaswani et al., 2017), which typically learn from
the data space and demonstrate significantly performance. Additionally, some
works attempt to incorporate knowledge into graph learning (Liu et al., 2023;
Huang et al., 2022). They utilize knowledge of molecular to learn the
properties of graphs, thereby improving the model’s expressiveness
efficiently. However, while these methods integrate knowledge into the model
learning framework, they lack the ability to combinatorially generalize to
other tasks to achieve parsimony learning. Our proposed framework diverges
from the classical roadmap and leverages knowledge as the bridge from
intricate data to parsimony learning. Consequently, by learning the duality of
data and knowledge in relational graphs, we can extract knowledge in the
symbolic space instead of directly learning from the intricate data space. By
combinatorially generalizing the knowledge in the functional space, we can
efficiently achieve the versatile learning, sample-efficient learning and
interpretable learning, ultimately towards the parsimony learning.

The remainder of this paper is organized as follows: Section “Research
Landscape” briefly describes how our approach address issues of versatile,
sample-efficient, and interpretability learning. Section “Parsimony on
Architecture” details how to utilize semantic relationships to recombine
simple architectures, achieving versatile learning by parsimony on
architecture. Section “Parsimony on Parameters” discusses how to mine
functional group properties to selectively tune parameters, achieving sample-
efficient learning by parsimony on parameters. Then, Section “Parsimony on
Inference” explores how to identify logical structures to support concise
inference process, achieving interpretable learning by parsimony on inference.
Lastly, we introduce the “Future Works” from theory, methodology and
application perspectives.

Figure 3: Our research landscape on knowledge-aware parsimony learning.

##  2 Research Landscape

Knowledge-aware parsimony learning is an approach in machine learning that
emphasizes using knowledge as a bridge to employ the simplest possible models
and solutions, achieving effective and efficient results while ensuring
generalizability and interpretability. In our framework, we develop stronger
models, which follow the way of knowledge-aware parsimony learning, for
versatile learning, sample-efficient learning, and interoperability. Our
research framework is in Figure 3, specifically,

  * •

Parsimony on Architecture. To achieve versatile learning on relational graphs,
we propose parsimony learning on architecture by utilizing semantic
relationships to recombine simple architectures. The representative works are
AutoBLM (Zhang et al., 2022a) and KGTuner (Zhang et al., 2022b). AutoBLM
introduces to search task-adaptive scoring functions by bilevel optimization,
resulting in significant performance improvements. KGTuner introduces an
efficient two-stage hyper-parameter search algorithm. When combined with
AutoBLM, it enables the efficient recombination of simple architectures for
versatile learning on relational graphs.

  * •

Parsimony on Parameters. To achieve sample-efficient learning, we propose
parsimony on parameters by mining functional group properties to selective
tune parameters (specific in the field of molecule). The representative works
are PAR (Wang et al., 2021; Yao et al., 2024) and PACIA (Wu et al., 2024).
Empirical results show that PAR achieves the state-of-the-art performance in
molecular property prediction, and PACIA greatly improves the adaptation speed
while significantly reduce the number of learnable parameters.

  * •

Parsimony on Inference. To achieve interpretable learning, we propose
parsimony on inference by identify logics structures to support concise
inference process. The representative works are RED-GNN (Zhang and Yao, 2022)
and EmerGNN (Zhang et al., 2023a). The results show that subgraph learning not
only improves representation ability, but also demonstrates the superiority of
the proposed methods in capturing logical relationships in graphs.

By applying parsimony learning to architecture, parameters and inference, we
search for structures in a knowledge-based way rather than simply stacking
them, effectively solving the problem of architectural incompatibility.
Additionally, by utilizing subgraphs as the knowledge for representation
learning, we address the challenge of the representation bottleneck.

##  3 Parsimony on Architecture

Figure 4: Illustration of ways to achieve versatile learning.

The structure of relational graph is complex and highly heterogeneous. For
example, in a knowledge graph, there can be a lot of variations in the
semantic relationships (such as symmetric, asymmetric and composition
patterns) (Ji et al., 2021). As a result, it is challenging to directly adapt
existing hand-designed models for processing specific properties across
different tasks and graphs. Moreover, there is no universal model that can
well adapt to all the semantic relationships. To achieve versatile learning on
relational graphs, we propose parsimony learning on architecture by utilizing
semantic relationships to recombine simple architectures. Specifically, it is
formulated as the following bi-level learning problem.

###  3.1 A Bi-level Learning Formulation

The key idea of parsimony learning on architectures is to mine the shared
knowledge about semantic relationships in model design, and recombine the
simple architectures to fit specific properties in different tasks. In
general, this contains three main steps: (i) extracting symbolic
representations for state-of-the-art model architectures, (ii) building a
search space to summarize these models, and (iii) adapting the model for
different tasks by searching in the search space. Mathematically, this problem
can be represented as the bi-level optimization problem:

$\displaystyle\max_{\alpha\in\mathcal{A}}$
$\displaystyle\mathcal{P}\big{(}F(\bm{w}^{*};G,\alpha),D_{\text{val}}\big{)}$
s.t.
$\displaystyle\bm{w}^{*}=\arg\min_{\bm{w}}\mathcal{L}\big{(}F(\bm{w};G,\alpha),D_{\text{tra}}\big{)},$

where the lower level minimizes model $F$’s parameters $\bm{w}$ on training
data $D_{\text{tra}}$ with architecture $\alpha$, and the upper level
maximizes the model’s performance on validation data $D_{\text{val}}$ by
adaptively searching architectures $\alpha$ in the search space $\mathcal{A}$.
The space $\mathcal{A}$ contains the transferable knowledge of different
models. By establishing a well-defined $\mathcal{A}$ with simple components
and searching the optimal $\alpha\in\mathcal{A}$ for different tasks on graph
$G$, the model can adapt to different tasks according to the performance on
validation data. In the following, we illustrate this by examples in automated
scoring function design and hyper-parameter optimization on knowledge graphs.

###  3.2 Automated Bi-linear Scoring Function Design

In knowledge graph learning, scoring function is a key component that measures
plausibility of the edges (Wang et al., 2017). Given the embeddings
$\bm{h},\bm{r},\bm{t}$ of a triplet $(h,r,t)$ in the knowledge graph, human
experts often design different kinds of scoring functions to measure
plausibility of the edge. Among them, bi-linear scoring functions achieve
outstanding performance by explicitly modeling the relation patterns. For
instance, DistMult (Yang et al., 2015) models symmetric relations (e.g.,
spouse, friend), while ComplEx (Trouillon et al., 2016) enhances the ability
on modeling anti-symmetric relations (e.g., ancestor). To achieve versatile
learning over different knowledge graphs, AutoBLM provides a unified view of
bi-linear scoring functions, and formulates the design of scoring functions as
a bi-level optimization problem in (3.1).

Table 2: MRR performance comparison of different scoring functions on OGB
datasets. The best results are bolded and the second-best underlined.

Type | Model | WN18RR | FB15k237 | YAGO3-10 | ogbl-biokg | ogbl-wikikg2
---|---|---|---|---|---|---
Translational distance | TransE | 0.186 | 0.233 | 0.279 | 0.745 | 0.426
RotatE | 0.476 | 0.388 | 0.488 | 0.7989 | 0.2530
PairE | 0.484 | 0.351 | 0.471 | 0.8163 | 0.5212
Bi-linear | DistMult | 0.443 | 0.352 | 0.552 | 0.8043 | 0.3729
ComplEx | 0.471 | 0.345 | 0.563 | 0.8095 | 0.4027
Neural networks | Interstellar | 0.48 | 0.32 | 0.51 | — | —
CompGCN | 0.479 | 0.355 | 0.421 | — | —
Versatile learning | AutoBLM | 0.490 | 0.360 | 0.571 | 0.8309 | 0.5186
AutoBLM+KGTuner | 0.492 | 0.364 | 0.577 | 0.8354 | 0.5222

Figure 5: Visualization of the scoring function search by AutoBLM.

As illustrated in Figure 4 (left), AutoBLM formulates a unified search space
where a symbolic matrix $\alpha\in\\{0,\pm 1,\dots,\pm K\\}^{K\times K}$
represents the architecture of a scoring function. With this symbolic
formulation, representative works can be obtained with different choices of
$\alpha$. Furthermore, new models that adapt better for a specific dataset can
be learned with an efficient evolutionary search algorithm, achieving
transfer-ability.

Similarly, Interstellar (Zhang et al., 2020) builds a recurrent symbolic
search space to capture the semantic and structural patterns in knowledge
graphs and proposes a hybrid search algorithm after analyzing properties of
configurations in the search space. SANE (Zhao et al., 2021) builds a unified
supernet of many representative graph neural networks, transforms the discrete
symbols into continuous space, and proposes a gradient descent based search
algorithm for efficient search.

###  3.3 Efficient Hyper-parameter Optimization

As in general machine learning models, hyper-parameters (HPs) play a crucial
role in the learning process of knowledge graphs (Claesen and De Moor, 2015;
Ruffinelli et al., 2020). However, efficiently identifying good configurations
for different knowledge graphs to sufficiently evaluate a model is challenging
due to the limited resources and a primitive understanding of HPs’. KGTuner
(Zhang et al., 2022b) proposes an efficient HP search algorithm. By conducting
a comprehensive study of key factors in the search space of HPs, shared
knowledge is extracted that (i) some configurations such as stochastic
gradient descent optimizer, and margin ranking loss are not effective choices;
(ii) HPs that perform well in random subgraphs tend to perform well on the
original graph; and (iii) tree-based surrogate models, such as the random
forest, can fit better of the complex search space.

Based on the important knowledge obtained from the huge search space, KGTuner
proposes a two-stage HP search algorithm, (Figure 4 (right)). In the first
stage, KGTuner uses Bayesian optimization with a random forest surrogate
(Breiman, 2001) and BORE acquisition function (Tiao et al., 2021) on a sampled
subgraph to efficiently explore various HP configurations. This stage aims to
approximately evaluate as many configurations as possible in a limited time
budget. In the second stage, it then fine-tunes the most promising
configurations by applying them to the entire graph, finding the configuration
that fit the dataset best. The two-stage search algorithm enables KGTuner to
explore efficiently in the search space of hyper-parameters.

Figure 6: PAR meta-learns a property-aware network to improve sample
efficiency.

###  3.4 Empirical Results

The performance of AutoBLM and KGTuner are evaluated on general link
prediction benchmarks on knowledge graph, including WN18RR (Dettmers et al.,
2018), FB15k-237 (Toutanova and Chen, 2015) and YAGO3-10 (Mahdisoltani et al.,
2013), and the large-scale datasets in Open Graph Benchmark (OGB) (Hu et al.,
2020), including ogbl-biokg and ogbl-wikikg2. The results, evaluated with mean
reciprocal ranking (MRR) metric, are shown in Table 2. The baseline models
include translational distance models, TransE (Bordes et al., 2013), RotatE
(Sun et al., 2018) and PairE (Chao et al., 2021), that model relation
embeddings as translation in the vector spaces; bi-linear models DistMult
(Yang et al., 2015) and ComplEx (Trouillon et al., 2016) that compute the
score with a bi-linear function; and neural network models Interstellar (Zhang
et al., 2020) and CompGCN (Vashishth et al., 2020). Compared with them,
AutoBLM gains significant improvement on the two datasets in terms of test
MRR. In particular, it recombines the simple architectures in the bi-linear
model and outperforms the deep neural network models, which often have better
expressiveness with higher computational costs. From the searched model
structures shown in Figure 5, we can observe that the searched structures are
distinctive on the five datasets. They are also novel compared with the
existing structures designed by human experts. Equipped with the hyper-
parameter optimization algorithm KGTuner, AutoBLM+KGTuner can gain better
performance than the reported results in AutoBLM, whose hyper-parameters are
tuned under experts’ supervision. The empirical results demonstrate how bi-
level learning problem defined in (3.1) can effectively and efficiently
recombine simple architectures, achieving versatile learning on relational
graphs.

##  4 Parsimony on Parameters

In AI-assisted scientific research, particularly in drug discovery and
biomedicine, the scarcity of labeled data presents significant challenges. It
is known that drug discovery takes more than 2 billion compounds and at least
10 years on average, while the clinical success rate is around 10% (Paul et
al., 2010). To address drug discovery in a sample-efficient manner, we have
developed meta-learning learning techniques that enforce parsimony on
parameters. These approaches ensure that parameters can be efficiently adapted
in relation to functional groups, optimizing their use and enhancing model
adaptability.

###  4.1 A Meta-learning Formulation

To improve sample efficiency, one can harnesses knowledge to efficiently
enable models to learn from datasets with scarce labeled data, reducing
reliance on extensive datasets and computational resources. Meta-learning
(Wang et al., 2020), a key aspect of this field, focuses on how models can be
quickly generalized to new tasks with minimal labeled data. This approach can
drastically cut the costs associated with collecting, processing, and
analyzing large-scale supervised data but also facilitates learning and mining
for scarce or emerging tasks.

Formally, meta-learning targets at learning a predictive model $f_{\bm{\Phi}}$
with parameters ${\bm{\Phi}}$ from a set of tasks
$\\{\mathcal{T}_{\tau}\\}_{j=1}^{N}$ that can generalize to new task
$\mathcal{T}^{\prime}$ given the few-shot support set. Specifically, the
target properties are different across tasks. Consider training a few-shot
molecular property prediction task $\mathcal{T}_{\tau}$ with respect to a
specific property, each sample $\mathcal{X}_{{\tau},i}$ is a molecular graph
and its label $y_{{\tau},i}\in\\{0,1\\}$ records whether the molecule is
active or inactive on the target property. Only a few labeled samples are
available in $\mathcal{T}_{\tau}$. This $\mathcal{T}_{\tau}$ is formulated as
a binary classification task, associating with a support set
$\mathcal{S}_{\tau}=\\{(\mathcal{X}_{{\tau},s},y_{{\tau},s})\\}_{s=1}^{N_{\tau}}$
containing labeled samples from active/inactive class, and a query set
$\mathcal{Q}_{\tau}=\\{({\mathcal{X}}_{{\tau},q},{y}_{{\tau},q})\\}_{q=1}^{M_{\tau}}$
containing $M_{\tau}$ samples whose labels are only used for evaluation.

The learning objective can be formulated as follows:

$\displaystyle\min_{{\bm{\Phi}}}$
$\displaystyle\sum\nolimits_{\tau}\ell_{\mathcal{Q}_{\tau}}(f_{\bm{\Phi}_{\tau}})$
(2) s.t.
$\displaystyle\;\bm{\Phi}_{\tau}=g({\bm{\Phi}},\ell_{\mathcal{S}_{\tau}}).$
(3)

The parameter update process typically consists of two iterative loops:

  * •

Inner Loop Update (2): This loop adapts the provided parameter set $\bm{\Phi}$
to the specific requirements of the current task $\mathcal{T}_{\tau}$. The
adaptation uses the loss $\ell_{\mathcal{S}_{\tau}}$, which is calculated on
the support set $\mathcal{S}_{\tau}$ in task $\mathcal{T}_{\tau}$. The
function $g(\cdot)$ is used to adjust $\bm{\Phi}$ to better fit the
idiosyncrasies of the given task by optimizing performance on this support
set.

  * •

Outer Loop Update (3): This loop updates the shared parameter set $\bm{\Phi}$
using information generalized across all training tasks. It utilizes the loss
$\ell_{\mathcal{Q}_{\tau}}$, calculated on the query sets $\mathcal{Q}_{\tau}$
across these tasks. The aim here is to refine $\bm{\Phi}$ so that it not only
performs well on individual tasks but also retains a broad applicability
across multiple tasks.

The objective encompasses both ensuring effective task-specific adaptation by
the inner loop and promoting generalization across tasks in the outer loop.
This dual focus helps in developing a model that can quickly adapt to new
tasks while maintaining robustness across varied scenarios.

Figure 7: PACIA introduces GNN adapter to achieve efficient hierarchical adaptation. Method | Tox21 | SIDER | MUV | ToxCast
---|---|---|---|---
10-shot | 1-shot | 10-shot | 1-shot | 10-shot | 1-shot | 10-shot | 1-shot
GNN-ST | $61.23_{(0.89)}$ | $55.49_{(2.31)}$ | $56.25_{(1.50)}$ | $52.98_{(2.12)}$ | $54.26_{(3.61)}$ | $51.42_{(5.11)}$ | $55.66_{(1.47)}$ | $51.80_{(1.99)}$
MAT | $64.84_{(0.93)}$ | $54.90_{(1.89)}$ | $57.45_{(1.26)}$ | $52.97_{(3.00)}$ | $56.19_{(2.88)}$ | $52.01_{(4.05)}$ | $58.50_{(1.62)}$ | $52.41_{(2.34)}$
GNN-MT | $69.56_{(1.10)}$ | $62.08_{(1.25)}$ | $60.97_{(1.02)}$ | $55.39_{(1.83)}$ | $66.24_{(2.40)}$ | $60.78_{(2.91)}$ | $65.72_{(1.19)}$ | $62.38_{(1.67)}$
ProtoNet | $72.99_{(0.56)}$ | $68.22_{(0.46)}$ | $61.34_{(1.08)}$ | $57.41_{(0.76)}$ | $68.92_{(1.64)}$ | $64.81_{(1.95)}$ | $65.29_{(0.82)}$ | $63.73_{(1.18)}$
MAML | $79.59_{(0.33)}$ | $75.63_{(0.18)}$ | $70.49_{(0.54)}$ | $68.63_{(1.51)}$ | $68.38_{(1.27)}$ | $65.82_{(2.49)}$ | $68.43_{(1.85)}$ | $66.75_{(1.62)}$
Siamese | $80.40_{(0.29)}$ | $65.00_{(11.69)}$ | $71.10_{(1.68)}$ | $51.43_{(2.83)}$ | $59.96_{(3.56)}$ | $50.00_{(0.19)}$ | - | -
EGNN | $80.11_{(0.31)}$ | $75.71_{(0.21)}$ | $71.24_{(0.37)}$ | $66.36_{(0.29)}$ | $68.84_{(1.35)}$ | $62.72_{(1.97)}$ | $66.42_{(0.77)}$ | $63.98_{(1.20)}$
IterRefLSTM | $81.10_{(0.10)}$ | $80.97_{(0.06)}$ | $69.63_{(0.16)}$ | $71.73_{(0.06)}$ | $49.56_{(2.32)}$ | $48.54_{(1.48)}$ | - | -
PAR | $82.13_{(0.26)}$ | $\underline{80.02}_{(0.30)}$ | $\underline{75.15}_{(0.35)}$ | $\underline{72.33}_{(0.47)}$ | $68.08_{(2.23)}$ | $65.62_{(3.49)}$ | $70.01_{(0.85)}$ | $\underline{68.22}_{(1.34)}$
ADKF-IFT | $\underline{82.43}_{(0.60)}$ | $77.94_{(0.91)}$ | $67.72_{(1.21)}$ | $58.69_{(1.44)}$ | $\bm{98.18}_{(3.05)}$ | $\underline{67.04}_{(4.86)}$ | $\underline{72.07}_{(0.81)}$ | $67.50_{(1.23)}$
PACIA | $\bm{84.25_{(0.31)}}$ | $\bm{82.77_{(0.15)}}$ | $\bm{82.40_{(0.26)}}$ | $\bm{77.72_{(0.34)}}$ | $\underline{72.58}_{(2.23)}$ | $\bm{68.80}_{(4.01)}$ | $\bm{72.38_{(0.96)}}$ | $\bm{69.89_{(1.17)}}$
Pre-GNN | $83.02_{(0.13)}$ | $82.75_{(0.09)}$ | $77.55_{(0.14)}$ | $67.34_{(0.30)}$ | $67.22_{(2.16)}$ | $65.79_{(1.68)}$ | $73.03_{(0.67)}$ | $71.26_{(0.85)}$
GraphLoG | $81.61_{(0.35)}$ | $79.23_{(0.93)}$ | $75.18_{(0.27)}$ | $67.52_{(1.40)}$ | $67.83_{(1.65)}$ | $66.56_{(1.46)}$ | $73.92_{(0.15)}$ | $73.10_{(0.39)}$
MGSSL | $83.24_{(0.09)}$ | $\underline{83.21}_{(0.12)}$ | $77.87_{(0.18)}$ | $69.66_{(0.21)}$ | $68.58_{(1.32)}$ | $66.93_{(1.74)}$ | $73.51_{(0.45)}$ | $72.89_{(0.63)}$
GraphMAE | $84.01_{(0.27)}$ | $81.54_{(0.18)}$ | $76.07_{(0.15)}$ | $67.60_{(0.38)}$ | $67.99_{(1.28)}$ | $\underline{67.50}_{(2.12)}$ | $74.15_{(0.33)}$ | $72.67_{(0.71)}$
Meta-MGNN | $83.44_{(0.14)}$ | $82.67_{(0.20)}$ | ${77.84}_{(0.34)}$ | $74.62_{(0.41)}$ | ${68.31}_{(3.06)}$ | $66.10_{(3.98)}$ | $74.69_{(0.57)}$ | $73.29_{(0.85)}$
Pre-PAR | ${84.95}_{(0.24)}$ | $\underline{83.01}_{(0.28)}$ | $\underline{78.05}_{(0.15)}$ | $\underline{75.29}_{(0.32)}$ | $69.88_{(1.57)}$ | $66.96_{(2.63)}$ | ${75.48}_{(0.99)}$ | $\underline{73.90}_{(1.21)}$
Pre-ADKF-IFT | $\underline{86.06}_{(0.35)}$ | $80.97_{(0.48)}$ | $70.95_{(0.60)}$ | $62.16_{(1.03)}$ | $\bm{95.74}_{(0.37)}$ | $67.25_{(3.87)}$ | $\bm{76.22}_{(0.13)}$ | $71.13_{(1.15)}$
Pre-PACIA | $\bm{86.40_{(0.27)}}$ | $\bm{84.35_{(0.14)}}$ | $\bm{83.97_{(0.22)}}$ | $\bm{80.70_{(0.28)}}$ | $\underline{73.43}_{(1.96)}$ | $\bm{69.26_{(2.35)}}$ | $\bm{76.22_{(0.73)}}$ | $\bm{75.09_{(0.95)}}$
Table 3: Test ROC-AUC (%) obtained on MoleculeNet. The best results are
bolded, second-best results are underlined. We compare with baselines both
without pretrained GNN encoders (the above group) and with pretrained GNN
encoders (the below group).

###  4.2 Property-Aware Relation Networks

Our first work towards this problem is the property-aware relation networks
(PAR) (Wang et al., 2021; Yao et al., 2024). PAR uses a property-aware
molecular encoder to transform the generic molecular embeddings to property-
aware ones. To fully leverage the supervised learning signal, PAR learns to
estimate the molecular relation graph by a query-dependent relation graph
learning module, in which molecular embeddings are refined w.r.t. the target
property. Thus, the facts that both property-related information and
relationships among molecules change across different properties are utilized
to better learn and propagate molecular embeddings. Besides, we propose to
selectively update $\bm{\bm{\theta}}$ that represent generic information and
$\bm{\Phi}$ that represent property-aware information. We only update
$\bm{\Phi}$ in the inner-loop update, while simultaneously update
$\bm{\bm{\theta}}$ and $\bm{\Phi}$ in the outer-loop update. We use gradient
descents to update parameters in both loops. Through the selective update
strategy, the model can capture generic and property-aware information
separately in the training procedure.

###  4.3 Parameter-Efficient GNN Adapter

We further introduce parameter-efficient GNN adapter (PACIA) (Wu et al.,
2024). By adopting this approach, PACIA significantly reduces the risk of
overfitting. Moreover, it offers the advantage of faster inference speeds, as
the adapted parameters are generated through a single forward pass rather than
through iterative optimization steps. Additionally, initializing the function
$g(\cdot)$ with neural networks allows for more flexible forms of updates
compared to traditional gradient descent methods. We first summarize existing
works into an encoder-predictor framework: an encoder which maps $\mathcal{X}$
to molecular representation which is a fixed-length vector, and a predictor
which assigns label for a query molecule given support molecules. Recall that
GNN-based predictor proposed in PAR, which operates on relation graphs of
molecules is found to effectively compensate for the lack of supervised
information, outperforms existing predictors. In PACIA, we let the GNN act as
both encoder and predictor. Further, we design a hierarchical adaptation
mechanism in the framework: Task-level adaptation is achieved in the encoder
since the structural features in molecular graphs needs to be captured in a
property-adaptive manner, while query-level adaptation is achieved in the
predictor based on the property-adaptive representations. To adapt GNN’s
parameter-efficiently, we design a hypernetwork-based GNN adapter to generate
a few adaptive parameters to modulate the node embedding and propagation
depth, which are essential in message passing process. No further fine-tuning
is required.

###  4.4 Empirical Results

Following previous works (Altae-Tran et al., 2017; Guo et al., 2021), we
evaluate the proposed PAR and PACIA on the benchmark datasets Tox21, SIDER,
MUV and ToxCast from MoleculeNet (Wu et al., 2018). We compare with two groups
of baselines: (i) baselines with molecular encoders trained from scratch,
including GNN-ST (Gilmer et al., 2017), GNN-MT (Corso et al., 2020; Gilmer et
al., 2017), MAT (Maziarka et al., 2020), Siamese (Koch et al., 2015), ProtoNet
(Snell et al., 2017), MAML (Finn et al., 2017), EGNN (Kim et al., 2019),
IterRefLSTM (Altae-Tran et al., 2017), PAR (Wang et al., 2021) and ADKF-IFT
(Chen et al., 2022); and (ii) baselines with molecular encoders pretrained on
ZINC15 dataset (Sterling and Irwin, 2015), including Pre-GNN (Hu et al.,
2019), GraphLoG (Xu et al., 2021), MGSSL (Hu et al., 2019), GraphMAE (Hou et
al., 2022), Meta-MGNN (Guo et al., 2021), and Pre-ADKF-IFT (Chen et al.,
2022).

Table 3 shows the prediction performance. Our analysis reveals that methods
utilizing pretrained graph-based molecular encoders generally outperform those
with encoders learned from scratch. This underscores the effectiveness of
pretrained encoders in capturing rich, generic molecular information,
subsequently providing superior molecular embeddings. In further evaluations,
meta-learning methods that learn relational graphs—specifically EGNN, PAR, and
PACIA—demonstrate enhanced performance. Notably, PACIA consistently achieves
the highest ROC-AUC scores, followed closely by PAR. ADKF-IFT, on the other
hand, excels in the 10-shot case on MUV, significantly outperforming other
methods. However, this might represent a special case where ADKF-IFT is
particularly effective but may not exhibit generalizable performance.
Additionally, among the baselines that incorporate pretrained encoders,
similar patterns emerge: PACIA equipped with a pretrained encoder (Pre-PACIA)
ranks as the most effective, followed by PAR with a pretrained encoder (Pre-
PAR). In summary, PACIA establishes itself as the new state-of-the-art for
predicting molecular properties. It not only delivers superior predictive
accuracy but also features significantly faster adaptation speed and requires
fewer adaptive parameters.

(a) Approach: subgraph learning

(b) Application: drug discovery

Figure 8: Illustration of ways to interpreting the inference process.

Table 4 summarizes the statistics involved in the adaptation process, i.e.,
time and parameters required to process the support set and the test
performance. PAR uses molecules in support set to take gradient steps, and
updates all parameters in GNN. We record each of a maximum five steps, where
we can find that it easily overfits as the testing ROC-AUC keeps dropping with
more steps. The time consumption also grows. In contrast, PACIA processes
molecules in support set by hypernetworks, which is much more efficient as
only one single forward pass is needed. PACIA can obtain better performance
due to the reduction of adaptive parameters, which also leads to better
generalization and alleviates the risk of overfitting to a few shots.

| PACIA | PAR |
---|---|---|---
# Total para. | 3.28M | 2.31M |
# Adaptive para. | 3.00K | 0.38M |
# Gradient steps | - | 1 | 2 | 3 | 4 | 5
ROC-AUC (%) | 84.26 | 82.07 | 81.85 | 80.32 | 79.09 | 77.25
Time (secs) | 1.09 | 2.02 | 3.62 | 5.34 | 6.76 | 8.10
Table 4: Selectively update parameters in few-shot learning

##  5 Parsimony on Inference

Interpretability is the foundation for downstream applications of machine
learning. In many fields, experts need to interpret and understand the results
of models in order to make informed decisions. In graph learning tasks, GNNs
have brought great benefits for the effectiveness in learning from relational
graphs. However, it is very challenging to interpret the inference process
when facing a massive amount of associated information on graphs. For graph
learning, the core problem in achieving model interpretability lies in
accurately capturing strong logical relationships and exhibiting inference
process with evidence. To interpret the inference processes on graph, our key
idea is to capture the logical rules inside graphs as supporting evidence and
use the logical interpretation of knowledge to clarify models’ reasoning
process. In the following, we introduce the idea of subgraph learning for
intepreting and its application to drug discovery.

Table 5: Performance comparison on knowledge graph reasoning tasks. The three
metrics MRR, Hit@1 and Hit@10 indicates better performance with larger values.
The best results are bolded and the second-best underlined.

Model | WN18RR | FB15k-237 | YAGO3-10
---|---|---|---
MRR | H@1 | H@10 | MRR | H@1 | H@10 | MRR | H@1 | H@10
RotatE | 0.477 | 42.8 | 57.1 | 0.337 | 24.1 | 53.3 | 0.495 | 40.2 | 67.0
HousE | 0.511 | 45.2 | 60.2 | 0.361 | 26.6 | 55.1 | 0.571 | 49.1 | 71.4
DRUM | 0.486 | 42.5 | 58.6 | 0.343 | 25.5 | 51.6 | 0.531 | 45.3 | 67.6
RNNLogic | 0.483 | 44.6 | 55.8 | 0.344 | 25.2 | 53.0 | 0.554 | 50.9 | 67.3
CompGCN | 0.487 | 44.3 | 54.6 | 0.355 | 26.4 | 53.5 | 0.421 | 39.2 | 57.7
NBFNet | 0.551 | 49.7 | 66.6 | 0.415 | 32.1 | 59.9 | 0.550 | 47.9 | 68.6
RED-GNN | 0.564 | 50.2 | 67.8 | 0.418 | 32.9 | 59.0 | 0.584 | 50.9 | 71.3

Figure 9: Visualization of the structure learned by EmerGNN. DB006204
(Tapentadol) is an existing drug, and DB00757 (Dolasetron) is an emerging
drug.

Table 6: Performance comparison on DDI prediction tasks. All the metrics
indicates better performance with larger values. The best results are bolded
and the second-best underlined. All the experiments run on 5 times with mean
and std values reported.

Datasets | DrugBank (S0) | DrugBank (S1) | TWOSIDES (S0) | TWOSIDES (S1)
---|---|---|---|---
Methods | F1-score | Accuracy | Kappa | F1-score | Accuracy | Kappa | PR-AUC | ROC-AUC | Accuracy | PR-AUC | ROC-AUC | Accuracy
CSMDDI |  45.5$\pm$1.8 |  62.6$\pm$2.8 |  55.0$\pm$3.2 |  19.8$\pm$3.1 |  37.3$\pm$4.8 |  22.0$\pm$4.9 |  73.2$\pm$2.6 |  74.2$\pm$2.9 |  69.9$\pm$2.2 |  55.8$\pm$4.9 |  57.0$\pm$6.1 |  55.1$\pm$4.2
HIN-DDI |  37.3$\pm$2.9 |  58.9$\pm$1.4 |  47.6$\pm$1.8 |  8.8$\pm$1.0 |  27.6$\pm$2.4 |  13.8$\pm$2.4 |  81.9$\pm$0.6 |  83.8$\pm$0.9 |  79.3$\pm$1.1 |  64.8$\pm$2.3 |  58.5$\pm$1.6 |  59.8$\pm$1.4
KG-DDI |  26.1$\pm$0.9 |  46.7$\pm$1.9 |  35.2$\pm$2.5 |  1.1$\pm$0.1 |  32.2$\pm$3.6 |  0.0$\pm$0.0 |  79.1$\pm$0.9 |  77.7$\pm$1.0 |  60.2$\pm$2.2 |  53.9$\pm$3.9 |  47.0$\pm$5.5 |  54.0$\pm$6.2
DeepLGF |  39.7$\pm$2.3 |  60.7$\pm$2.4 |  51.0$\pm$2.6 |  4.8$\pm$1.9 |  31.9$\pm$3.7 |  8.2$\pm$2.3 |  81.4$\pm$2.1 |  82.2$\pm$2.6 |  72.8$\pm$2.8 |  59.4$\pm$8.7 |  54.7$\pm 5.9$ |  54.0$\pm$6.2
EmerGNN |  62.0$\pm$2.0 |  68.6$\pm$3.7 |  62.4$\pm$4.3 |  25.0$\pm$2.8 |  46.3$\pm$3.6 |  31.9$\pm$3.8 |  90.6$\pm$0.7 |  91.5$\pm$1.0 |  84.6$\pm$0.7 |  81.4$\pm$7.4 |  79.6$\pm$7.9 |  73.0$\pm$8.2

###  5.1 Interpret with subgraph learning

The relational graph structures are complex and hard to understand by humans
directly. In comparison, relational paths with multiple connected edges are
more interpretable (Xiong et al., 2017; Das et al., 2018). Based on this
observation, RED-GNN (Zhang and Yao, 2022) introduces a new relational
structure, called relational di-graph (r-digraph). The r-digraphs generalize
relational paths to subgraphs by preserving the overlapped relational paths
and structures of relations for reasoning. By leveraging the GNN model with
attention mechanism to propagate information over the subgraph, significant
performance improvement has been achieved and logical paths in r-digraphs can
be captured.

An example of the r-digraph is shown in Figure 8(a), where all the relational
paths with lengths three between Sam and Spider-2 are used to construct this
subgraph. This is achieved by introducing reverse and identity relations as in
(Das et al., 2018; Vashishth et al., 2020; Zhang and Yao, 2022), and
separately representing the entities in different hops into different layers.
In this way, we can start encoding from the query entity (Sam), aggregate
information for three steps, and obtain the subgraph representation with the
final layer encoding of the answer entity (Spider-2). By introducing attention
mechanism in each layer, important edges leading to correct prediction can be
learned so that we can obtain some representative evidence from the subgraph
for interpretation.

Similar achievements can be found in NBFNet (Zhu et al., 2021) which develops
upon the Bellman Fold algorithm to encode paths in graphs. To reduce the size
of subgraphs, AdaProp (Zhang et al., 2023b) and A*Net (Zhu et al., 2024)
introduce pruning algorithms that gradually remove irrelevant edges in each
propagation layer. Benefited by the attention mechanism and subgraph learning,
all these methods achieve significant improvement over the traditional
methods, maintaining interpretable inference process as well.

In addition, the effectiveness of RED-GNN is also theoretically analyzed in
the context of learning rules from graphs. The special message-passing
mechanism of RED-GNN and NBFNet is first concluded as QL-GNN (Qiu et al.,
2024). To analyze expressiveness, the rules are described by first-order logic
with counting, and then the expressiveness is presented as the type of logical
formulas, i.e., logical formulas from model logic with counting (Theorem 3.2
in Qiu et al. (2024)), that QL-GNN can learn. However, classical methods,
e.g., CompGCN and RGCN, fail to learn these rules (Theorem 3.4 in Qiu et al.
(2024)). The theoretical work demonstrates that QL-GNN can learn more
expressive rules than classical methods, thus showing the superiority of RED-
GNN in capturing logical relationships in graphs.

###  5.2 Application in drug discovery

In the ever-changing world of pharmaceuticals, the intersection of scientific
advancements and regulatory changes has led to a significant breakthrough,
particularly in the rapid development of new drugs aimed at treating rare,
severe, or life-threatening diseases (Su et al., 2022; Ledford, 2022). Machine
learning models, particularly GNNs, have become powerful tools for predicting
drug interactions, capable of comprehensively capturing intricate
relationships on the interaction graphs (Vilar et al., 2014; Yu et al., 2021).
EmerGNN (Zhang et al., 2023a) takes this a step further by introducing a graph
learning framework that predicts interactions between emerging and existing
drugs by integrating drug-drug interaction (DDI) and biomedical networks,
extracting path-based subgraphs, and learning subgraph representations.

The general idea of EmerGNN is illustrated in Figure 8(b). Emerging drugs
typically have limited interactions with existing drugs. To address this
issue, EmerGNN utilizes biomedical network and extracts subgraph
$\mathcal{G}_{u,v}^{L}$ from it to connect emerging drug $u$ with existing
drug $v$. Drawing inspiration from the subgraph learning methods (Zhang and
Yao, 2022; Zhu et al., 2021), EmerGNN proposes a flow-based GNN
$g(\mathcal{G}_{u,v}^{L};\bm{\theta})$ with an attention mechanism to encode
pair-wise subgraph representations for drug pairs. These subgraph
representations are then used to directly predict the interaction of the drug
pair. This approach enables a more nuanced understanding of interactions
involving emerging drugs, incorporating knowledge from the interconnected
network of biomedical entities and their relationships, and providing
interpretable insights.

###  5.3 Empirical results

The effectiveness of subgraph learning methods are evaluated on general
knowledge graph reasoning benchmarks, including WN18RR (Dettmers et al.,
2018), FB15k-237 (Toutanova and Chen, 2015), and YAGO3-10 (Mahdisoltani et
al., 2013). As shown in Table 9, the subgraph learning methods NBFNet (Zhu et
al., 2021) and RED-GNN (Zhang and Yao, 2022) show significant advantage over
the embedding-based methods RotatE (Sun et al., 2018) and HousE (Li et al.,
2022), path-based methods DRUM (Sadeghian et al., 2019) and RNNLogic (Qu et
al., 2020), as well as GNN method CompGCN (Vashishth et al., 2020). For the
DDI prediction tasks, we evaluate the performance on two benchmarks DrugBank
(Wishart et al., 2018) and TWOSIDES (Tatonetti et al., 2012), where two
settings are considered: S1 setting predicts DDI between an emerging drug and
an existing drug, while S2 setting predicts DDI between two emerging drugs.
The subgraph learning method EmerGNN (Zhang et al., 2023a) outperforms CSMDDI
(Liu et al., 2022) that directly uses drug features to predict, HINDDI (Tanvir
et al., 2021) that counts meta-paths from biomedical network for prediction,
KG-DDI (Karim et al., 2019) that learns drug embeddings, and DeepLGF (Ren et
al., 2022) that models with GNN in both benchmarks and both settings.
Furthermore, we show a case study with the visualized subgraph in Figure 9 by
selecting important paths according to the attention values. The path
connecting two drugs through the binding protein Gene::1565 (CYP2D6), which is
a P450 enzyme that plays a key role in drug metabolism, for example, shows a
way to interpret the inference process with supporting evidence. From the
empirical comparison and case studies, we conclude that subgraph learning can
not only achieve improvements in representation learning, but also interpret
the inference process with supporting evidence.

##  6 Future Works

Finally, we talk about our ongoing future works from theory, method and
application.

###  6.1 Theory: Representation of diverse knowledge

Throughout the history of machine learning, parsimony has been a crucial
insight in developing effective models and algorithms. Prior to the deep-
learning era, parsimony was applied to vectors and matrices to facilitate
tractable and reliable solutions for various data recovery problems. For
instance, compressed sensing (Donoho, 2006) enforces sparsity on vectors to
enable signal recovery from only a few measurements. Similarly, low-rank
matrix recovery (Candes and Recht, 2012) assumes the restricted isometry
property of matrices to recover a low-rank matrix from limited observations.
For relational graphs, parsimony is formulated as more diverse and complex
forms of knowledge representations, which are essential for capturing the
complex relationships in the data. For example, the knowledge behind
relational graphs can be represented as symbolic logic, physical laws,
programs, and common sense. Different forms of knowledge are jointly trained
in deep neural networks. However, existing knowledge representations are often
limited by their singular focus, which hinders the capture of correlations
between different knowledge representations. To overcome this limitation, a
way of unified knowledge representation is necessary to capture and integrate
diverse knowledge, ultimately enhancing model performance.

###  6.2 Method: Integration with foundation models

Foundation models have achieved remarkable success in natural language
processing (Achiam et al., 2023), including tasks such as text generation,
question answering, and language understanding. Previous studies consider
foundation models to be powerful text compressors capable of capturing
essential information from textual data and generating coherent and
contextually relevant responses. Therefore, foundation models can be viewed as
a powerful parsimony learning model for text. However, foundation models’
performance is inferior to that of GNN when it comes to relational graphs (Pan
et al., 2024). The main reason behind this phenomenon is the difference in
knowledge representation between texts and relational graphs. Foundation
models extract knowledge from a sequence of tokens, resembling a chain-like
graph. On the other hand, relational graphs are more diverse and contain
encoded knowledge within their structure. Consequently, the knowledge
representation of foundation models does not align well with relational
graphs, resulting in subpar performance on tasks involving such graphs. To
improve foundation models, our future research aims to incorporate parsimony
learning into foundation models that can effectively reduce redundant
information in the models and enable reliable reasoning for various tasks
using different types of knowledge.

###  6.3 Applications: Usage in AI4Science

In terms of application, our future work involves applying parsimony learning
to various scenarios in scientific problems, particularly focusing on material
science, drug development, and physics. Parsimony learning aims to simplify
models while retaining their ability to effectively explain and predict
complex data. In material science, we will use parsimony learning to predict
material properties and discover new materials by analyzing relational graphs
of atomic structures and extracting key physical and chemical knowledge. For
drug development, we will enhance drug discovery processes by modeling
molecular interactions with biological targets using simplified, yet
effective, graph-based representations. In physics, parsimony learning will
help us understand and predict phenomena in areas such as particle physics and
quantum mechanics by leveraging simplified models that capture the essential
relationships between physical entities. Through these applications, we aim to
demonstrate that parsimony learning can address diverse and complex scientific
challenges more efficiently and interpretably than traditional methods.

##  7 Conclusion

This paper introduces a novel framework for learning from relational graphs
using knowledge-aware Parsimony Learning. By leveraging the duality between
data and knowledge, our method extracts symbolic logic and physical laws
during the learning process and applies combinatorial generalization to
various tasks. This approach effectively overcomes the limitations of
traditional scaling methods, such as architectural incompatibility and
representation bottlenecks. Experimental results demonstrate that our
framework significantly improves model performance, showcasing its ability to
achieve versatile, sample-efficient, and interpretable learning. These
findings underscore the potential of integrating knowledge into machine
learning models, offering a promising direction for future research and
applications.

## References

  * Achiam et al. [2023] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al.  Gpt-4 technical report.  _arXiv preprint_ , 2023.
  * Altae-Tran et al. [2017] Han Altae-Tran, Bharath Ramsundar, Aneesh S Pappu, and Vijay Pande.  Low data drug discovery with one-shot learning.  _ACS Central Science_ , 3(4):283–293, 2017.
  * Bommasani et al. [2021] Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al.  On the opportunities and risks of foundation models.  _arXiv preprint_ , 2021.
  * Bordes et al. [2013] Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko.  Translating embeddings for modeling multi-relational data.  _NeurIPS_ , 26, 2013.
  * Breiman [2001] Leo Breiman.  Random forests.  _Machine Learning_ , 45:5–32, 2001.
  * Brown et al. [2020] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al.  Language models are few-shot learners.  In _NeurIPS_ , volume 33, pages 1877–1901, 2020.
  * Candes and Recht [2012] Emmanuel Candes and Benjamin Recht.  Exact matrix completion via convex optimization.  _CACM_ , 55(6):111–119, 2012.
  * Chao et al. [2021] Linlin Chao, Jianshan He, Taifeng Wang, and Wei Chu.  PairRE: Knowledge graph embeddings via paired relation vectors.  In _ACL_ , pages 4360–4369, 2021.
  * Chen et al. [2022] Wenlin Chen, Austin Tripp, and José Miguel Hernández-Lobato.  Meta-learning adaptive deep kernel gaussian processes for molecular property prediction.  In _ICLR_ , 2022.
  * Claesen and De Moor [2015] Marc Claesen and Bart De Moor.  Hyperparameter search in machine learning.  _arXiv preprint_ , 2015.
  * Corso et al. [2020] Gabriele Corso, Luca Cavalleri, Dominique Beaini, Pietro Liò, and Petar Veličković.  Principal neighbourhood aggregation for graph nets.  In _NeurIPS_ , pages 13260–13271, 2020.
  * Dai et al. [2016] Hanjun Dai, Bo Dai, and Le Song.  Discriminative embeddings of latent variable models for structured data.  In _ICML_ , pages 2702–2711, 2016.
  * Das et al. [2018] Rajarshi Das, Shehzaad Dhuliawala, Manzil Zaheer, Luke Vilnis, Ishan Durugkar, Akshay Krishnamurthy, Alex Smola, and Andrew McCallum.  Go for a walk and arrive at the answer: Reasoning over paths in knowledge bases using reinforcement learning.  In _ICLR_ , 2018.
  * Dettmers et al. [2018] Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel.  Convolutional 2d knowledge graph embeddings.  In _AAAI_ , volume 32, 2018.
  * Donoho [2017] David Donoho.  50 years of data science.  _JCGS_ , 26(4):745–766, 2017.
  * Donoho [2006] David L Donoho.  Compressed sensing.  _IEEE TIT_ , 52(4):1289–1306, 2006.
  * Finn et al. [2017] Chelsea Finn, Pieter Abbeel, and Sergey Levine.  Model-agnostic meta-learning for fast adaptation of deep networks.  In _ICML_ , pages 1126–1135, 2017.
  * Gilmer et al. [2017] Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl.  Neural message passing for quantum chemistry.  In _ICML_ , pages 1263–1272, 2017.
  * Guo et al. [2021] Zhichun Guo, Chuxu Zhang, Wenhao Yu, John Herr, Olaf Wiest, Meng Jiang, and Nitesh V Chawla.  Few-shot graph learning for molecular property prediction.  In _WebConf_ , pages 2559–2567, 2021.
  * He et al. [2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In _CVPR_ , pages 770–778, 2016.
  * Hou et al. [2022] Zhenyu Hou, Xiao Liu, Yuxiao Dong, Chunjie Wang, Jie Tang, et al.  GraphMAE: Self-supervised masked graph autoencoders.  In _KDD_ , pages 594–604, 2022.
  * Hu et al. [2019] Weihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitnik, Percy Liang, Vijay Pande, and Jure Leskovec.  Strategies for pre-training graph neural networks.  In _ICLR_ , 2019.
  * Hu et al. [2020] Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec.  Open graph benchmark: Datasets for machine learning on graphs.  _NeurIPS_ , 33:22118–22133, 2020.
  * Huang et al. [2022] Wenbing Huang, Jiaqi Han, Yu Rong, Tingyang Xu, Fuchun Sun, and Junzhou Huang.  Equivariant graph mechanics networks with constraints.  _arXiv preprint_ , 2022.
  * Ji et al. [2021] Shaoxiong Ji, Shirui Pan, Erik Cambria, Pekka Marttinen, and S Yu Philip.  A survey on knowledge graphs: Representation, acquisition, and applications.  _IEEE TKDE_ , 33(2):494–514, 2021.
  * Jordan and Mitchell [2015] Michael I Jordan and Tom M Mitchell.  Machine learning: Trends, perspectives, and prospects.  _Science_ , 349(6245):255–260, 2015.
  * Kaplan et al. [2020] Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei.  Scaling laws for neural language models.  _arXiv preprint_ , 2020.
  * Karim et al. [2019] Md Rezaul Karim, Michael Cochez, Joao Bosco Jares, Mamtaz Uddin, Oya Beyan, and Stefan Decker.  Drug-drug interaction prediction based on knowledge graph embeddings and convolutional-lstm network.  In _BCB_ , pages 113–123, 2019.
  * Kim et al. [2019] Jongmin Kim, Taesup Kim, Sungwoong Kim, and Chang D Yoo.  Edge-labeling graph neural network for few-shot learning.  In _CVPR_ , pages 11–20, 2019.
  * Kipf and Welling [2017] Thomas N. Kipf and Max Welling.  Semi-supervised classification with graph convolutional networks.  In _ICLR_ , 2017.
  * Koch et al. [2015] Gregory Koch, Richard Zemel, and Ruslan Salakhutdinov.  Siamese neural networks for one-shot image recognition.  In _ICML Deep Learning Workshop_ , 2015.
  * Landajuela et al. [2022] Mikel Landajuela, Chak Shing Lee, Jiachen Yang, Ruben Glatt, Claudio P Santiago, Ignacio Aravena, Terrell Mundhenk, Garrett Mulcahy, and Brenden K Petersen.  A unified framework for deep symbolic regression.  In _NeurIPS_ , volume 35, pages 33985–33998, 2022.
  * Lavrac and Dzeroski [1994] Nada Lavrac and Saso Dzeroski.  Inductive logic programming.  In _WLP_ , pages 146–160. Springer, 1994.
  * Ledford [2022] Heidi Ledford.  Hundreds of covid trials could provide a deluge of new drugs.  _Nature_ , pages 25–27, 2022.
  * Li et al. [2022] Rui Li, Jianan Zhao, Chaozhuo Li, Di He, Yiqi Wang, et al.  House: Knowledge graph embedding with householder parameterization.  In _IICML_ , pages 13209–13224. PMLR, 2022.
  * Liang et al. [2018] Xiaodan Liang, Zhiting Hu, Hao Zhang, Liang Lin, and Eric P Xing.  Symbolic graph reasoning meets convolutions.  In _NeurIPS_ , volume 31, 2018.
  * Liu et al. [2017] Bo Liu, Ying Wei, Yu Zhang, and Qiang Yang.  Deep neural networks for high dimension, low sample size data.  In _IJCAI_ , volume 2017, pages 2287–2293, 2017.
  * Liu et al. [2023] Yang Liu, Jiashun Cheng, Haihong Zhao, Tingyang Xu, Peilin Zhao, Fugee Tsung, Jia Li, and Yu Rong.  Improving generalization in equivariant graph neural networks with physical inductive biases.  In _ICLR_ , 2023.
  * Liu et al. [2024] Yixin Liu, Kai Zhang, Yuan Li, Zhiling Yan, Chujie Gao, Ruoxi Chen, Zhengqing Yuan, Yue Huang, Hanchi Sun, Jianfeng Gao, Lifang He, and Lichao Sun.  Sora: A review on background, technology, limitations, and opportunities of large vision models, 2024.
  * Liu et al. [2022] Zun Liu, Xing-Nan Wang, Hui Yu, Jian-Yu Shi, and Wen-Min Dong.  Predict multi-type drug–drug interactions in cold start scenario.  _BMC bioinformatics_ , 23(1):75, 2022.
  * Mahdisoltani et al. [2013] Farzaneh Mahdisoltani, Joanna Biega, and Fabian M Suchanek.  Yago3: A knowledge base from multilingual wikipedias.  In _CIDR_ , 2013.
  * Maziarka et al. [2020] Lukasz Maziarka, Tomasz Danel, Slawomir Mucha, Krzysztof Rataj, Jacek Tabor, and Stanislaw Jastrzebski.  Molecule attention transformer.  _arXiv preprint_ , 2020.
  * Nt and Maehara [2019] Hoang Nt and Takanori Maehara.  Revisiting graph neural networks: All we have is low-pass filters.  _arXiv preprint_ , 2019.
  * Ouyang et al. [2022] Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al.  Training language models to follow instructions with human feedback.  _NeurIPS_ , 35:27730–27744, 2022.
  * Pan et al. [2024] Shirui Pan, Linhao Luo, Yufei Wang, Chen Chen, Jiapu Wang, and Xindong Wu.  Unifying large language models and knowledge graphs: A roadmap.  _IEEE TKDE_ , 2024.
  * Paul et al. [2010] Steven M Paul, Daniel S Mytelka, Christopher T Dunwiddie, Charles C Persinger, Bernard H Munos, Stacy R Lindborg, and Aaron L Schacht.  How to improve R&D productivity: The pharmaceutical industry’s grand challenge.  _Nature Reviews Drug Discovery_ , 9(3):203–214, 2010.
  * Porta et al. [2006] Sergio Porta, Paolo Crucitti, and Vito Latora.  The network analysis of urban streets: a primal approach.  _Environment and Planning B: planning and design_ , 33(5):705–725, 2006.
  * Qiu et al. [2024] Haiquan Qiu, Yongqi Zhang, Yong Li, and quanming yao.  Understanding expressivity of GNN in rule learning.  In _ICLR_ , 2024.
  * Qu et al. [2020] Meng Qu, Junkun Chen, Louis-Pascal Xhonneux, Yoshua Bengio, and Jian Tang.  Rnnlogic: Learning logic rules for reasoning on knowledge graphs.  In _ICLR_ , 2020.
  * Rabiner and Juang [1993] L.R. Rabiner and B.H. Juang.  _Fundamentals of Speech Recognition_.  PTR Prentice Hall, 1993.  ISBN 9780130151575.
  * Ren et al. [2022] Zhong-Hao Ren, Zhu-Hong You, Chang-Qing Yu, Li-Ping Li, Yong-Jian Guan, Lu-Xiang Guo, and Jie Pan.  A biomedical knowledge graph-based method for drug–drug interactions prediction through combining local and global features with deep neural networks.  _BIB_ , 23(5):bbac363, 2022.
  * Ruffinelli et al. [2020] Daniel Ruffinelli, Samuel Broscheit, and Rainer Gemulla.  You can teach an old dog new tricks! on training knowledge graph embeddings.  2020\.
  * Sadeghian et al. [2019] Ali Sadeghian, Mohammadreza Armandpour, Patrick Ding, and Daisy Zhe Wang.  Drum: End-to-end differentiable rule mining on knowledge graphs.  _NeurIPS_ , 32, 2019.
  * Snell et al. [2017] Jake Snell, Kevin Swersky, and Richard Zemel.  Prototypical networks for few-shot learning.  In _NeurIPS_ , pages 4080–4090, 2017.
  * Sterling and Irwin [2015] Teague Sterling and John J Irwin.  ZINC 15–ligand discovery for everyone.  _Journal of Chemical Information and Modeling_ , 55(11):2324–2337, 2015.
  * Su et al. [2022] Xian Su, Haixue Wang, Nan Zhao, Tao Wang, and Yimin Cui.  Trends in innovative drug development in china.  _Nature reviews. Drug discovery_ , 21(10):709–710, 2022.
  * Sun et al. [2018] Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, and Jian Tang.  Rotate: Knowledge graph embedding by relational rotation in complex space.  In _ICLR_ , 2018.
  * Tanvir et al. [2021] Farhan Tanvir, Muhammad Ifte Khairul Islam, and Esra Akbas.  Predicting drug-drug interactions using meta-path based similarities.  In _CIBCB_ , pages 1–8. IEEE, 2021.
  * Tatonetti et al. [2012] Nicholas P Tatonetti, Patrick P Ye, Roxana Daneshjou, and Russ B Altman.  Data-driven prediction of drug effects and interactions.  _Science translational medicine_ , 4(125):125ra31–125ra31, 2012.
  * Tiao et al. [2021] Louis C Tiao, Aaron Klein, Matthias W Seeger, Edwin V Bonilla, Cedric Archambeau, and Fabio Ramos.  Bore: Bayesian optimization by density-ratio estimation.  In _ICML_ , pages 10289–10300, 2021.
  * Topping et al. [2021] Jake Topping, Francesco Di Giovanni, Benjamin Paul Chamberlain, Xiaowen Dong, and Michael M Bronstein.  Understanding over-squashing and bottlenecks on graphs via curvature.  _arXiv preprint_ , 2021.
  * Toutanova and Chen [2015] Kristina Toutanova and Danqi Chen.  Observed versus latent features for knowledge base and text inference.  In _Proceedings of the 3rd workshop on continuous vector space models and their compositionality_ , pages 57–66, 2015.
  * Trouillon et al. [2016] Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard.  Complex embeddings for simple link prediction.  In _ICML_ , pages 2071–2080, 2016.
  * Vashishth et al. [2020] Shikhar Vashishth, Soumya Sanyal, Vikram Nitin, and Partha Talukdar.  Composition-based multi-relational graph convolutional networks.  In _ICLR_ , 2020.
  * Vaswani et al. [2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  _NeurIPS_ , 30, 2017.
  * Vilar et al. [2014] Santiago Vilar, Eugenio Uriarte, Lourdes Santana, Tal Lorberbaum, George Hripcsak, Carol Friedman, and Nicholas P Tatonetti.  Similarity-based modeling in large-scale prediction of drug-drug interactions.  _Nature protocols_ , 9(9):2147–2163, 2014.
  * Wang et al. [2017] Quan Wang, Zhendong Mao, Bin Wang, and Li Guo.  Knowledge graph embedding: A survey of approaches and applications.  _IEEE TKDE_ , 29(12):2724–2743, 2017.
  * Wang et al. [2020] Yaqing Wang, Quanming Yao, James T Kwok, and Lionel M Ni.  Generalizing from a few examples: A survey on few-shot learning.  _CSUR_ , 53(3):1–34, 2020.
  * Wang et al. [2021] Yaqing Wang, Abulikemu Abuduweili, Quanming Yao, and Dejing Dou.  Property-aware relation networks for few-shot molecular property prediction.  _NeurIPS_ , 34:17441–17454, 2021.
  * Wishart et al. [2018] David S Wishart, Yannick D Feunang, An C Guo, Elvis J Lo, Ana Marcu, Jason R Grant, Tanvir Sajed, Daniel Johnson, Carin Li, Zinat Sayeeda, et al.  Drugbank 5.0: a major update to the drugbank database for 2018.  _Nucleic acids research_ , 46(D1):D1074–D1082, 2018.
  * Wu et al. [2024] Shiguang Wu, Yaqing Wang, and Quanming Yao.  PACIA: Parameter-efficient adapter for few-shot molecular property prediction.  In _IJCAI_ , 2024.
  * Wu et al. [2018] Zhenqin Wu, Bharath Ramsundar, Evan N Feinberg, Joseph Gomes, Caleb Geniesse, Aneesh S Pappu, Karl Leswing, and Vijay Pande.  MoleculeNet: A benchmark for molecular machine learning.  _Chemical Science_ , 9(2):513–530, 2018.
  * Xiong et al. [2017] Wenhan Xiong, Thien Hoang, and William Yang Wang.  Deeppath: A reinforcement learning method for knowledge graph reasoning.  In _EMNLP_ , pages 564–573, 2017.
  * Xu et al. [2021] Minghao Xu, Hang Wang, Bingbing Ni, Hongyu Guo, and Jian Tang.  Self-supervised graph-level representation learning with local and global structure.  In _ICML_ , pages 11548–11558, 2021.
  * Yang et al. [2015] Bishan Yang, Scott Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng.  Embedding entities and relations for learning and inference in knowledge bases.  In _ICLR_ , 2015.
  * Yao et al. [2024] Quanming Yao, Zhenqian Shen, Yaqing Wang, and Dejing Dou.  Property-aware relation networks for few-shot molecular property prediction.  _IEEE TPAMI_ , 2024.
  * Ying et al. [2018] Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L Hamilton, and Jure Leskovec.  Graph convolutional neural networks for web-scale recommender systems.  In _KDD_ , pages 974–983, 2018.
  * Yu et al. [2021] Yue Yu, Kexin Huang, Chao Zhang, Lucas M Glass, Jimeng Sun, and Cao Xiao.  Sumgnn: multi-typed drug interaction prediction via efficient knowledge graph summarization.  _Bioinformatics_ , 37(18):2988–2995, 2021.
  * Zhang and Yao [2022] Yongqi Zhang and Quanming Yao.  Knowledge graph reasoning with relational digraph.  In _WebConf_ , pages 912–924, 2022.
  * Zhang et al. [2020] Yongqi Zhang, Quanming Yao, and Lei Chen.  Interstellar: Searching recurrent architecture for knowledge graph embedding.  _NeurIPS_ , 33:10030–10040, 2020.
  * Zhang et al. [2022a] Yongqi Zhang, Quanming Yao, and James T Kwok.  Bilinear scoring function search for knowledge graph learning.  _IEEE TPAMI_ , 45(2):1458–1473, 2022a.
  * Zhang et al. [2022b] Yongqi Zhang, Zhanke Zhou, Quanming Yao, and Yong Li.  Efficient hyper-parameter search for knowledge graph embedding.  In _ACL_ , pages 2715–2735, 2022b.
  * Zhang et al. [2023a] Yongqi Zhang, Quanming Yao, Ling Yue, Xian Wu, Ziheng Zhang, Zhenxi Lin, and Yefeng Zheng.  Emerging drug interaction prediction enabled by a flow-based graph neural network with biomedical network.  _Nature Computational Science_ , 3(12):1023–1033, 2023a.
  * Zhang et al. [2023b] Yongqi Zhang, Zhanke Zhou, Quanming Yao, Xiaowen Chu, and Bo Han.  Adaprop: Learning adaptive propagation for graph neural network based knowledge graph reasoning.  In _SIGKDD_ , pages 3446–3457, 2023b.
  * Zhao et al. [2021] Huan Zhao, Quanming Yao, and Weiwei Tu.  Search to aggregate neighborhood for graph neural network.  In _ICDE_ , pages 552–563. IEEE, 2021.
  * Zhu et al. [2021] Zhaocheng Zhu, Zuobai Zhang, Louis-Pascal Xhonneux, and Jian Tang.  Neural bellman-ford networks: A general graph neural network framework for link prediction.  _NeurIPS_ , 34:29476–29490, 2021.
  * Zhu et al. [2024] Zhaocheng Zhu, Xinyu Yuan, Michael Galkin, Louis-Pascal Xhonneux, Ming Zhang, Maxime Gazeau, and Jian Tang.  A* net: A scalable path-based reasoning approach for knowledge graphs.  _NeurIPS_ , 36, 2024.


# Weak convergence of Monge-Ampère measures on compact Hermitian manifolds

Sławomir Kołodziej and Ngoc Cuong Nguyen  Faculty of Mathematics and Computer
Science, Jagiellonian University, Łojasiewicza 6, 30-348 Kraków, Poland
<EMAIL_ADDRESS>Department of Mathematical Sciences, KAIST,
291 Daehak-ro, Yuseong-gu, Daejeon 34141, South Korea<EMAIL_ADDRESS>
###### Abstract.

We give a sufficient condition on a sequence of uniformly bounded
$\omega$-plurisubharmonic functions, $\omega$ being a Hermitian metric, for
which the sequence of associated Monge-Ampère measures converges weakly. This
criterion can be used to obtained a bounded $\omega$-plurisubharmonic solution
to the Monge-Ampère equation.

##  1\. Introduction

Let $(X,\omega)$ be a compact Hermitian manifold of dimension $n$. The complex
Monge-Ampère equation on those manifolds has been studied extensively in
recent years. The classical solution in the smooth case was provided by
Tosatti-Weinkowe [TW10] (for $n=2$ the equation was solved earlier by Cherrier
[Ch87]). The weak solutions to the equation were studied in [DK12], [KN15,
KN18, KN19a, KN21a], [LPT21] and recent advances for the semi-positive
Hermitian form are contained in [GL21b]. The equation has found numerous
geometric applications [To15], [Ng16], [Di16], [Ni17], [Td18] [KT21] and
[GL21a].

To obtain the weak solutions one often uses the stability of potentials of the
approximating equations. It is well-known that the $L^{p}$-convergence of
potentials does not imply the weak convergence of corresponding Monge-Ampère
measures. Furthermore, in the Hermitian setting there are fewer such
criterions available as compared to the Kähler manifolds. In this note, we
prove such a criterion under suitable assumptions: of uniform boundedness and
domination by Monge-Ampère measures of another ”nice” sequence. Recall the
Bedford-Taylor capacity: for a Borel subset $E\subset X$,

$cap_{\omega}(E):=\sup\left\\{\int_{E}(\omega+dd^{c}w)^{n}:w\in
PSH(X,\omega),0\leq w\leq 1\right\\}.$

Here $PSH(X,\omega)$ denotes the set of all $\omega$-plurisubharmonic
($\omega$-psh) functions on $X$. A sequence
$\\{\varphi_{j}\\}_{j=1}^{\infty}\subset PSH(X,\omega)$ is said converge in
capacity to $\varphi\in PSH(X,\omega)$ if for a given $\varepsilon>0$,

$\lim_{j\to+\infty}cap_{\omega}(|\varphi_{j}-\varphi|>\varepsilon)=0.$

The main result of this expository note is as follows.

######  Theorem 1.1.

Let $\\{u_{j}\\}$ be a uniformly bounded sequence of $\omega$-psh functions.
Assume $(\omega+dd^{c}u_{j})^{n}\leq C(\omega+dd^{c}\varphi_{j})^{n}$ for some
uniformly bounded sequence $\\{\varphi_{j}\\}$ such that
$\varphi_{j}\to\varphi\in PSH(X,\omega)$ in capacity. If $u_{j}\to u\in
PSH(X,\omega)\cap L^{\infty}(X)$ in $L^{1}(X)$, then a subsequence of
$(\omega+dd^{c}u_{j})^{n}$ converges weakly to $(\omega+dd^{c}u)^{n}$.

This is a generalization of [CK06, Lemma 2.1] from the local setting to
compact Hermitian manifolds that is pointed out in the proof of [KN23b, Lemma
2.11] (see also [KN23a]). On compact Käher manifolds there are stronger
results [H08, Theorem 2.1] and [DH12]. We will see that if the dimension
$n=2$, then we have a similar result (Proposition 2.5). However, we do not
know if it still holds for dimensions $n\geq 3$. It is also possible to extend
the theorem to the case of Hermitian semi-positive $(1,1)$-forms $\alpha$ and
we will consider it in a future paper.

An application of the theorem is that it provides a shorter proof of the
existence of bounded $\omega$-psh solutions to Monge-Ampère equations with the
right hand side in $L^{p}(X)$, $p>1$.

######  Corollary 1.2.

Let $0\leq f\in L^{p}(X)$ and $p>1$. Then, there exists a bounded function
$u\in PSH(X,\omega)\cap L^{\infty}(X)$ and a constant $c>0$ solving
$\omega_{u}^{n}=cf\omega^{n}.$

###### Proof.

Approximating $f_{j}\to f$ by smooth positive function $f_{j}$. By the
Tosatti-Weinkove theorem [TW10] there are $u_{j}\in PSH(X,\omega)\cap
C^{\infty}(X)$ and $c_{j}>0$ solving

$(\omega+dd^{c}u_{j})^{n}=c_{j}f_{j}\omega^{n},\quad\sup_{X}u_{j}=0.$

Using [KN15, Eq. (5.12)] we have $c_{j}\leq C_{0}$ with a uniform $C_{0}>0$.
Therefore, by [DK12] we have $-C_{1}\leq u_{j}\leq 0$. By passing to a
subsequence we may assume that $u_{j}\to u$ in $L^{1}(X)$ and $c_{j}\to c\geq
0$. Then,

$\int_{X}|u_{j}-u|(\omega+dd^{c}u_{j})^{n}=\int_{X}|u-u_{j}|c_{j}f_{j}\omega^{n}\leq
C\|u_{j}-u\|_{L^{1}(X)}^{\frac{1}{q}},$

where $1/p+1/q=1$. Hence, by Lemma 2.4 (below) there is a subsequence of
$\\{u_{j}\\}$, for simplicity still denoted by $\\{u_{j}\\}$, such that
$\omega_{u_{j}}^{n}\to\omega_{u}^{n}=cf\omega^{n}$ weakly. Note that since $u$
is bounded, it follows from [KN15, Remark 5.7] that $c>0$. ∎

Another application is proving the existence of bounded solutions for a more
general class of measures. A positive Radon measure $\mu$ on $X$ is well
dominated by capacity or belong to $\mathcal{F}(X,h)$ (see [Ko05]) if

(1.1) $\mu(E)\leq F_{h}(cap_{\omega}(E)),$

for any Borel set $E\subset X$, where $F_{h}(x)=x/h(x^{-\frac{1}{n}})$ for
some increasing function $h:\mathbb{R}_{+}\rightarrow(0,\infty)$ satisfying

(1.2) $\int_{1}^{\infty}\frac{1}{x[h(x)]^{\frac{1}{n}}}\,dx<+\infty.$

######  Corollary 1.3.

Let $\mu$ be a positive Radon measure with $\mu(X)>0$. Assume that $\mu$ is
well dominated by capacity. Then, there exists a bounded solution $u\in
PSH(X,\omega)$ and $c>0$ solving $\omega_{u}^{n}=c\mu.$

###### Proof.

It follows easily from the proof of [KN23a, Theorem 3.1]. ∎

######  Remark 1.4.

The solutions obtained in Corollary 2.7 and Corollary 2.8 are Hölder
continuous and continuous, respectively. However, to get these statements, we
need a stronger stability of solutions as in [KN15, KN21a].

Acknowledgement. The first author is partially supported by grant no.
2021/41/B/ST1/01632 from the National Science Center, Poland. The second
author is partially supported by the start-up grant G04190056 of KAIST and the
National Research Foundation of Korea (NRF) grant no. 2021R1F1A1048185.

##  2\. Proof of Theorem 1.1 and convergence in capacity

This section is devoted to prove the main theorem. Since all functions under
consideration are uniformly bounded, by normalizing

(2.1) $\sup_{X}u_{j}=\sup_{X}\varphi_{j}=0,$

they belong to the subset $\mathcal{P}_{0}$ of $PSH(X,\omega)$ given by

$\mathcal{P}_{0}=\left\\{v\in PSH(X,\omega)\cap
L^{\infty}(X):\sup_{X}v=0\right\\}.$

Let us denote $dV=\omega^{n}$ the volume form of $X$. We have the following
result essentially due to Cegrell [Ce98], a detailed proof is given in [KN23a,
Lemma 2.1].

######  Lemma 2.1.

Let $d\lambda$ be a finite positive Radon measure on $X$ vanishing on
pluripolar sets. Suppose moreover that $\\{u_{j}\\}\subset\mathcal{P}_{0}$
converges $dV$-a.e. to $u\in\mathcal{P}_{0}$. Then there exists a subsequence
$\\{u_{j_{s}}\\}\subset\\{u_{j}\\}$ such that

$\lim_{j_{s}\to+\infty}\int_{X}u_{j_{s}}d\lambda=\int_{X}ud\lambda.$

An immediate consequence is

######  Corollary 2.2.

There exists a subsequence, still denoted by $\\{u_{j}\\}$, such that

$\lim_{j\to\infty}\int_{X}|u_{j}-u|d\lambda=0.$

###### Proof.

Applying Lemma 2.1 twice to the sequences $\\{u_{j}\\}$ and
$\max\\{u_{j},u\\}$, we have (still denoting by $\\{u_{j}\\}$ the resulting
subsequence)

$\lim_{j\to\infty}\int_{X}u_{j}d\lambda=\int_{X}ud\lambda,\quad\lim_{j\to\infty}\int_{X}\max\\{u_{j},u\\}d\lambda=\int_{X}ud\lambda.$

Since $\max\\{u,u_{j}\\}=(u_{j}+u+|u_{j}-u|)/2$, integrating both sides and
using the previous equations we get that $u_{j}\to u$ in $L^{1}(d\lambda).$ ∎

The next result is a global analogue of [KN23b, Lemma 2.3] which says that
under the assumption of Theorem 1.1, the sequence converges uniformly in
$L^{1}$-norm with respect to a family of measures. On compact complex
manifolds the Cegrell inequality will not be needed. For the reader
convenience, we give details of the proof.

######  Lemma 2.3.

Let $\\{w_{j}\\}_{j=1}^{\infty}\subset\mathcal{P}_{0}$ be a uniformly bounded
sequence that converges in capacity to $w\in\mathcal{P}_{0}$. Assume
$\sup_{j}\int_{X}(\omega+dd^{c}w_{j})^{n}\leq C_{1}$ for some $C_{1}>0.$ Then,

$\lim_{j\to\infty}\int_{X}|u-u_{j}|(\omega+dd^{c}w_{j})^{k}\wedge\omega^{n-k}=0\quad\text{for
}k=0,1,...,n.$

###### Proof.

The case $k=0$ is the assumption on $\\{u_{j}\\}$, and we only give here the
proof of the last inductive step, the other steps are the very similar. In the
proof that follows the constant $C>0$ is a uniform constant depending only on
$X,\omega$ and the uniform bounds for $\|u_{j}\|_{L^{\infty}}$ and
$\|w_{j}\|_{L^{\infty}}$, it may change from line to line.

Note that $|u-u_{j}|=(\max\\{u,u_{j}\\}-u_{j})+(\max\\{u,u_{j}\\}-u)$. By
quasi-continuity of $\omega$-psh functions and Hartogs’ lemma, we have
$\phi_{j}:=\max\\{u,u_{j}\\}\to u$ in capacity. Fix $\varepsilon>0$. Then,
when $j$ is large,

$\displaystyle\int_{X}(\max\\{u,u_{j}\\}-u)(\omega+dd^{c}w_{j})^{n}$
$\displaystyle\leq\int_{\\{|\phi_{j}-u|>\varepsilon\\}}(\omega+dd^{c}w_{j})^{n}+\varepsilon\int_{X}(\omega+dd^{c}w_{j})^{n}$
$\displaystyle\leq Ccap_{\omega}(|\phi_{j}-u|>\varepsilon)+C_{1}\varepsilon.$

Therefore, $\lim_{j\to\infty}\int_{X}(\phi_{j}-u)(\omega+dd^{c}w_{j})^{n}=0$.
Next, we have for $j>k$,

$\int_{X}(\phi_{j}-u_{j})(\omega+dd^{c}w_{j})^{n}-\int_{X}(\phi_{j}-u_{j})(\omega+dd^{c}w_{k})^{n}=\int_{X}(\phi_{j}-u_{j})dd^{c}(w_{j}-w_{k})\wedge
T,$

where
$T=T(j,k)=\sum_{s=1}^{n-1}\omega_{w_{j}}^{s}\wedge\omega_{w_{k}}^{n-1-s}$. By
integration by parts,

$\displaystyle\int_{X}(\phi_{j}-u_{j})dd^{c}(w_{j}-w_{k})\wedge T$
$\displaystyle=\int_{X}(w_{j}-w_{k})dd^{c}\left[(\phi_{j}-u_{j})\wedge
T\right].$

Note that for $h=\phi_{j}-u_{j}$,

$\displaystyle dd^{c}(hT)$ $\displaystyle=dd^{c}h\wedge T+dh\wedge
d^{c}T-d^{c}h\wedge dT+hdd^{c}T$ $\displaystyle=dd^{c}h\wedge T+dh\wedge
d^{c}\omega\wedge T_{1}-d^{c}h\wedge d\omega\wedge T_{2}$
$\displaystyle=:S_{0}+S_{1}+S_{2}.$

Here notice that $T_{1}$ and $T_{2}$ are positive currents, and similar type
as of $T$.

We now estimate each term $S_{0},S_{1}$ and $S_{2}$ separately. Firstly, for
the term $S_{0}$,

$\int_{X}(w_{j}-w_{k})dd^{c}(\phi_{j}-u_{j})\wedge
T\leq\int_{X}|w_{j}-w_{k}|(\omega_{\phi_{j}}+\omega_{u_{j}})\wedge T.$

Since $\|w_{j}\|_{\infty},\|u_{j}\|_{\infty}\leq A$ in $X$, it follows that

$\displaystyle\int_{X}|w_{j}-w_{k}|(\omega_{\phi_{j}}+\omega_{u_{j}})\wedge T$
$\displaystyle\leq
A\int_{\\{|w_{j}-w_{k}|>\varepsilon\\}}(\omega_{\phi_{j}}+\omega_{u_{j}})\wedge
T$
$\displaystyle\quad+\varepsilon\int_{\\{|w_{j}-w_{k}|\leq\varepsilon\\}}(\omega_{\phi_{j}}+\omega_{u_{j}})\wedge
T$ $\displaystyle\leq
A^{n+1}cap_{\omega}(|w_{j}-w_{k}|>\varepsilon)+C\varepsilon,$

where the uniform bound for the second integral on the right hand side follows
from uniform boundedness of the potentials (see. e.g., [DK12, Proposition
2.3]). Since $w_{j}\to w$ in capacity, it follows that there exists $k_{0}$
such that for every $j>k\geq k_{0}$ the left hand side is less than
$2C\varepsilon$.

Secondly, for the term $S_{1}$, by the Cauchy-Schwarz inequality [Ng16,
Proposition 1.4] in the Hermitian setting,

$\displaystyle\left|\int_{X}(w_{j}-w_{k})dh\wedge d^{c}\omega\wedge
T_{1}\right|^{2}$ $\displaystyle\leq C\int_{X}|w_{j}-w_{k}|dh\wedge
d^{c}h\wedge\omega\wedge T_{1}\int_{X}|w_{j}-w_{k}|\omega^{2}\wedge T_{1}$
$\displaystyle\leq C\int_{X}|w_{j}-w_{k}|\omega^{2}\wedge T_{1}.$

Therefore, by a similar argument as in the first case for the integral on the
right hand side is bounded by $C\varepsilon$ for every $j>k\geq k_{0}$ (we may
increase $k_{0}$ if necessary).

Lastly, the term $S_{2}$ is estimated similarly as $S_{1}$. Thus,

$\displaystyle\int_{X}(\phi_{j}-u_{j})(\omega+dd^{c}w_{j})^{n}$
$\displaystyle\leq\int_{X}(\phi_{j}-u_{j})(\omega+dd^{c}w_{k})^{n}$
$\displaystyle\quad+\left|\int_{X}(\phi_{j}-u_{j})\omega_{w_{j}}^{n}-\int_{X}(\phi_{j}-u_{j})\omega_{w_{k}}^{n}\right|$
$\displaystyle\leq\int_{X}(\phi_{j}-u_{j})(\omega+dd^{c}w_{k})^{n}+2C\varepsilon$
$\displaystyle\leq\int_{X}|u-u_{j}|(\omega+dd^{c}w_{k})^{n}+2C\varepsilon.$

Fixing $k=k_{0}$ and applying Corollary 2.2 for
$d\lambda=(\omega+dd^{c}w_{k_{0}})^{n}$, we get that

$\int_{X}(\phi_{j}-u_{j})(dd^{c}w_{j})^{n}\leq(2C+1)\varepsilon\quad\text{for
}j\geq k_{1}\geq k_{0}.$

Since $\varepsilon>0$ is arbitrary, the proof of the lemma is completed. ∎

The proof of Theorem 1.1 is an immediate consequence of Lemma 2.3 and the
following weak convergence.

######  Lemma 2.4.

Suppose that

(2.2) $\lim_{j\to+\infty}\int_{X}|u_{j}-u|\omega_{u_{j}}^{n}=0.$

Then, there exists a subsequence $\\{u_{j_{s}}\\}$ of $\\{u_{j}\\}$ such that
$\omega_{u_{j_{s}}}^{n}$ converges to $\omega_{u}^{n}$ weakly.

###### Proof.

Let $A>0$ be such that $-A\leq u_{j},u\leq 0$. By passing to a subsequence we
may assume further that $u_{j}\to u$ a.e. in $X$ with respect to $dV$. Note
that

$u=(\limsup_{j\to\infty}u_{j})^{*}=\lim_{j\to\infty}(\sup_{\ell\geq
j}u_{\ell})^{*}.$

Set

(2.3) $w_{j}=\max\\{u_{j},u-1/j\\}.$

By the Hartogs lemma $w_{j}$ converges to $u$ in capacity. Therefore, by the
convergence theorem in [DK12] (see also [BT82]),
$\lim_{j\to\infty}\omega_{w_{j}}^{n}=\omega_{u}^{n}.$ Thanks to Lemma 2.3 and
the assumption $w_{j}\to u$ in capacity, we have

(2.4) $\int_{X}|u_{j}-u|(\omega+dd^{c}w_{j})^{n}\to 0\quad\text{as
}j\to+\infty.$

Now we are ready to conclude the proof of the lemma. By (2.2) and (2.4) we can
choose a subsequence $\\{u_{j_{s}}\\}\subset\\{u_{j}\\}$ so that

$\int_{X}|u-u_{j_{s}}|(\omega+dd^{c}u_{j_{s}})^{n}+\int_{X}|u-u_{j_{s}}|(\omega+dd^{c}w_{s})^{n}<1/s^{2}.$

Recall from (2.3) that $w_{s}=\max\\{u_{j_{s}},u-1/s\\}$ and this implies

${\bf 1}_{\\{u_{j_{s}}>u-1/s\\}}(\omega+dd^{c}w_{s})^{n}={\bf
1}_{\\{u_{j_{s}}>u-1/s\\}}(\omega+dd^{c}u_{j_{s}})^{n}.$

Therefore, for $\eta\in C^{\infty}(X)$,

$\displaystyle\left|\int_{X}\eta\omega_{u}^{n}-\int_{X}\eta\omega_{u_{j_{s}}}^{n}\right|$
$\displaystyle\leq\left|\int_{X}\eta\omega_{u}^{n}-\int_{X}\eta\omega_{w_{s}}^{n}\right|+\left|\int_{X}\eta\omega_{w_{s}}^{n}-\int_{X}\eta\omega_{u_{j_{s}}}^{n}\right|$
$\displaystyle\leq\left|\int_{X}\eta\omega_{u}^{n}-\int_{X}\eta\omega_{w_{s}}^{n}\right|+\left|\int_{\\{u_{j_{s}}\leq
u-1/s\\}}\eta\omega_{w_{s}}^{n}-\eta\omega_{u_{j_{s}}}^{n}\right|.$

The first term on the right hand side goes to zero as
$\omega_{w_{s}}^{n}\to\omega_{u}^{n}$. It remains to estimate the second term.
Firstly, by the choice of $\\{u_{j_{s}}\\}$ at the beginning of this proof,

$\displaystyle\left|\int_{\\{u_{j_{s}}\leq
u-1/s\\}}\eta\omega_{u_{j_{s}}}^{n}\right|$
$\displaystyle\leq\|\eta\|_{L^{\infty}}\int_{\\{u_{j_{s}}\leq
u-1/s\\}}\omega_{u_{j_{s}}}^{n}$ $\displaystyle\leq
s\|\eta\|_{L^{\infty}}\int_{X}|u-u_{j_{s}}|\omega_{u_{j_{s}}}^{n}\leq\frac{1}{s}\|\eta\|_{L^{\infty}}\to
0\quad\text{as }s\to+\infty.$

Similarly,

$\displaystyle\left|\int_{\\{u_{j_{s}}\leq
u-1/s\\}}\eta\omega_{w_{s}}^{n}\right|$
$\displaystyle\leq\|\eta\|_{L^{\infty}}\left|\int_{\\{u_{j_{s}}\leq
u-1/s\\}}\eta\omega_{w_{s}}^{n}\right|$ $\displaystyle\leq
s\|\eta\|_{L^{\infty}}\int_{X}|u-u_{j_{s}}|\omega_{w_{s}}^{n}\to
0\quad\text{as }s\to+\infty.$

The last two estimates complete the proof of the lemma. This also completed
the proof of Theorem 1.1. ∎

We shall have a similar criterion for convergence in capacity [KN23a,
Proposition 2.8]. Let us recall [DK12, Theorem 3.5] and [KN15, Lemmas 2.1,
2.2]. Let $B>0$ be a uniform constant satisfying

$\displaystyle-B\omega^{2}\leq 2ndd^{c}\omega\leq B\omega^{2},$
$\displaystyle-B\omega^{3}\leq 4n^{2}d\omega\wedge d^{c}\omega\leq
B\omega^{3}.$

Then, for $\varphi,\psi\in PSH(X,\omega)\cap L^{\infty}(X)$ satisfying
$\sup_{\\{\varphi<\psi\\}}(\psi-\varphi)\leq 1$,

(2.5) $\displaystyle\int_{\\{\varphi<\psi\\}}\omega_{\psi}^{n}$
$\displaystyle\leq\int_{\\{\varphi<\psi\\}}\omega_{\varphi}^{n}+C_{n}\max\\{1,B\\}^{n}\sum_{k=0}^{n}\int_{\\{\psi<\varphi\\}}\omega_{\varphi}^{k}\wedge\omega^{n-k},$

where $C_{n}$ is a dimensional constant. If the dimension $n=2$, then we have
a better estimate, namely,

(2.6) $\displaystyle\int_{\\{\varphi<\psi\\}}\omega_{\psi}^{2}$
$\displaystyle\leq\int_{\\{\varphi<\psi\\}}\omega_{\varphi}^{2}+C_{n}\max\\{1,B\\}\int_{\\{\psi<\varphi\\}}\omega^{2},$

######  Proposition 2.5.

Let $\\{u_{j}\\}$ be the sequence in Theorem 1.1. Then, $u_{j}$ converges to
$u$ in capacity if and only if

(2.7)
$\lim_{j\to+\infty}\int_{X}|u_{j}-u|\omega_{u_{j}}^{k}\wedge\omega^{n-k}=0,\quad
k=0,...,n.$

In particular, if $n=2$, then the sequence converges in capacity.

###### Proof.

Suppose that $u_{j}\to u$ in capacity. It follows from Lemma 2.3 that (2.7)
holds for every $k=0,1,...,n$. Conversely, suppose that (2.7) holds true. Let
$\varepsilon>0$ be fixed and without loss of generality we may assume that
$-1\leq u_{j},u\leq 0.$ We have

$cap_{\omega}(|u_{j}-u|>\varepsilon)\leq
cap_{\omega}(u_{j}-u>\varepsilon)+cap_{\omega}(u-u_{j}>\varepsilon).$

By Hartogs’ lemma, $\max\\{u_{j},u\\}\to u$ in capacity. Hence,

$\lim_{j\to+\infty}cap_{\omega}(u_{j}-u>\varepsilon)\leq\lim_{j\to+\infty}cap_{\omega}(\max\\{u_{j},u\\}-u>0)=0.$

It remains to prove that $cap_{\omega}(u-u_{j}>\varepsilon)\to 0$ as
$j\to+\infty$. Let $-1\leq\rho\leq 0$ be a function in $PSH(X,\omega)$. Then,

$\\{u_{j}<u-\varepsilon\\}\subset\left\\{u_{j}<(1-\frac{\varepsilon}{4})u+\frac{\varepsilon}{4}\rho-\frac{3\varepsilon}{4}\right\\}\subset\left\\{u_{j}<u-\frac{\varepsilon}{2}\right\\}$

Applying (2.5) for $\varphi=u_{j}$ and
$\psi=(1-\varepsilon/4)u+\varepsilon\rho/4-3\varepsilon/4$ and then using the
previous inclusions for domains of integration on the left hand side and on
the right hand side, we have

$\displaystyle\int_{\\{u_{j}<u-\varepsilon\\}}\left[\omega+(1-\frac{\varepsilon}{4})dd^{c}u+\frac{\varepsilon}{4}dd^{c}\rho\right]^{n}$
$\displaystyle\leq\int_{\\{u_{j}<u-\varepsilon/2\\}}\omega_{u_{j}}^{n}+C_{n}\max\\{1,B\\}^{n}\sum_{k=0}^{n}\int_{\\{u_{j}<u-\varepsilon/2\\}}\omega_{u_{j}}^{k}\wedge\omega^{n-k},$

Since the integrand on the left hand side is larger than
$(\varepsilon/4)^{n}\omega_{\rho}^{n}$, it follows that

(2.8)
$\displaystyle\left(\frac{\varepsilon}{4}\right)^{n}cap_{\omega}(u_{j}<u-\varepsilon)$
$\displaystyle\leq\int_{\\{u_{j}<u-\varepsilon/2\\}}\omega_{u_{j}}^{n}$
$\displaystyle\quad+C_{n}\max\\{1,B\\}^{n}\sum_{k=0}^{n}\int_{\\{u_{j}<u-\varepsilon/2\\}}\omega_{u_{j}}^{k}\wedge\omega^{n-k}.$

Notice that for $k=0,...,n$ we have

$\int_{\\{u_{j}<u-\varepsilon/2\\}}\omega_{u_{j}}^{k}\wedge\omega^{n-k}\leq\frac{2}{\varepsilon}\int_{X}|u_{j}-u|\omega_{u_{j}}^{k}\wedge\omega^{n-k}.$

Combining this inequality with (2.7) and (2.8) we obtain
$\lim_{j\to+\infty}cap_{\omega}(u_{j}<u-\varepsilon)=0$. The proof is
completed.

Finally let us finish the proof in the case $n=2$. Indeed, by (2.6) the
inequality corresponding to (2.8) reads

$\left(\frac{\varepsilon}{4}\right)^{2}cap_{\omega}(\\{u_{j}<u-\varepsilon\\})\leq\int_{\\{u_{j}<u-\varepsilon/2\\}}\omega_{u_{j}}^{2}+C_{n}\max\\{1,B\\}\int_{\\{u_{j}<u-\varepsilon/2\\}}\omega^{2}.$

The right hand side is bounded by

$\frac{2}{\varepsilon}\int_{X}|u_{j}-u|\omega_{u_{j}}^{2}+\frac{2C_{n}}{\varepsilon}\max\\{1,B\\}\int_{X}|u_{j}-u|\omega^{2},$

which tends to zero as $j$ goes to infinity. Since $\varepsilon>0$ is fixed we
conclude the convergence in capacity in dimension 2. ∎

## References

  * [BT82] E. Bedford and B. A. Taylor, A new capacity for plurisubharmonic functions, Acta Math. 149 (1982) 1–40.
  * [Ce98] U. Cegrell, Pluricomplex energy, Acta Math. 180:2 (1998) 187-217.
  * [CK06] U. Cegrell and S. Kołodziej, The equation of complex Monge-Ampère type and stability of solutions, Math. Ann. 334 (2006), no. 4, 713–729.
  * [Ch87] P. Cherrier, Équations de Monge-Ampère sur les variétés Hermitiennes compactes. Bull. Sci. Math. 111 (2) (1987) 343–385.
  * [Di16] S. Dinew, Pluripotential theory on compact Hermitian manifolds, Ann. Fac. Sci. Toulouse Math. (6) 25 (2016), no. 1, 91–139.
  * [DH12] S. Dinew and P.-H. Hiep, Convergence in capacity on compact Kähler manifolds, Ann. Sc. Norm. Super. Pisa Cl. Sci. (5) 11 (2012), no. 4, 903–919.
  * [DK12] S. Dinew and S. Kołodziej, Pluripotential estimates on compact Hermitian manifolds. Adv. Lect. Math. (ALM), 21 (2012) International Press, Boston.
  * [GL21a] V. Guedj, C.-H. Lu, Quasi-plurisubharmonic envelopes 2: Bounds on Monge-Ampère volumes, Preprint, arXiv: 2106.04272. to appear in J. Algebraic Geometry.
  * [GL21b] V. Guedj, C.-H. Lu, Quasi-plurisubharmonic envelopes 3: Solving Monge - Ampère equations on hermitian manifolds, arXiv: 2107.01938.
  * [H08] P.-H. Hiep, On the convergence in capacity on compact Kahler manifolds and its applications, Proc. Amer. Math. Soc. 136 (2008), no. 6, 2007–2018.
  * [Ko05] S. Kołodziej, The complex Monge-Ampère equation and pluripotential theory. Memoirs Amer. Math. Soc. 178 (2005) pp. 64.
  * [KN15] S. Kołodziej and N.-C. Nguyen, Weak solutions to the complex Monge-Ampère equation on Hermitian manifolds. Analysis, complex geometry, and mathematical physics: in honor of Duong H. Phong, Contemporary Mathematics, vol. 644 (American Mathematical Society, Providence, RI, 2015) 141-158.
  * [KN18] S. Kołodziej and N.-C. Nguyen, Hölder continuous solutions of the Monge-Ampère equation on compact Hermitian manifolds, Ann. Inst. Fourier (Grenoble) 68 (2018) 2951–2964.
  * [KN19a] S. Kołodziej and N.-C. Nguyen, Stability and regularity of solutions of the Monge-Ampère equation on Hermitian manifolds, Adv. Math. 346 (2019) 264–304.
  * [KN21a] S. Kołodziej and N.-C. Nguyen, Continuous solutions to Monge-Ampère equations on Hermitian manifolds for measures dominated by capacity, Calc. Var. Partial Differential Equations 60 (2021) Paper No. 93, 18 pp.
  * [KN23a] S. Kołodziej and N.-C. Nguyen,The Dirichlet problem for the Monge–Ampère equation on Hermitian manifolds with boundary, Calc. Var. Partial Differential Equations 62 (2023), no. 1, Paper No. 1.
  * [KN23b] S. Kołodziej and N.-C. Nguyen, Weak solutions to Monge–Ampère type equations on compact Hermitian manifold with boundary, J. Geom. Anal. 33 (2023), no. 1, Paper No. 15.
  * [KT21] S. Kołodziej and V. Tosatti, Morse-type integrals on non-Kähler manifolds, Pure Appl. Math. Q. 17 (2021), no. 3, 991–1004.
  * [LPT21] C. H. Lu, T.-T. Phung and T.-D. Tô, Stability and Hölder regularity of solutions to complex Monge-Ampère equations on compact Hermitian manifolds, Ann. Inst. Fourier (Grenoble) 71 (2021), no. 5, 2019–2045.
  * [Ni17] X. Nie, Weak solution of the Chern-Ricci flow on compact complex surfaces, Math. Res. Lett. 24 (2017) 1819–1844.
  * [Ng16] N.-C. Nguyen, The complex Monge-Ampère type equation on compact Hermitian manifolds and Applications. Adv. Math. 286 (2016) 240-285.
  * [Td18] T.-D. Tô, Regularizing properties of complex Monge-Ampère flows II: Hermitian manifolds, Math. Ann. 372 (2018), no. 1-2, 699–741.
  * [To15] V. Tosatti, Non-Kähler Calabi-Yau manifolds, in Analysis, complex geometry, and mathematical physics: in honor of Duong H. Phong, 261–277, Contemp. Math., 644, Amer. Math. Soc., Providence, RI.
  * [TW10] V. Tosatti and B. Weinkove, The complex Monge-Ampère equation on compact Hermitian manifolds. J. Amer. Math. Soc. 23 (2010) 1187–1195.