See pages 1-last of
Alternating_Optimization_for_Capacity_Region_of_Gaussian_MIMO_BC_with_PAPC.pdf


# Optimal Regulation of Prosumers and Consumers in Smart Energy Communities

Syed Eqbal Alam and Dhirendra Shukla∗ ∗Dhirendra Shukla is also affiliated
with Gray Wolf Analytics, Fredericton, New Brunswick, Canada. J. Herbert Smith
Centre for Technology Management & Entrepreneurship
University of New Brunswick
Fredericton, New Brunswick, Canada

###### Abstract

In a smart energy community, energy prosumers and consumers group together to
achieve the community’s social welfare. Prosumers are the users that both
consume and produce energy. In this paper, we develop algorithms to regulate
the number of prosumers and the number of consumers in the smart energy
community. We consider that the prosumers have heterogeneous energy sources,
such as solar photovoltaic panels and wind turbines. Each prosumer has one of
the systems installed in their household. The prosumers and the consumers keep
their information private and do not share it with other prosumers or
consumers in the community. However, we consider a community manager that
keeps track of the total number of active prosumers and consumers and sends
feedback signals in the community at each time step. Over a long time, the
average number of times a prosumer is active reaches its optimal value;
analogously, the average number of times a consumer is active reaches its
optimal value and the community achieves the social optimum value. We present
the experimental results to check the efficacy of the algorithms.

###### Keywords:

Distributed optimization, optimal control, energy trading, energy producer,
energy consumer, smart city, energy communities.

##  I Introduction

Smart energy communities have attracted significant interest from researchers
recently. In smart energy communities, community members group together to
achieve a common goal. The smart energy community utilizes the existing local
energy infrastructures and enables community members’ participation,
cooperation, and coordination for the community’s welfare. In addition, the
smart energy communities facilitate balancing local energy production and
consumption and also facilitate balancing community members’ needs and
preferences fairly and help keep monetary benefit within the community. These
can help in becoming an energy community self-sustainable [1], [2], [3].
Furthermore, it can help in achieving a cleaner and more affordable energy.
Roughly speaking, smart energy communities can facilitate social interaction
and build relationships among the community members that are unavailable in
the traditional energy systems.

In smart energy communities, one of the technological challenges is to develop
mechanisms to handle the uncertainty in energy production and consumption
patterns. Our work is a small contribution towards handling these challenges.
The smart energy community consists of energy prosumers and consumers. Energy
prosumers are the consumers that can also produce energy. For example,
households connected to the power grid and also have solar photovoltaic panels
on their rooftops to produce energy locally. Moreover, in a smart energy
community, community members collaborate to achieve the community’s overall
welfare. The prosumers can sell the surplus produced energy in real-time to
community members or a grid or store energy in batteries (to build a community
energy storage). They may sell the surplus produced energy to other community
members for monetary benefits or can also participate in philanthropic works
and help in handling social challenges such as energy poverty. Energy poverty
is the lack of sufficient energy for primary human activities. In particular,
the members can support specially-abled people or improve the local
environment by using sustainable energy sources. These can have an impact on
the prosperity of the society and can help in achieving net-zero emission
targets, too [4].

Now let us consider that a smart energy community consists of households that
are energy prosumers and consumers. The community members make a cooperative
group and aim to achieve the community’s social welfare. Furthermore, we
consider that the prosumers have heterogeneous renewable energy sources, some
prosumers install solar panels, and others install wind turbines in their
households. The prosumers provide excess produced energy to other community
members, the consumers. Note that one of the challenges in deploying renewable
energy sources is the local weather conditions. The local weather affects the
energy production by renewable sources—when the energy sources are
homogeneous, this effect is more. Nevertheless, we can reduce the effect by
using heterogeneous energy sources. In this paper, we developed algorithms
that regulate the number of prosumers and the number of consumers in a smart
energy community with heterogeneous energy sources to minimize the overall
cost to the community. The algorithms are motivated by the ideas from adaptive
control and stochastic approximation [5]. [6] and [7] are the starting point
of our work. Each prosumer and each consumer has a cost function that depends
on the average number of times a prosumer or a consumer actively participates.
They participate in the community using a probabilistic rule. Our aim is that
the number of times a prosumer and a consumer are active achieve optimal
values over long-term averages. In addition, the prosumers and consumers do
not need to share their information—such as the cost functions or derivatives
of the cost functions. They also do not need to share the probabilistic rule
they use in deciding when to share the produced energy or to consume energy.
Furthermore, in the smart energy community, a community manager keeps track of
the aggregate number of active prosumers and consumers at a time step. The
community manager updates and broadcasts a feedback signal in the community at
each time step based on the total number of active prosumers and consumers.
After receiving the feedback signal, a prosumer decides whether to participate
or not in a probabilistic way. The response probability mainly depends on the
feedback signal, the derivative of the cost function, and the average number
of times a prosumer or a consumer was active. Following the algorithms, the
prosumers and consumers obtain optimal values on long-term averages to achieve
minimum overall cost to the energy community.

##  II Related work

This section briefly presents the recent works on smart energy communities.
Savelli and Morstyn proposed the idea of smart energy neighborhoods wherein
the community members utilize the local existing energy infrastructures and
coordinate and cooperate to achieve a social good for the community [4]. Chen
et al. proposed a peer-to-peer energy sharing platform for the dynamic network
that reduces the power losses to the network [8]. Furthermore, peer-to-peer
energy sharing and trading were also studied in [9]. Werner and co-authors
studied the pricing flexibility in energy markets in [10]. Peer-to-peer energy
transaction fairness mechanisms are proposed for uncertainties in [11] that
maximize the social welfare of the smart energy communities. In another
direction, a reinforcement learning-based mechanism is proposed for peer-to-
peer energy trading in smart energy communities [12]. Bhattacharjee and Nandi
developed a voting-based mechanism for supply and demand balancing in energy
communities [13] with heterogeneous renewable energy sources—solar, wind, and
biomass. For blockchain based peer to peer energy trading systems, readers can
refer to [14], [15], [16], [17] and for energy trading with distributed
ledgers to [18]. Finally, readers may refer to [19], [20], [21], [22], and
[23] for a detailed survey on smart energy communities.

##  III Preliminaries

Let us suppose that energy prosumers and consumers make a community; moreover,
let in the community $N$ prosumers have solar panels and $M$ prosumers have
wind turbines to produce energy. The prosumers sell the excess energy produced
to the consumers in the community or share the excess energy produced with
needy consumers in the community as philanthropic work. Let $U$ be the number
of energy consumers in the community. Let us now consider that the smart
energy community has a community manager that overseas the number of active
prosumers and consumers in the energy community. Every prosumer with a solar
panel has a private cost function that depends on the average number of times
the prosumer was active in sharing the produced solar energy. Let the total
desired number of prosumers with solar panels, or the capacity, be
$C_{\textnormal{s}}$ and the desired number of prosumers with wind turbines be
$C_{\textnormal{w}}$ in the community.

Let $\xi_{i}(k)\in\\{0,1\\}$ denote the prosumer $i$ with solar panel is
active at time step $k$ or not. When the prosumer $i$ with solar panel is
active then $\xi_{i}(k)=1$ is updated; otherwise, $\xi_{i}(k)=0$. Let
$\eta_{j}(k)\in\\{0,1\\}$ denote the prosumer $j$ with wind turbine is active
at time step $k$ or not. When the prosumer $j$ with wind turbines is active
then $\eta_{j}(k)=1$ is updated; otherwise, $\eta_{j}(k)=0$. Furthermore, let
$x_{i}(k)\in[0,1]$ denote the average of the number of times prosumer $i$ with
solar panels was active until time step $k$, and let $y_{j}(k)\in[0,1]$ denote
the average of the number of times prosumer $j$ with wind turbine was active
until time step $k$. We define the average for prosumer $i$ with solar panel
by $x_{i}(k)$, for $i=1,2,\ldots,N$, as follows:

$\displaystyle{x}_{i}(k)\triangleq\frac{1}{k+1}\sum_{\ell=0}^{k}\xi_{i}(\ell),\quad
i=1,2,\ldots,N.$ (1)

We define the average for prosumer $j$ with wind turbine $y_{j}(k)$ as
follows:

$\displaystyle{y}_{j}(k)\triangleq\frac{1}{k+1}\sum_{\ell=0}^{k}\eta_{j}(\ell),\textnormal{
for }j=1,2,\ldots,M.$ (2)

Let $\zeta_{u}(k)\in\\{0,1\\}$ denote the energy consumer $u$ at time step $k$
was active or not. When the consumer $u$ is active then $\zeta_{u}(k)=1$ is
updated; otherwise, $\zeta_{u}(k)=0$. Furthermore, let $z_{u}(k)\in[0,1]$
denote average of the number of times consumer $u$ was active until time step
$k$. We define it as follows:

$\displaystyle{z}_{u}(k)\triangleq\frac{1}{k+1}\sum_{\ell=0}^{k}\zeta_{u}(\ell),\textnormal{
for }u=1,2,\ldots,U.$ (3)

Note that $\xi_{i}(k)$, $\eta_{j}(k)$, and $\zeta_{u}(k)$ are independent
Bernoulli random variables.

Let $\mathbf{x}=(x_{1},x_{2},\ldots,x_{N})\in[0,1]^{N}$, and let
$\mathbf{y}=(y_{1},y_{2},\ldots,y_{M})\in[0,1]^{M}$, and let
$\mathbf{z}=(z_{1},z_{2},\ldots,z_{U})\in[0,1]^{U}$; the bold letters denote
the vector entries. Let each prosumer has a cost function associated with the
production of energy; analogously, each consumer has a cost function
associated with the consumption of energy. More specifically, let
$f_{i}:[0,1]^{N}\to\mathbb{R}_{+}$ be the cost function of prosumer $i$ which
is associated with a cost to the production of solar energy for prosumer $i$,
for $i=1,2,\ldots,N$. Furthermore, let $g_{j}:[0,1]^{M}\to\mathbb{R}_{+}$ be
the cost function of the prosumer $j$ which is associated with a cost to
production of wind energy, for $j=1,2,\ldots,M$. In addition, let
$h_{u}:[0,1]^{U}\to\mathbb{R}_{+}$ be the cost function of consumer associated
with a cost to consumption of energy, for $u=1,2,\ldots,U$. We assume that the
cost function $f_{i}$, $g_{j}$, and $h_{u}$ are twice continuously
differentiable, convex, and increasing. Furthermore, we assume that the
prosumers and consumers do not share their information with other prosumers or
consumers in the community. Additionally, let $f_{i}^{\prime}(x_{i})$ denote
the derivative of the cost function $f_{i}$ of prosumer $i$ with solar panels
with respect to $x_{i}$; $g_{j}^{\prime}(y_{j})$ denote the derivative of the
cost function $g_{j}$ of prosumer $j$ with wind turbines with respect to
$y_{j}$; and $h_{u}^{\prime}(z_{u})$ denote the derivative of the cost
function $h_{u}$ of consumer $u$ with respect to $z_{u}$, for
$i=1,2,\ldots,N$, $j=1,2,\ldots,M$, and $u=1,2,\ldots,U$.

Our aim is to minimize the overall cost to the energy community. We formulate
the optimization problem as follows:

$\displaystyle\begin{split}\min_{\mathbf{x},\mathbf{y},\mathbf{z}}\quad&\sum_{i=1}^{N}f_{i}(x_{i})+\sum_{j=1}^{M}g_{j}(y_{j})+\sum_{u=1}^{U}h_{u}(z_{u}),\\\
\mbox{subject to }\quad&\sum_{i=1}^{N}x_{i}=C_{\textnormal{s}},\\\
&\sum_{j=1}^{M}y_{j}=C_{\textnormal{w}},\\\
&\sum_{u=1}^{U}z_{u}=\sum_{i=1}^{N}x_{i}+\sum_{j=1}^{M}y_{j},\\\ &x_{i}\geq
0,\quad\mbox{for }i=1,\ldots,N,\\\ &y_{j}\geq 0,\quad\mbox{for
}j=1,\ldots,M,\\\ &z_{u}\geq 0,\quad\mbox{for }u=1,\ldots,U.\end{split}$ (4)

Let the optimal value of the average of the number of times a prosumer with
solar panels be denoted by
$\mathbf{x}^{*}=({x}_{1}^{*},\ldots,{x}_{N}^{*})\in(0,1]^{N}$ and optimal
value for prosumers with wind turbines be noted by
$\mathbf{y}^{*}=({y}_{1}^{*},\ldots,{y}_{M}^{*})\in(0,1]^{M}$, and optimal
value for energy consumers be denoted by
$\mathbf{z}^{*}=({z}_{1}^{*},\ldots,{z}_{U}^{*})\in(0,1]^{U}$. They represent
the solutions to the optimization problem (4). Moreover, let $\mathbb{N}$
denote the set of natural numbers, and let $k\in\mathbb{N}$ denote the time
steps. In this paper, we develop distributed stochastic algorithms that
determine the active number of energy prosumers with solar panels and the
number of active energy prosumers with wind turbines, and also the number of
active energy consumers at a time step to ensure that the average of the
number of times prosumers and consumers are active, as defined in (1), (2),
and (3), respectively, reach optimal values over long term. Precisely, we have
for the number of active energy prosumers with solar panels reach optimal
value as stated below:

$\displaystyle\lim_{k\to\infty}{x}_{i}(k)={x}_{i}^{*},\text{ for
}i=1,2,\ldots,N.$ (5)

We have for the number of active energy prosumers with wind turbines reach the
respective optimal value as in (6).

$\displaystyle\lim_{k\to\infty}{y}_{j}(k)={y}_{j}^{*},\text{ for
}j=1,2,\ldots,M.$ (6)

Analogously, we have for the number of active energy consumers reach its
respective optimal value as in (7).

$\displaystyle\lim_{k\to\infty}{z}_{u}(k)={z}_{u}^{*},\text{ for
}u=1,2,\ldots,U.$ (7)

Thus, on long-term averages the community members achieve the minimum cost
$\sum_{i=1}^{N}f_{i}(x_{i}^{*})+\sum_{j=1}^{M}g_{j}(y_{j}^{*})+\sum_{u=1}^{U}h_{u}(z_{u}^{*})$
to the community.

##  IV Algorithm

In this section, we present distributed and stochastic algorithms for
prosumers, consumers, and the community manager. The prosumers with solar
panels and the prosumers with wind turbines run their respective algorithms to
share produced energy. Analogously, each consumer runs its algorithm to
consume energy. At a time step, a prosumer is either active, producing energy,
or not active; analogously, an energy consumer is either active, consuming
energy, or not active. They decide to participate in energy production or
consumption based on their response probabilities. The community manager
updates and broadcasts feedback signals for prosumers with solar panels and
prosumers with wind turbines and also for the consumers in the energy
community. Let us now define the parameters used. For $k\in\mathbb{N}$, let
$\Theta^{\textnormal{s}}(k)$ denote the feedback signal for prosumers with
solar panels and let $\Theta^{\textnormal{w}}(k)$ denote the feedback signal
for the prosumers with wind turbines. Furthermore, let
$\Theta^{\textnormal{c}}(k)$ denote the feedback signal for the energy
consumers in the community. Let $\tau^{\textnormal{s}}\in(0,1)$ denote the
step size for feedback signals for prosumers with solar panels. Analogously,
let $\tau^{\textnormal{w}}\in(0,1)$ denote the step size for feedback signals
for prosumers with wind turbines, and let $\tau^{\textnormal{c}}\in(0,1)$
denote the step size for feedback signals for consumers. When a prosumer with
solar panels joins the smart energy community at time step $k$,
$k\in\mathbb{N}$, it receives the parameter $\Theta^{\textnormal{s}}(k)$ from
the community manager. Analogously, prosumers with wind turbines receive
$\Theta^{\textnormal{w}}(k)$ and the energy consumers receive
$\Theta^{\textnormal{c}}(k)$ from the community manager. The feedback signal
for solar energy prosumers $\Theta^{\textnormal{s}}(k)$ depends on its value
at the previous time step, the step size $\tau^{\textnormal{s}}$, the desired
number of solar panels in the energy community $C_{\textnormal{s}}$, and the
total active solar energy prosumers at the previous time step, for
$k\in\mathbb{N}$. The community manager updates $\Theta^{\textnormal{s}}(k)$
according to (8) at each time step $k$ and broadcasts it to all solar energy
prosumers in the community:

$\displaystyle\Theta^{\textnormal{s}}(k+1)$
$\displaystyle=\Theta^{\textnormal{s}}(k)-\frac{\tau^{\textnormal{s}}}{k+1}\left(\sum_{i=1}^{N}\xi_{i}(k)-C_{\textnormal{s}}\right).$
(8)

Similarly, the community manager updates $\Theta^{\textnormal{w}}(k)$
according to (9) at each time step $k$, $k\in\mathbb{N}$, and broadcasts it to
wind energy prosumers in the community.

$\displaystyle\Theta^{\textnormal{w}}(k+1)$
$\displaystyle=\Theta^{\textnormal{w}}(k)-\frac{\tau^{\textnormal{w}}}{k+1}\left(\sum_{j=1}^{M}\eta_{j}(k)-C_{\textnormal{w}}\right).$
(9)

Furthermore, the community manager updates $\Theta^{\textnormal{c}}(k)$
according to (IV) at each time step $k$ and broadcasts it to all energy
consumers in the community.

$\displaystyle\Theta^{\textnormal{c}}(k+1)$
$\displaystyle=\Theta^{\textnormal{c}}(k)-\frac{\tau^{\textnormal{c}}}{k+1}\left(\sum_{u=1}^{U}\zeta_{u}(k)-\sum_{i=1}^{N}\xi_{i}(k)-\sum_{j=1}^{M}\eta_{j}(k)\right).$
(10)

Notice that for a fixed $\tau^{\textnormal{s}}\in(0,1)$, the step size
$\frac{\tau^{\textnormal{s}}}{k+1}$ is decreasing; similarly,
$\frac{\tau^{\textnormal{w}}}{k+1}$ and $\frac{\tau^{\textnormal{c}}}{k+1}$
are also decreasing.

After receiving the feedback signal, a solar energy prosumer responds in a
probabilistic manner. Let $\varphi^{\textnormal{s}}_{i}(x_{i}(k))$ denote the
response probability for the prosumer with solar panels. The solar energy
prosumer calculates the response probability
$\varphi^{\textnormal{s}}_{i}(x_{i}(k))$ using its average number of times it
was active $x_{i}(k)$ up to time step $k$ and the derivative of its cost
function, for $k\in\mathbb{N}$, as presented in (11). The solar energy
prosumer $i$ calculates the Bernoulli outcome at time step $k\in\mathbb{N}$.
Note that outcome $1$ occurs with probability
$\varphi^{\textnormal{s}}_{i}(x_{i}(k))$ and outcome $0$ with probability
$1-\varphi^{\textnormal{s}}_{i}(x_{i}(k))$. Based on these outcomes, the solar
energy prosumer decides whether to be active or not. If the value is $1$, then
the solar prosumer is active; otherwise, it is not active. This process
repeats over time. The wind energy prosumers and the energy consumers also
follow similar steps. Following this, the average number of times a prosumer
was active converge to the respective optimal value. The proposed algorithm
for the community manager is presented in Algorithm 1, the algorithm for
prosumers with solar panels is presented in Algorithm 2, and the algorithm for
prosumers with wind turbines is presented in Algorithm 3. Additionally, the
algorithm for consumers is presented in Algorithm 4.

1Input: $C_{\textnormal{s}}$, $C_{\textnormal{w}}$,
$\tau^{\textnormal{s}},\tau^{\textnormal{w}}$, $\xi_{1}(k),\ldots,\xi_{N}(k)$,
$\eta_{1}(k),\ldots,\eta_{M}(k)$, and $\zeta_{1}(k),\ldots,\zeta_{U}(k)$, for
$k\in\mathbb{N}$.

2Output: $\Theta^{\textnormal{s}}(k+1),\Theta^{\textnormal{w}}(k+1)$,
$\Theta^{\textnormal{c}}(k+1)$, for $k\in\mathbb{N}$.

3Initialization: $\Theta^{\textnormal{s}}(0)\leftarrow 0.350$,
$\Theta^{\textnormal{w}}(0)\leftarrow
0.350$,$\Theta^{\textnormal{c}}(0)\leftarrow 0.350$ ;

4

5foreach  _$k\in\mathbb{N}$_ do

6

7 calculate $\Theta^{\textnormal{s}}(k+1)$ as in (8) and broadcast it in the
community;

8 calculate $\Theta^{\textnormal{w}}(k+1)$ as in (9) and broadcast it in the
community;

9 calculate $\Theta^{\textnormal{c}}(k+1)$ as in (IV) and broadcast it in the
community;

10 end foreach

Algorithm 1 Algorithm of the community manager.

1Input: $\Theta^{\textnormal{s}}(k)$, for $k\in\mathbb{N}$.

2Output: $\xi_{i}(k+1)$, for $k\in\mathbb{N}$.

3Initialization: $\xi_{i}(0)\leftarrow 1$ and
${x}_{i}(0)\leftarrow\xi_{i}(0)$.

4foreach  _$k\in\mathbb{N}$_ do

5

6 generate Bernoulli independent random variable $b_{i}(k)$ with
$\varphi^{\textnormal{s}}_{i}(x_{i}(k))$ probability of success,
$\varphi^{\textnormal{s}}_{i}(x_{i}(k))$ is calculated as in (11);

7 if  _ $b_{i}(k)=1$_ then

8 $\xi_{i}(k+1)\leftarrow 1$;

9 else

10 $\xi_{i}(k+1)\leftarrow 0$;

11 end if

12

13 end foreach

Algorithm 2 Algorithm of prosumer $i$ with solar panels.

1Input: $\Theta^{\textnormal{w}}(k)$, for $k\in\mathbb{N}$.

2Output: $\eta_{j}(k+1)$, for $k\in\mathbb{N}$.

3Initialization: $\eta_{j}(0)\leftarrow 1$ and
${y}_{j}(0)\leftarrow\eta_{j}(0)$.

4foreach  _$k\in\mathbb{N}$_ do

5

6 generate Bernoulli independent random variable $b^{\prime}_{j}(k)$ with
$\varphi^{\textnormal{w}}_{j}(y_{j}(k))$ probability of success,
$\varphi^{\textnormal{w}}_{j}(y_{j}(k))$ is calculated as in (12);

7 if  _ $b^{\prime}_{j}(k)=1$_ then

8 $\eta_{j}(k+1)\leftarrow 1$;

9 else

10 $\eta_{j}(k+1)\leftarrow 0$;

11 end if

12

13 end foreach

Algorithm 3 Algorithm of prosumer $j$ with wind turbines.

1Input: $\Theta^{\textnormal{c}}(k)$, for $k\in\mathbb{N}$.

2Output: $\zeta_{u}(k+1)$, for $k\in\mathbb{N}$.

3Initialization: $\zeta_{u}(0)\leftarrow 1$ and
${z}_{u}(0)\leftarrow\zeta_{u}(0)$.

4foreach  _$k\in\mathbb{N}$_ do

5

6 generate Bernoulli independent random variable $b^{\prime\prime}_{u}(k)$
with $\varphi^{\textnormal{c}}_{u}(z_{u}(k))$ probability of success,
$\varphi^{\textnormal{c}}_{u}(z_{u}(k))$ is calculated as in (13);

7 if  _ $b^{\prime\prime}_{u}(k)=1$_ then

8 $\zeta_{u}(k+1)\leftarrow 1$;

9 else

10 $\zeta_{u}(k+1)\leftarrow 0$;

11 end if

12

13 end foreach

Algorithm 4 Algorithm of energy consumer $u$.

We chose small _gain parameters_ $\tau^{\textnormal{s}}$,
$\tau^{\textnormal{w}}$, $\tau^{\textnormal{c}}$ in $(0,1)$. After receiving
the feedback signal $\Theta^{\textnormal{s}}(k)$ from the community manager at
time step $k$, prosumer $i$ with solar panels responds with probability
$\varphi^{\textnormal{s}}_{i}(x_{i}(k))$ at the next time step, we define it
as follows.

$\displaystyle\varphi^{\textnormal{s}}_{i}(x_{i}(k))\triangleq\Theta^{\textnormal{s}}(k)\frac{x_{i}(k)}{f_{i}^{\prime}(x_{i}(k))}.$
(11)

Furthermore, after receiving the feedback signal $\Theta^{\textnormal{w}}(k)$
from the community manager at time step $k$, prosumer $j$ with wind turbines
responds with probability $\varphi^{\textnormal{w}}_{j}(y_{j}(k))$ at the next
time step as follows.

$\displaystyle\varphi^{\textnormal{w}}_{j}(y_{j}(k))\triangleq\Theta^{\textnormal{w}}(k)\frac{y_{j}(k)}{g_{j}^{\prime}(y_{j}(k))}.$
(12)

Analogously, after receiving the feedback signal $\Theta^{\textnormal{c}}(k)$
from the community manager at time step $k$, consumer $u$ responds with
probability $\varphi^{\textnormal{c}}_{u}(z_{u}(k))$ at next time step as
follows.

$\displaystyle\varphi^{\textnormal{c}}_{u}(z_{u}(k))\triangleq\Theta^{\textnormal{c}}(k)\frac{z_{u}(k)}{h_{u}^{\prime}(z_{u}(k))}.$
(13)

Note that $\Theta^{\textnormal{s}}(k)$, $\Theta^{\textnormal{w}}(k)$, and
$\Theta^{\textnormal{c}}(k)$ are the feedback signals, they bound
$\varphi^{\textnormal{s}}_{i}(x_{i}(k)),\varphi^{\textnormal{w}}_{j}(y_{j}(k)),\varphi^{\textnormal{c}}_{u}(z_{u}(k))\in(0,1)$,
respectively, for $i=1,2,\ldots,N$; $j=1,2,\ldots,M$; $u=1,2,\ldots,U$, and
$k\in\mathbb{N}$. Following the above algorithms, the average number of times
a prosumer with solar panels is active reaches optimal value over time.
Similarly, the average number of times a prosumer with wind turbines is active
reaches optimal value over time and the average number of times a consumer is
active reaches optimal value over time, and asymptotically, smart energy
community achieves optimum social cost.

##  V Numerical Results

Let us consider that a smart energy community has $N$ prosumers with solar
panels, $M$ prosumers with wind turbines, and $U$ energy consumers. The
production and consumption of energy may incur costs to the prosumers and the
consumers. The community members aim to minimize the overall cost to the
community. Let $C_{\textnormal{s}}$ denote the total number of prosumers with
solar panels, and $C_{\textnormal{w}}$ denote the total number of prosumers
with wind panels. In addition, we assume that each prosumer with solar panels
has a private cost function $f_{i}(\cdot)$ which depends on the average of the
number of times prosumer $i$ was active, we denote the average up to $k$ time
steps by $x_{i}(k)$, for $i=1,2,\ldots,N$. Furthermore, each prosumer with
wind turbines has a private cost function $g_{j}(\cdot)$ which depends on the
average of the number of times prosumer $j$ with wind turbines was active; we
denote the average until $k$ time steps by $y_{j}(k)$, for $j=1,2,\ldots,M$.
Also, each consumer has a private cost function $h_{u}(\cdot)$ which depends
on the average of number of times consumer $u$ was active, we denote the
average up to $k$ time steps by $z_{u}(k)$, for $u=1,2,\ldots,U$. Moreover, we
consider the community manager aggregates the total number of active prosumers
and consumers and broadcasts feedback signals in the community at each time
step. Using the feedback signals, the derivative of the cost function, and the
average number of times a member was active, it decides whether to be active
in producing or consuming energy or not.

###  V-A Setup

We chose $N=100$ prosumers with solar panels, $M=80$ prosumers with wind
turbines, and $U=160$ consumers in the energy community. Additionally, we
chose capacities $C_{\textnormal{s}}=50$ and $C_{\textnormal{w}}=60$. To
capture the costs, we chose the following cost functions for solar energy
prosumers:

$f_{i}(x_{i})=x_{i}+a_{1i}x_{i}^{2}+b_{1i}x_{i}^{4},\quad\textnormal{ for
}i=1,2,\ldots,N.$ (14)

Furthermore, we chose the following cost functions for wind energy prosumers:

$g_{j}(y_{j})=y_{j}+a_{2j}y_{j}^{2}+b_{2j},\quad\textnormal{ for
}j=1,2,\ldots,M.$ (15)

And, for energy consumers, we chose the following cost functions:

$h_{u}(z_{u})=a_{3u}z_{u}^{2}+b_{3u}z_{u}^{4}+c_{3u},\quad\textnormal{ for
}u=1,2,\ldots,U.$ (16)

To create randomized cost functions, we use random variables $a_{1i},b_{1i}$,
$a_{2j}$, $b_{2j}$, $a_{3u}$, $b_{3u}$, and $c_{3u}$ that are uniformly
distributed.

###  V-B Results

We present the experimental results in this section. Figure 1 shows the
evolution of the average number of times prosumers with solar panels,
prosumers with wind turbines, and the energy consumers are active.
Furthermore, we observe that the time-averaged values converge over time.
Suppose that $K$ denotes the last time step of the experiment. To check how
close the average values for prosumers and consumers reach to the respective
optimal values, we plot the absolute difference of the average number of times
prosumers and consumers are active up to time step $K$ and the respective
optimal values, in other words, we plot $|x_{i}(K)-x_{i}^{*}|$,
$|y_{j}(K)-y_{j}^{*}|$, and $|z_{u}(K)-z_{u}^{*}|$, for $i=1,2,\ldots,N$,
$j=1,2,\ldots,M$, and $u=1,2,\ldots,U$, as histograms as presented in Figure
2. To compare the results, we solved the optimization problem (4) in a
centralized way using CVX. Note that $x_{i}^{*}$, $y_{j}^{*}$, and $z_{u}^{*}$
are the optimal values obtained by the CVX solver, for $i=1,2,\ldots,N$,
$j=1,2,\ldots,M$, and $u=1,2,\ldots,U$. Moreover, from Figure 2, we observe
that for most of the prosumers and consumers, the absolute difference between
the averages and the optimal values are close to zero. For
$\ell=1,2,\ldots,K$, Figure 3 shows the evolution of the ratio of total cost
$\sum_{i=1}^{N}f_{i}(x_{i}(\ell))+\sum_{j=1}^{M}g_{j}(y_{j}(\ell))+\sum_{u=1}^{U}h_{u}(z_{u}(\ell))$
and the total optimal cost by the solver CVX
$\sum_{i=1}^{N}f_{i}(x_{i}^{*})+\sum_{j=1}^{M}g_{j}(y_{j}^{*})+\sum_{u=1}^{U}h_{u}(z_{u}^{*})$.
We observe that the ratio of total costs reaches close to $1$, that is, the
total cost obtained by the algorithms and the cost obtained by the solver are
very close.

Finally, Figure 4 illustrates the evolution of the total number of active
prosumers with solar panels, the evolution of the total number of active
prosumers with wind turbines, and the evolution of the total number of active
energy consumers. We observe that the total number of active prosumers with
solar panels is around its capacity, $C_{\textnormal{s}}$, and the total
number of active prosumers with wind turbines is around its capacity
$C_{\textnormal{w}}$. Furthermore, we observe that the total number of active
consumers is around the sum of the capacities of prosumers with solar panels
and the prosumers with wind turbines, that is, it fluctuates around
$C_{\textnormal{s}}+C_{\textnormal{w}}$, because of the randomized nature of
the algorithms.

(a)  (b)  (c)

Figure 1: (a) The evolution of the average number of times prosumers with
solar panels are active. (b) The evolution of the average number of times
prosumers with wind turbines are active. (c) The evolution of the average
number of times consumers are active.

(a)  (b)  (c)

Figure 2:  (a) The histogram of absolute difference of average number of
active solar prosumers and the optimal value. (b) The histogram of absolute
difference of average number of active wind prosumers and the optimal value.
(c) The histogram of absolute difference of average number of active consumers
and the optimal value. (a)  Figure 3: The evolution of the ratio of total cost
obtained by the algorithms and the total optimal cost obtained by the solver.

(a)  (b)  (c)

Figure 4: (a) The evolution of the total number of active producers with solar
panels, (b) the evolution of the total number of active producers with wind
turbines, and (c) the evolution of the total number of active energy
consumers.

##  VI Conclusion

We proposed new distributed stochastic algorithms to regulate the number of
prosumers and the number of consumers in a smart energy community. The smart
energy community consists of energy prosumers and consumers. We consider that
the prosumers have heterogeneous energy sources—have either solar panels or
wind turbines installed. In the model, the prosumers and consumers do not need
to share their information with others. Each community member keeps its
information private such as its cost function, the derivative of the cost
function, and when they are active. However, a community manager is needed to
keep track of the aggregate prosumption and consumption. Through experimental
results, we show that the average number of times a prosumer or a consumer is
active reaches optimal value over time.

As future work, it will be interesting to extend this work to facilitate the
consumers to track the energy source using blockchain technology. Furthermore,
we can also facilitate the prosumers to track if the produced energy reaches
the needy people when the prosumers are involved in philanthropic work.
Additionally, it will be exciting to develop a blockchain-based system that
does not require a community manager; nevertheless, the prosumers and
consumers receive optimal values. Last, it will also be interesting to provide
a theoretical guarantee on the convergence of average values and study the
convergence rate of the algorithms.

## References

  * [1] E. Mengelkamp, J. Gärttner, K. Rock, S. Kessler, L. Orsini, and C. Weinhardt, “Designing microgrid energy markets: A case study: The brooklyn microgrid,” _Applied Energy_ , vol. 210, pp. 870–880, 2018.
  * [2] T. Morstyn, N. Farrell, S. J. Darby, and M. D. McCulloch, “Using peer-to-peer energy-trading platforms to incentivize prosumers to form federated power plants,” _Nature Energy_ , vol. 3, no. 2, pp. 94–101, 2018.
  * [3] L. Ableitner, V. Tiefenbeck, A. Meeuw, A. Wörner, E. Fleisch, and F. Wortmann, “User behavior in a real-world peer-to-peer electricity market,” _Applied Energy_ , vol. 270, 2020.
  * [4] I. Savelli and T. Morstyn, “Better together: Harnessing social relationships in smart energy communities,” _Energy Research & Social Science_, vol. 78, 2021.
  * [5] V. S. Borkar, _Stochastic Approximation_. Cambridge University Press, 2008.
  * [6] W. M. Griggs, J. Y. Yu, F. R. Wirth, F. Hausler, and R. Shorten, “On the design of campus parking systems with QoS guarantees,” _IEEE Trans. Intelligent Transportation Systems_ , vol. 17, no. 5, pp. 1428–1437, 2016.
  * [7] S. E. Alam, R. Shorten, F. Wirth, and J. Y. Yu, _Distributed Algorithms for Internet-of-Things-Enabled Prosumer Markets: A Control Theoretic Perspective_. Cham: Springer International Publishing, 2020, pp. 125–149.
  * [8] L. Chen, N. Liu, C. Li, S. Zhang, and X. Yan, “Peer-to-peer energy sharing with dynamic network structures,” _Applied Energy_ , vol. 291, 2021.
  * [9] A. Mahmud, S. N. Islam, and I. Lilley, “A smart energy hub for smart cities: Enabling peer-to-peer energy sharing and trading,” _IEEE Consumer Electronics Magazine_ , 2021.
  * [10] L. Werner, A. Wierman, and S. H. Low, “Pricing flexibility of shiftable demand in electricity markets,” in _ACM International Conference on Future Energy Systems_ , ser. e-Energy ’21, 2021, pp. 1–14.
  * [11] E. Oh and S.-Y. Son, “Peer-to-peer energy transaction mechanisms considering fairness in smart energy communities,” _IEEE Access_ , vol. 8, pp. 216 055–216 068, 2020.
  * [12] S. Zhou, Z. Hu, W. Gu, M. Jiang, and X. P. Zhang, “Artificial intelligence based smart energy community management: A reinforcement learning approach,” _CSEE Journal of Power and Energy Systems_ , vol. 5, no. 1, pp. 1–10, 2019\.
  * [13] S. Bhattacharjee and C. Nandi, “Design of a voting based smart energy management system of the renewable energy based hybrid energy system for a small community,” _Energy_ , vol. 214, 2021.
  * [14] S. Wang, A. F. Taha, J. Wang, K. Kvaternik, and A. Hahn, “Energy crowdsourcing and peer-to-peer energy trading in blockchain-enabled smart grids,” _IEEE Transactions on Systems, Man, and Cybernetics: Systems_ , vol. 49, no. 8, pp. 1612–1623, 2019.
  * [15] A. Bajpai, J. Schindler, and R. Baun, “Systematic design of blockchain based testbed for peer-to-peer energy trading,” in _IEEE International Smart Cities Conference (ISC2)_ , 2021, pp. 1–7.
  * [16] T. AlSkaif, J. L. Crespo-Vazquez, M. Sekuloski, G. van Leeuwen, and J. P. S. Catalão, “Blockchain-based fully peer-to-peer energy trading strategies for residential energy systems,” _IEEE Transactions on Industrial Informatics_ , vol. 18, no. 1, pp. 231–241, 2022.
  * [17] M. Gough, S. F. Santos, A. Almeida, M. Lotfi, M. S. Javadi, D. Z. Fitiwi, G. J. Osorio, R. Castro, and J. P. S. Catalao, “Blockchain-based transactive energy framework for connected virtual power plants,” _IEEE Transactions on Industry Applications_ , vol. 58, no. 1, pp. 986–995, 2022.
  * [18] M. F. Zia, M. Benbouzid, E. Elbouchikhi, S. M. Muyeen, K. Techato, and J. M. Guerrero, “Microgrid transactive energy: Review, architectures, distributed ledger technologies, and market analysis,” _IEEE Access_ , vol. 8, pp. 19 410–19 432, 2020.
  * [19] F. Ceglia, P. Esposito, E. Marrasso, and M. Sasso, “From smart energy community to smart energy municipalities: Literature review, agendas and pathways,” _Journal of Cleaner Production_ , vol. 254, p. 120118, 2020.
  * [20] W. Tushar, C. Yuen, T. K. Saha, T. Morstyn, A. C. Chapman, M. J. E. Alam, S. Hanif, and H. V. Poor, “Peer-to-peer energy systems for connected communities: A review of recent advances and emerging challenges,” _Applied Energy_ , vol. 282, 2021.
  * [21] M. E. Honarmand, V. Hosseinnezhad, B. Hayes, M. Shafie-Khah, and P. Siano, “An overview of demand response: From its origins to the smart energy community,” _IEEE Access_ , vol. 9, pp. 96 851–96 876, 2021.
  * [22] D. de Sao Jose, P. Faria, and Z. Vale, “Smart energy community: A systematic review with metanalysis,” _Energy Strategy Reviews_ , vol. 36, p. 100678, 2021.
  * [23] M. Thornbush and O. Golubchikov, “Smart energy cities: The evolution of the city-energy-sustainability nexus,” _Environmental Development_ , vol. 39, 2021.


#  Trial2Vec: Zero-Shot Clinical Trial Document Similarity Search using Self-
Supervision

Zifeng Wang and Jimeng Sun
University of Illinois Urbana-Champaign
<EMAIL_ADDRESS>

###### Abstract

Clinical trials are essential for drug development but are extremely expensive
and time-consuming to conduct. It is beneficial to study similar historical
trials when designing a clinical trial. However, lengthy trial documents and
lack of labeled data make trial similarity search difficult. We propose a
zero-shot clinical trial retrieval method, called Trial2Vec, which learns
through self-supervision without the need for annotating similar clinical
trials. Specifically, the meta-structure of trial documents (e.g., title,
eligibility criteria, target disease) along with clinical knowledge (e.g.,
UMLS knowledge base 111https://www.nlm.nih.gov/research/umls/index.html) are
leveraged to automatically generate contrastive samples. Besides, Trial2Vec
encodes trial documents considering meta-structure thus producing compact
embeddings aggregating multi-aspect information from the whole document. We
show that our method yields medically interpretable embeddings by
visualization and it gets 15% average improvement over the best baselines on
precision/recall for trial retrieval, which is evaluated on our labeled 1600
trial pairs. In addition, we prove the pretrained embeddings benefit the
downstream trial outcome prediction task over 240k trials. 222Software is
available at https://github.com/RyanWangZf/Trial2Vec.

##  1 Introduction

Clinical trials are essential for developing new medical interventions
Friedman et al. (2015). Many considerations come into the design of a clinical
trial, including study population, target disease, outcome, drug candidates,
trial sites, and eligibility criteria, as in Table 1. It is often beneficial
to learn from related clinical trials from the past to design an optimal trial
protocol Wang et al. (2022b). However, accurate similarity search based on the
lengthy trial documents is still in dire need.

Table 1: An example of the meta-structure of clinical trial document drawn
from ClinicalTrials.gov.

Title |  Effects of Electroacupuncture With Different Frequencies for Major Depressive Disorder
---|---
Description |  Two groups of subjects will be included 55 subjects in electroacupuncture with 2Hz group…
Eligibility Criteria |  1\. Inclusion Criteria:
1.1. Patients suffering from MDD in accordance with the diagnostic criteria;
1.2. Hamilton Depression Scale score is between 21 and 35 (mild to moderate
MDD);…
2\. Exclusion Criteria:
2.1 Patients with bipolar disorder;
2.2 Patients with schizophrenia or other mental disorders; …
Outcome Measures |  1\. Change in anxiety and depression severity measure by Self-rating depression scale
2\. Change in the severity of depression measure by Hamilton depression scale
..
Disease |  Major Depressive Disorder
Intervention |  electroacupuncture
… |  …

Self-supervision based pretraining has delivered promising performances for
many NLP and CV tasks with fine-tuning Devlin et al. (2019); Liu et al.
(2019); He et al. (2021); Bao et al. (2021); Wang et al. (2022c).
Nevertheless, we find there was few work on zero-shot document retrieval as
most address document retrieval in a supervised fashion Humeau et al. (2019);
Khattab and Zaharia (2020); Guu et al. (2020); Karpukhin et al. (2020); Lin et
al. (2020); Luan et al. (2021); Wang et al. (2021); Hofstätter et al. (2020);
Li et al. (2020); Zhan et al. (2021); Hofstätter et al. (2021b, a); Jiang et
al. (2022) or improve document pre-training for further supervision Beltagy et
al. (2020); Zaheer et al. (2020); Ainslie et al. (2020); Zhang et al. (2021).

Recently, a burgeoning body of research Gao et al. (2021); Wu et al. (2021);
Wang et al. (2022a) proposes to execute self-supervised learning to train
semantic-meaningful sentence embeddings free of labels. However, there are
still challenges to apply them for document similarity search:

  * •

Lengthy documents. These zero-shot BERT retrieval methods all work on short
sentences (usually below 10 words) similarity search while trial documents are
often above 1k words. Simply encoding lengthy trials by truncating and
averaging embeddings of all remaining tokens inevitable leads to poor
retrieval quality.

  * •

Inefficient contrastive supervision. These unsupervised methods take simple
instance discriminative contrastive learning (CL) within batch, e.g., SimCSE
Gao et al. (2021) takes one sentence into the encoder twice to get the
positive pairs and all other sentences as the negative. This paradigm has low
supervision efficiency to require a large batch size, large data, and long
training time, which is infeasible for learning from long trial documents.

In this work, we propose Clinical Trial TO Vectors, Trial2Vec, a zero-shot
trial document similarity search using self-supervision. We design a trial
encoding framework considering the meta-structure to rid the risk that
semantic meaning vanishes due to the uniform average of token embeddings.
Meanwhile, the meta-structure is utilized to generate contrastive samples for
efficient supervision. Medical knowledge is introduced to further enhance the
negative sampling for CL. Our main contributions are:

  * •

We are the first to study the trial-to-trial retrieval task by proposing a
label-free SSL model which is able to encode long trials into semantic
meaningful embeddings without labels.

  * •

We propose a data-efficient CL method on medical knowledge and trial meta-
structure, which is promising to be extended to further zero-shot structured
document retrieval.

  * •

We demonstrate the superiority of Trial2Vec on a trial relevance dataset of
1600 trials annoated by domain experts. Also, we show Trial2Vec can assist
better downstream trial outcome prediction on a dataset of 240k trials.

##  2 Related works

###  2.1 Text & document retrieval

General texts. Early information retrieval methods depend on manual
engineering Robertson and Zaragoza (2009); Yang et al. (2017). By contrast,
dense retrieval methods based on distributional word representations, e.g.,
Word2Vec Mikolov et al. (2013), Glove Pennington et al. (2014), Doc2Vec Le and
Mikolov (2014), etc., become popular crediting to their superior performance.
The advent of deep models, especially the contexualized encoders like BERT
Devlin et al. (2019), encourages an explosion of neural retrieval methods Van
Gysel et al. (2016); Zamani et al. (2018); Guo et al. (2016); Dehghani et al.
(2017); Onal et al. (2018); Reimers and Gurevych (2019); Chang et al. (2019);
Nogueira and Cho (2019); Chen et al. (2021); Lin et al. (2020); Xiong et al.
(2020); Karpukhin et al. (2020); Yates et al. (2021). However, most of them
are based on supervised training on sentence pairs from general texts, e.g.,
SNLI Bowman et al. (2015). When label is expensive to acquire, as in the
clinical trial case, we need zero-shot learning models. Although, there arose
some works to perform post-processing on pretrained BERT embeddings to improve
their retrieval quality Li et al. (2020); Su et al. (2021), their performances
are far from optimal without specific training.

Clinical trials. Traditional clinical trial query search systems Tasneem et
al. (2012); Tsatsaronis et al. (2012); Jiang and Weng (2014); Park et al.
(2020) are established on protocol databases. Contrast to dense retrieval,
these methods rely on entity matching with rules thus not flexible enough.
Recent works Roy et al. (2019); Rybinski et al. (2020, 2021) propose
supervised neural ranking for clinical trial query search. However, all of
them work on matching trial titles or relevant segments with an input user
query. While Trial2Vec can also assist query search, it is the first to encode
complete trial documents for the trial-level similarity search.

Figure 1: Overview of the proposed Trial2Vec framework. Top left: the training
strategy that accounts for unlabeled input trial documents with meta-structure
along with an external medical knowledge database, e.g., UMLS. Top right: The
contrastive supervision splits into meta-structure and knowledge guided,
respectively. Bottom left: our method hierarchically encodes trials into local
and global embeddings on the trial meta-structure. Bottom right: The encoded
trial-level embeddings can be used to trial search, query trial search and
downstream tasks.

###  2.2 Text contrastive learning

Contrastive learning is a heated discussed topic recently in NLP and CV Chen
et al. (2020a, b); Chen and He (2021); Carlsson et al. (2020); Zhang et al.
(2020); Wu et al. (2020); Yan et al. (2021); Gao et al. (2021); Wang et al.
(2020b); Wang and Sun (2022). CL is one main topic under the SSL domain. It
sheds light on reaching comparable performance as supervised learning free of
manual annotations. While CL has been applied to enhance downstream NLP
applications like text classification Li et al. (2021); Zhang et al. (2022), a
few Wang et al. (2020a); Zhang et al. (2020); Yan et al. (2021); Yang et al.
(2021) are able to do zero-shot retrieval. Nonetheless, all focus on enhancing
sentence embeddings by manipulating text only therefore are suboptimal when
facing lengthy documents. By contrast, Trial2Vec uses the document meta-
structure with domain knowledge to obtain and facilitate document embeddings.

##  3 Method

In this section, we present the details of Trial2Vec. The main idea is to
jointly learn the global and local representations from trial documents
considering their meta-structure. Specifically, observed in Table 1, trial
document consists of multiple sections while the key attributes (e.g., title,
disease, intervention, etc.) occupy a small portion of the whole document.
This motivates us to design a hierarchical encoding and the corresponding
contrastive learning framework. The overview is illustrated in Fig. 1. Our
method generates local attribute embeddings using the TrialBERT backbone
separately, then aggregating local embeddings with a learnable attention
module to obtain the global trial embeddings that emphasize significant
attributes. We present the pretraining of backbone encoder in §3.1; then we
describe the hierarchical encoding process based on the backbone encoder in
§3.2; the hierarchical constrastive learning methods considering meta-
structure and medical knowledge are elucidated in §3.3; at last, we elicit the
applications of the proposed framework in §3.4.

###  3.1 Backbone encoder: TrialBERT

We leverage the BERT architecture as the backbone encoder in the framework. In
detail, we use the WordPiece tokenizer together with the BioBERT Lee et al.
(2020) pretrained weights as the start point. We continue the pretraining with
Masked Language Modeling (MLM) loss on three trial-related data sources:
ClinicalTrial.gov 333https://clinicaltrials.gov/, Medical Encyclopedia
444https://medlineplus.gov/encyclopedia.html, and Wikipedia Articles
555https://www.wikipedia.org/, see Table 6, to get TrialBERT.
ClinicalTrials.gov is a database that contains around 400k clinical trials
conducted in 220 countries. Medical Encyclopedia has 4K high-quality articles
introducing terminologies in medicine. We also retrieve relevant Wikipedia
articles corresponding to the 4k terminologies of Medical Encyclopedia.

###  3.2 Global and local embeddings by Trial2Vec

TrialBERT embeddings pretrained with MLM on clinical corpora still hold weak
semantic meaning. Meanwhile, previous sentence embedding BERTs all take an
average pooling over token embeddings, which causes the semantic meaning
vanishing when applied to lengthy clinical trials. Therefore, we propose
Trial2Vec architecture that exploits the global and local embeddings for trial
based on its meta-structure.

We split the attributes of a trial into two distinct sets: key attributes and
contexts. The first component includes the trial title, intervention,
condition, and main measurement, which are sufficient to retrieve a pool of
coarsely relevant trial candidates; the second includes descriptions,
eligibility criteria, references, etc., which differentiate trials targeting
similar diseases or interventions because they provide the multi-facet details
regarding disease phases, study designs, targeted populations, etc. According
to this design, local embeddings
$\\{\mathbf{v}_{att}\\}_{l=1}^{L}\in\mathbb{R}^{L\times D}$ are produced
separately on each key attribute. On the other hand, a context embedding is
obtained by encoding the context texts $\mathbf{v}_{ctx}\in\mathbb{R}^{D}$.
Note that the above encoding is all conducted by the same encoder.

We further refine the local embeddings by context embeddings and aggregate
them to yield the global trial embedding $\mathbf{v}_{g}\in\mathbb{R}^{D}$.
The refinement is performed by multi-head attention, as

$\mathbf{v}_{g}=\texttt{MultiHeadAttn}(\mathbf{v}_{ctx},\\{\mathbf{v}_{l}\\}_{l}^{L},\mathbf{W}),$
(1)

which relocates the attention over key attributes to enhance discrminative
power of the yielded global embedding.

###  3.3 Hierarchical contrastive learning

For data-efficient contrastive learning, we utilize the meta-structure &
medical knowledge for contrasting local and global embeddings hierarchically.

Global contrastive loss. The first objective is to maximize the semantic in
trial embeddings for similarity search. Instead of doing in-batch instance-
wise contrastive loss like SimCSE, we propose to sample informative negative
pairs by exploiting the trial meta-structure. As shown by Fig. 1, some trials
may be linked by a common attribute like disease or intervention. Denote a
trial consisting of several attributes by

$\mathbf{x}=\\{x^{\text{title}},x^{\text{intv}},x^{\text{dise}},x^{\text{out}},x^{\text{ctx}}\\},$
(2)

we can build an informative negative sample by replacing its title with a
trial which also targets for disease $x^{\text{dise}}$ by

$\mathbf{x}^{-}=\\{x^{\text{title}-},x^{\text{intv}},x^{\text{dise}},x^{\text{out}},x^{\text{ctx}}\\}.$
(3)

Meanwhile, we apply a random attribute dropout towards $\mathbf{x}$ to
formulate a positive sample as

$\mathbf{x}^{+}=\\{x^{\text{title}},x^{\text{dise}},x^{\text{out}},x^{\text{ctx}}\\}.$
(4)

InfoNCE loss is utilized in a batch of $B$ trials as

$\mathcal{L}_{g}=-\sum_{i=1}^{B}\log\frac{\exp(\psi(\mathbf{v}_{gi},\mathbf{v}_{gi}^{+}))}{\sum_{v_{gi}^{-}\in\mathcal{V}_{i}^{-}}\exp(\psi(\mathbf{v}_{gi},\mathbf{v}_{gi}^{-}))},$
(5)

where the negative sample set
$\mathcal{V}_{i}^{-}=\\{\mathbf{v}_{gi}^{-}\\}\cup\\{\mathbf{v}_{gj}\\}_{j\neq
i}$; $\psi(\cdot,\cdot)$ measures the cosine similarity between two vectors.
The global contrastive loss here encourages the model to capture the attribute
of interest by discriminating the subtle differences of input trial
attributes, which prevent the semantic meanings from vanishing due to the
average pooling over all trial texts.

Local contrastive loss. In addition to the global trial embeddings, we put
supervision on local embeddings to inject medical knowledge into the model.
Unlike general texts, two medical texts can be overlapped word-wise
dramatically but still describe two distinct things666For instance, replacing
Olaparib in ”A Phase I, Open-Label, 2 Part Multicentre Study to Assess the
Safety and Efficacy of Olaparib” with another intervention like Vitamin D
renders a total different study topic., which is challenging for similarity
computing. To strengthen TrialBERT discriminative power for medical texts, we
extract key medical entities in each text as 777Done by SciSpacy
https://scispacy.apps.allenai.org/.

$E(x^{att})=\\{e_{1},e_{2},e_{3},e_{4}\\},$ (6)

then a positive sample is built by mapping one entity $e_{1}$ to its canonical
name or a similar entity under the same parental conception $\hat{e}_{1}$
defined by UMLS as

$E(x^{att+})=\\{\hat{e}_{1},e_{2},e_{3},e_{4}\\}.$ (7)

Similarly, negative sample is built by deletion or replacing one entity with
another dissimilar one. InfoNCE loss is therefore used by

$\mathcal{L}_{l}=-\sum_{i=1}^{B}\log\frac{\psi(\mathbf{v}_{li},\mathbf{v}_{li}^{+})}{\sum_{\mathcal{V}_{li}^{-}}\exp(\psi(\mathbf{v}_{li},\mathbf{v}_{li}^{-}))}.$
(8)

We at last jointly optimize the global and contrastive losses as

$\mathcal{L}=\mathcal{L}_{g}+\mathcal{L}_{l}.$ (9)

###  3.4 Application of global & local embeddings

The hierarchical contrastive learning offers extraordinary flexibility of
Trial2Vec for various downstream tasks in zero-shot learning. At first, the
global trial embeddings $\mathbf{v}_{g}$ can be directly used for similarity
search by comparing trial pair-wise cosine similarities. The computed trial
embeddings can also help identify and discover research topics when we apply
visualization techniques. On the other hand, we can also execute query search
using partial attributes crediting to the contrastive learning between local
and global embeddings. When we need do trial-level predictive tasks, e.g.,
trial termination prediction, a classifier can be attached to the pretrained
global trial embeddings and learned; the backbone TrialBERT is also capable of
offering short medical sentence retrieval because of local contrastive
learning.

##  4 Experiments

In this section, we conduct five types of experiments to answer the following
research questions:

  * •

Exp 1 & 2. How does Trial2Vec perform in complete and partial retrieval
scenarios?

  * •

Exp 3. How do the proposed SSL tasks / embedding dimension contribute to the
performance?

  * •

Exp 4. Is the trial embedding space interpretable and aligned with medical
ontology?

  * •

Exp 5. How useful do well-trained Trial2Vec contribute to downstream tasks,
e.g., trial outcome prediction, after fine-tuned?

  * •

Exp 6. Qualitative analysis of the retrieval results and what are the
differences of Trial2Vec and baselines?

Table 2: Statistics of trial status in ClinicalTrials.gov database where we
conclude _Approved_ & _Completed_ as completion; _Suspended_ , _Terminated_ ,
and _Withdrawn_ as the termination for trial outcome prediction.

Approved | Completed | Suspended | Terminated | Withdrawn
---|---|---|---|---
174 | 210,237 | 1,658 | 22,208 | 10,439
Available | Enrolling | Unavailable | Not recruiting | Recruiting
237 | 3,662 | 45,128 | 18,171 | 60,362
Completion | Termination | Summary | Others |
210,411 | 34,305 | 244,716 | 127,560 |

Table 3: Precision/Recall and nDCG of the retrieval models on the labeled test
set. Values in parenthesis show 95% confidence interval. Best values are in
bold.

Method | Prec@1 | Prec@2 | Prec@5 | Rec@1 | Rec@2 | Rec@5 | nDCG@5
---|---|---|---|---|---|---|---
TF-IDF | 0.5132(0.063) | 0.4386(0.045) | 0.3828(0.057) | 0.1871(0.038) | 0.3172(0.026) | 0.6147(0.044) | 0.5480(0.034)
BM25 | 0.7015(0.044) | 0.5640(0.041) | 0.4246(0.032) | 0.3358(0.038) | 0.4841(0.050) | 0.7666(0.031) | 0.7312(0.033)
Word2Vec | 0.7492(0.071) | 0.6476(0.044) | 0.4712(0.033) | 0.3008(0.054) | 0.4929(0.042) | 0.7939(0.041) | 0.7712(0.032)
BERT | 0.7264(0.050) | 0.6219(0.060) | 0.4324(0.027) | 0.3257(0.051) | 0.4896(0.054) | 0.7611(0.041) | 0.7370(0.047)
BERT${}_{\text{Whiten}}$ | 0.7476(0.094) | 0.6630(0.045) | 0.4525(0.029) | 0.3672(0.045) | 0.5832(0.042) | 0.8355(0.021) | 0.8129(0.024)
BERT${}_{\text{SimCSE}}$ | 0.6788(0.039) | 0.5995(0.035) | 0.4714(0.021) | 0.2824(0.034) | 0.4566(0.035) | 0.8098(0.025) | 0.7308(0.038)
MonoT5${}_{\text{Med}}$ | 0.6799(0.068) | 0.5810(0.061) | 0.4439(0.051) | 0.2904(0.032) | 0.4657(0.049) | 0.7570(0.037) | 0.7171(0.043)
Trial2Vec | 0.8810(0.026) | 0.7912(0.049) | 0.5055(0.039) | 0.4216(0.046) | 0.6465(0.060) | 0.8919(0.030) | 0.8825(0.029)

###  4.1 Dataset & Setup

Trial Similarity Search. We created a labeled trial dataset to evaluate the
retrieval performance where paired trials are labeled as relevant or not. We
keep 311,485 interventional trials from the total 399,046 trials. We uniformly
sample 160 trials as the query trials. To overcome the sparsity of relevance,
we take advantage of TF-IDF Salton et al. (1983) to retrieve ranked top-10
trials as the candidate to be labeled, resulting in 1,600 labeled pairs of
clinical trials. Unlike general documents, the clinical trial document
contains many medical terms and formulations. We recruited clinical
informatics researchers, and each is assigned 400 pairs to label as relevant
or not using label $\\{1,0\\}$. To keep labeling processes in line, we specify
the minimum annotation guide for judging relevance: (1) same disease; or (2)
same intervention and similar diseases (e.g., cancer on distinct body parts).
We use precision@k (prec@k), recall@k (rec@k), and nDCG@5 to evaluate and
report performances.

$\displaystyle prec@k$ $\displaystyle=\frac{\text{\\# of relevant trials in
the top k results}}{k},$ (10) $\displaystyle rec@k$
$\displaystyle=\frac{\text{\\# of relevant trials in the top k
results}}{\text{\\# of relevant trials in all candidate trials}}.$ (11)

Trial termination prediction. We can take the pretrained Trial2Vec embeddings
for predicting the trial outcomes, i.e., if the trial will be terminated or
not. We add one additional fully-connected layer on the tail of Trial2Vec. The
targeted outcomes are in the status section of clinical trials, described by
Table 2. We formulate the outcome prediction as a binary classification
problem to predict the Completion or Termination of trials where we get
210,411 and 34,305 trials as positive and negative labeled, respectively. We
take 70% of all as the training set and 20% as the test set; the remaining 10%
is used as the validation set for tuning and early stopping. We utilize three
metrics for evaluation: accuracy (ACC), area under the Receiver Operating
Characteristic (ROC-AUC), and area under Precision-Recall curve (PR-AUC).

###  4.2 Baselines & Implementations

We take the following baselines for retrieval: TF-IDF Salton et al. (1983);
Salton and Buckley (1988), BM25 Trotman et al. (2014), Word2Vec Mikolov et al.
(2013), BERT-Whitening Huang et al. (2021); Su et al. (2021), BERT-SimCSE Gao
et al. (2021), and MonoT5 Roberts et al. (2021); Pradeep et al. (2022).
Details of these methods can be seen in Appendix A.

We keep all methods’ embedding dimensions at 768. We start from a BERT-base
model to continue pre-training on clinical domain corpora, yielding our
TrialBERT, which supports as the backbone for BERT-Whitening and BERT-SimCSE
for fair comparison. We take 5 epochs with batch size 100 and the learning
rate 5e-5. In the second SSL training phase, AdamW optimizer with a learning
rate of 2e-5, batch size of 50, and weight decay of 1e-4 is used. Experiments
were done with 6 RTX 2080 Ti GPUs.

Figure 2: Performance of Trial2Vec on the partial retrieval scenarios. We use
a different part of the trial as queries to retrieve similar trials, including
keyword kw, intervention intv, disease dz, context ctx. Error bars indicate
the 95% confidence interval of results.

Figure 3: Ablation study on the contribution of each Task to the final result.
att, mc, ctx are short for attribute, matching, context, respectively. all
indicate the full Trial2Vec that all tasks are used.

Figure 4: Analysis of the influence of embedding dimensions on retrieval
quality by Trial2Vec: embedding dim in 128, 256, 512, 768. Error bars show the
95% confidence interval.

Figure 5: 2D visualization of the trial-level embeddings obtained by Trial2Vec
(dimension reduced by t-SNE). It can be seen trials are automatically
classified into clusters by topic (diseases) in the embedding space. For
example, a series of tumor-related trials (e.g., Breast and Pancreatic
Cancers) are on the bottom of the embedding space.

###  4.3 Exp 1. Complete Trial Similarity Search

Since labels are unavailable in the training phase, we only chose
unsupervised/self-supervised baselines. Results are shown by Table 3.
Trial2Vec outperforms all baselines with a great margin. It has around 15%
improvement on each metrics than the best baselines on average. For baselines,
all except for TF-IDF have similar performance. When $k$ is small, the
precision gap between Trial2Vec and baselines is large; when $k$ is large, all
methods encounter precision reduction. That is because the pool of candidate
trials are 10 but the number of positive pairs for each are often less than 5,
which limits the maximum of the numerator of $prec@k$ in Eq. (10). Likewise,
Trial2Vec also shows stronger performance in $rec@k$ because it is discounted
by the maximum number of positive pairs.

Table 4: Trial outcome prediction performances of baselines and Trial2Vec,
after fine-tuned.

Method | ACC | ROC-AUC | PR-AUC
---|---|---|---
TF-IDF | 0.8571(0.002) | 0.7194(0.004) | 0.2960(0.008)
Word2Vec | 0.8574(0.002) | 0.7189(0.005) | 0.2906(0.007)
TrialBERT | 0.8559(0.002) | 0.7277(0.006) | 0.3109(0.006)
Trial2Vec | 0.8622(0.002) | 0.7332(0.004) | 0.3137(0.007)

Table 5: Case studies comparing the retrieval performance of the Trial2Vec
with baseline models. Due to the space limits, only title and NCT ID of trials
are given.

Query Trial |  TF-IDF |  TrialBERT |  Trial2Vec
---|---|---|---
[NCT02972294] HiFIT Study : Hip Fracture: Iron and Tranexamic Acid (HiFIT) |  [NCT01221389] Study Using Plasma for Patients Requiring Emergency Surgery (SUPPRES) |  [NCT04744181] Patient Blood Management In CARdiac sUrgical patientS (ICARUS) |  [NCT01535781] Study of the Effect of Tranexamic Acid Administered to Patients With Hip Fractures. Can Blood Loss be Reduced? [t]
[NCT01590342] Diclofenac for Submassive PE (AINEP-1) |  [NCT04006145] A Phase 2 Study of Elobixibat in Adults With NAFLD or NASH |  [NCT04156854] Intravascular Volume Expansion to Neuroendocrine-Renal Function Profiles in Chronic Heart Failure |  [NCT00247052] Non Steroidal Anti Inflammatory Treatment for Post Operative Pericardial Effusion

Interestingly, the state-of-the-art sentence BERTs, e.g., BERT-whitening and
BERT-simCSE, have limited improvement over original BERT and even Word2Vec.
Unlike general documents, clinical trials may be overlapped in much content
but still be irrelevant if the key entities are different. This special
characteristic causes the assumption of a document with similar passage is
relevant Craswell et al. (2020) used in general document retrieval but
invalidated in clinical trial retrieval. Without well-designed SSL, it is hard
for these methods to learn these subtle differences. Moreover, clinical trial
documents are often much longer than the general documents in those open
datasets. There are 622.4 words per trial on average, while the general STS
benchmark has below 15 words per sample, e.g., STS-12: 10.8, STS-13: 8.8,
STS-14: 9.1, etc Cer et al. (2017). We also observed the simple negative
sampling strategy of SimCSE is insufficient to learn effective long document
embeddings. In comparison, Trial2Vec leverages the meta-structure of clinical
trials to focus on the most informative attributes, with additional context-
based refinement, producing embeddings superior in semantic representation.

###  4.4 Exp 2. Partial Query Trial Retrieval

We further investigate the partial trial retrieval scenario where users intend
to find similar trials with short and incomplete descriptions, e.g., partial
attributes. Results are illustrated by Fig 2. We start by measuring how well
Trial2Vec only utilizes the title for trial retrieval. It is witnessed that
using title is sufficient to yield comparable performance as the best baseline
for complete retrieval shown in Table 3. Nonetheless, we identify that
concatenating keywords or intervention with the title reduces performance.
Combining title and disease yields similar performance as involving all
attributes. This phenomenon signifies that the disease plays a vital role in
trial similarity and is always recommended to be involved in query trial
retrieval.

###  4.5 Exp 3. Ablation Studies

We conducted ablation studies to measure how SSL tasks and embedding
dimensions contribute to final results. Results are shown by Fig. 4, where we
remove one Task for each setting and reevaluate. Here, att mc and ctx mc
corresponds to the global contrastive loss by negative sampling on key
attributes and contexts, respectively; semantic mc indicates the local
contrastive loss. We observe that ctx mc is very important. Without it, only
attributes of trials are included in the training and inference of Trial2Vec,
thus resulting in a significant performance drop. However, even only using a
small segment of trials (the attributes), Trial2Vec still reaches similar
performance as BERT-SimCSE that receives the whole trial document as inputs.
This demonstrates the importance of picking high-quality negative samples
during the CL process. Similarly, we observe other two tasks also improve the
retrieval quality.

Fig. 4 illustrates the retrieval performance on different embedding
dimensions. We identify that reducing embedding dimension does not affect the
performance of Trial2Vec much, i.e., one can choose a small embedding
dimension (e.g., 128) without suffering much performance degradation while
saving lots of storage and computational resources.

###  4.6 Exp 4. Embedding Space Visualization

Fig. 5 plots the 2D visualization of the embedding space of Trial2Vec using
t-SNE Van der Maaten and Hinton (2008) where around 2k trials uniformly
sampled from 300k trials. The tag texts illustrate the target diseases of
trials with different colors. We observe that these trials embeddings show
interpretable clusters corresponding to target disease categories. More
discussions about this visualization can be referred to Appendix B.

###  4.7 Exp 5. Trial Termination Prediction

Results are illustrated by Table 4. Compared with the shallow models, BERT-
based methods gain better performance, which credits the deep architecture of
transformers with stronger learning capability. Trial2Vec takes a hierarchical
encoding for trial documents on meta-structure thus better revealing the trial
characteristics, which plays a central role in predicting its potention
outcomes.

###  4.8 Exp 6. Case Study

We perform a qualitataive analysis of similarity search results and two
baselines. Results are shown in Table 5. These two case studies show that TF-
IDF and BERT models all tend to put attention on frequent words in query
trials, e.g., blood and iron in case study 1; and heart failure in case study
2. This bias comes from the average pooing taken onto all token embeddings.
The top-1 relevant clinical trial retrieved by Trial2Vec, on the other hand,
provides a more similar trial thanks to the hierarchical encoding and specific
local and global contrastive learning. We add more explanations regarding
these cases in Appendix C.

##  5 Conclusion

This paper investigated utilizing BERT with self-supervision for encoding
trial into dense embeddings for similarity search. Experiments show our method
can succeed in zero-shot trial search under various settings. The embeddings
are also useful for trial downstream predictive tasks. The qualitative
analysis, including embedding space visualization and case studies, further
verifies that Trial2Vec gets a medically meaningful understanding of clinical
trials.

## Limitations

The empirical evaluation of this method is mainly done on the clinical trial
documents drawn from ClinicalTrials.gov which were fully written in English.
It might be the best fit when this method is applied to documents in other
languages. Although we have tried our best to collect trial relevance
datasets, it is still possible that the datasets used for evaluation are not
able to cover all cases.

The proposed framework encodes trial documents into compact embeddings for
search. It encounters failure cases some time as wrong trials are retrieved.
It should be used with discretion when applied to clinical trial research or
by individual volunteers who intend to look for trials research. Retrieved
results in practice should be used under the supervision with professional
clinicians.

## References

  * Ainslie et al. (2020) Joshua Ainslie, Santiago Ontanon, Chris Alberti, Vaclav Cvicek, Zachary Fisher, Philip Pham, Anirudh Ravula, Sumit Sanghai, Qifan Wang, and Li Yang. 2020.  ETC: Encoding long and structured inputs in transformers.  In _Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 268–284.
  * Bao et al. (2021) Hangbo Bao, Li Dong, and Furu Wei. 2021.  BEiT: Bert pre-training of image transformers.  _arXiv preprint arXiv:2106.08254_.
  * Beltagy et al. (2020) Iz Beltagy, Matthew E Peters, and Arman Cohan. 2020.  Longformer: The long-document transformer.  _arXiv preprint arXiv:2004.05150_.
  * Bowman et al. (2015) Samuel R Bowman, Gabor Angeli, Christopher Potts, and Christopher D Manning. 2015\.  A large annotated corpus for learning natural language inference.  In _Conference on Empirical Methods in Natural Language Processing_ , pages 632–642.
  * Carlsson et al. (2020) Fredrik Carlsson, Amaru Cuba Gyllensten, Evangelia Gogoulou, Erik Ylipää Hellqvist, and Magnus Sahlgren. 2020.  Semantic re-tuning with contrastive tension.  In _International Conference on Learning Representations_.
  * Cer et al. (2017) Daniel Cer, Mona Diab, Eneko Agirre, Inigo Lopez-Gazpio, and Lucia Specia. 2017\.  Semeval-2017 task 1: Semantic textual similarity-multilingual and cross-lingual focused evaluation.  _arXiv preprint arXiv:1708.00055_.
  * Chang et al. (2019) Wei-Cheng Chang, X Yu Felix, Yin-Wen Chang, Yiming Yang, and Sanjiv Kumar. 2019\.  Pre-training tasks for embedding-based large-scale retrieval.  In _International Conference on Learning Representations_.
  * Chen et al. (2020a) Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020a.  A simple framework for contrastive learning of visual representations.  In _International Conference on Machine Learning_ , pages 1597–1607. PMLR.
  * Chen et al. (2020b) Ting Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi, and Geoffrey E Hinton. 2020b.  Big self-supervised models are strong semi-supervised learners.  _Advances in Neural Information Processing Systems_ , 33:22243–22255.
  * Chen et al. (2021) Xiaoyang Chen, Kai Hui, Ben He, Xianpei Han, Le Sun, and Zheng Ye. 2021.  Co-BERT: A context-aware bert retrieval model incorporating local and query-specific context.  _arXiv preprint arXiv:2104.08523_.
  * Chen and He (2021) Xinlei Chen and Kaiming He. 2021.  Exploring simple siamese representation learning.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 15750–15758.
  * Craswell et al. (2020) Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, and Ellen M Voorhees. 2020.  Overview of the trec 2019 deep learning track.  _arXiv preprint arXiv:2003.07820_.
  * Dehghani et al. (2017) Mostafa Dehghani, Hamed Zamani, Aliaksei Severyn, Jaap Kamps, and W Bruce Croft. 2017.  Neural ranking models with weak supervision.  In _International ACM SIGIR Conference on Research and Development in Information Retrieval_ , pages 65–74.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _NAACL-HLT_.
  * Ethayarajh (2019) Kawin Ethayarajh. 2019.  How contextual are contextualized word representations? comparing the geometry of bert, elmo, and gpt-2 embeddings.  In _Proceedings of the Conference on Empirical Methods in Natural Language Processing_ , pages 55–65.
  * Friedman et al. (2015) Lawrence M. Friedman, Curt D. Furberg, David L. DeMets, David M. Reboussin, and Christopher B. Granger. 2015.  _Fundamentals of Clinical Trials_. Springer, New York, NY.
  * Gao et al. (2021) Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021.  Simcse: Simple contrastive learning of sentence embeddings.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 6894–6910.
  * Guo et al. (2016) Jiafeng Guo, Yixing Fan, Qingyao Ai, and W Bruce Croft. 2016.  A deep relevance matching model for ad-hoc retrieval.  In _ACM International on Conference on Information and Knowledge Management_ , pages 55–64.
  * Guu et al. (2020) Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Wei Chang. 2020.  Realm: Retrieval-augmented language model pre-training.  _arXiv preprint arXiv:2002.08909_.
  * He et al. (2021) Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. 2021.  Masked autoencoders are scalable vision learners.  _arXiv preprint arXiv:2111.06377_.
  * Hofstätter et al. (2021a) Sebastian Hofstätter, Sheng-Chieh Lin, Jheng-Hong Yang, Jimmy Lin, and Allan Hanbury. 2021a.  Efficiently teaching an effective dense retriever with balanced topic aware sampling.  In _International ACM SIGIR Conference on Research and Development in Information Retrieval_ , pages 113–122.
  * Hofstätter et al. (2021b) Sebastian Hofstätter, Bhaskar Mitra, Hamed Zamani, Nick Craswell, and Allan Hanbury. 2021b.  Intra-document cascading: Learning to select passages for neural document ranking.  In _International ACM SIGIR Conference on Research and Development in Information Retrieval_ , pages 1349–1358.
  * Hofstätter et al. (2020) Sebastian Hofstätter, Hamed Zamani, Bhaskar Mitra, Nick Craswell, and Allan Hanbury. 2020.  Local self-attention over long text for efficient document retrieval.  In _Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval_ , pages 2021–2024.
  * Huang et al. (2021) Junjie Huang, Duyu Tang, Wanjun Zhong, Shuai Lu, Linjun Shou, Ming Gong, Daxin Jiang, and Nan Duan. 2021.  Whiteningbert: An easy unsupervised sentence embedding approach.  In _Findings of the Association for Computational Linguistics: EMNLP 2021_ , pages 238–244.
  * Humeau et al. (2019) Samuel Humeau, Kurt Shuster, Marie-Anne Lachaux, and Jason Weston. 2019.  Poly-encoders: Architectures and pre-training strategies for fast and accurate multi-sentence scoring.  In _International Conference on Learning Representations_.
  * Jiang and Weng (2014) Silis Y Jiang and Chunhua Weng. 2014.  Cross-system evaluation of clinical trial search engines.  _AMIA Summits on Translational Science Proceedings_ , 2014:223.
  * Jiang et al. (2022) Ting Jiang, Shaohan Huang, Zihan Zhang, Deqing Wang, Fuzhen Zhuang, Furu Wei, Haizhen Huang, Liangjie Zhang, and Qi Zhang. 2022.  PromptBERT: Improving bert sentence embeddings with prompts.  _arXiv preprint arXiv:2201.04337_.
  * Karpukhin et al. (2020) Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020.  Dense passage retrieval for open-domain question answering.  In _Conference on Empirical Methods in Natural Language Processing_ , pages 6769–6781.
  * Khattab and Zaharia (2020) Omar Khattab and Matei Zaharia. 2020.  ColBERT: Efficient and effective passage search via contextualized late interaction over BERT.  In _International ACM SIGIR Conference on Research and Development in Information Retrieval_ , pages 39–48.
  * Koopman and Zuccon (2016) Bevan Koopman and Guido Zuccon. 2016.  A test collection for matching patients to clinical trials.  In _International ACM SIGIR conference on Research and Development in Information Retrieval_ , pages 669–672.
  * Le and Mikolov (2014) Quoc Le and Tomas Mikolov. 2014.  Distributed representations of sentences and documents.  In _International Conference on Machine Learning_ , pages 1188–1196. PMLR.
  * Lee et al. (2020) Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang. 2020.  BioBERT: a pre-trained biomedical language representation model for biomedical text mining.  _Bioinformatics_ , 36(4):1234–1240.
  * Li et al. (2020) Bohan Li, Hao Zhou, Junxian He, Mingxuan Wang, Yiming Yang, and Lei Li. 2020.  On the sentence embeddings from pre-trained language models.  In _EMNLP_.
  * Li et al. (2021) Peizhao Li, Jiuxiang Gu, Jason Kuen, Vlad I Morariu, Handong Zhao, Rajiv Jain, Varun Manjunatha, and Hongfu Liu. 2021.  Selfdoc: Self-supervised document representation learning.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 5652–5660.
  * Lin et al. (2020) Sheng-Chieh Lin, Jheng-Hong Yang, and Jimmy Lin. 2020.  Distilling dense representations for ranking using tightly-coupled teachers.  _arXiv preprint arXiv:2010.11386_.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Roberta: A robustly optimized bert pretraining approach.  _arXiv preprint arXiv:1907.11692_.
  * Luan et al. (2021) Yi Luan, Jacob Eisenstein, Kristina Toutanova, and Michael Collins. 2021.  Sparse, dense, and attentional representations for text retrieval.  _Transactions of the Association for Computational Linguistics_ , 9:329–345.
  * Mikolov et al. (2013) Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013.  Efficient estimation of word representations in vector space.  _arXiv preprint arXiv:1301.3781_.
  * Nogueira and Cho (2019) Rodrigo Nogueira and Kyunghyun Cho. 2019.  Passage re-ranking with bert.  _arXiv preprint arXiv:1901.04085_.
  * Onal et al. (2018) Kezban Dilek Onal, Ye Zhang, Ismail Sengor Altingovde, Md Mustafizur Rahman, Pinar Karagoz, Alex Braylan, Brandon Dang, Heng-Lu Chang, Henna Kim, Quinten McNamara, et al. 2018.  Neural information retrieval: At the end of the early years.  _Information Retrieval Journal_ , 21(2):111–182.
  * Park et al. (2020) Junseok Park, Seongkuk Park, Kwangmin Kim, Woochang Hwang, Sunyong Yoo, Gwan-su Yi, and Doheon Lee. 2020.  An interactive retrieval system for clinical trial studies with context-dependent protocol elements.  _PloS one_ , 15(9):e0238290.
  * Pennington et al. (2014) Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014.  Glove: Global vectors for word representation.  In _Conference on Empirical Methods in Natural Language Processing_ , pages 1532–1543.
  * Pradeep et al. (2022) Ronak Pradeep, Yilin Li, Yuetong Wang, and Jimmy Lin. 2022.  Neural query synthesis and domain-specific ranking templates for multi-stage clinical trial matching.  In _International ACM SIGIR Conference on Research and Development in Information Retrieval_.
  * Raffel et al. (2020) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J Liu, et al. 2020.  Exploring the limits of transfer learning with a unified text-to-text transformer.  _J. Mach. Learn. Res._ , 21(140):1–67.
  * Reimers and Gurevych (2019) Nils Reimers and Iryna Gurevych. 2019.  Sentence-BERT: Sentence embeddings using siamese bert-networks.  In _Conference on Empirical Methods in Natural Language Processing_ , pages 3982–3992.
  * Roberts et al. (2021) Kirk Roberts, Dina Demner-Fushman, Ellen M. Voorhees, Steven Bedrick, and Willian R. Hersh. 2021.  Overview of the trec 2021 clinical trials track.  In _Proceedings of the Thirteenth Text REtrieval Conference (TREC 2021)_.
  * Robertson and Zaragoza (2009) Stephen Robertson and Hugo Zaragoza. 2009.  _The probabilistic relevance framework: BM25 and beyond_.  Now Publishers Inc.
  * Roy et al. (2019) Soumyadeep Roy, Koustav Rudra, Nikhil Agrawal, Shamik Sural, and Niloy Ganguly. 2019\.  Towards an aspect-based ranking model for clinical trial search.  In _International Conference on Computational Data and Social Networks_ , pages 209–222. Springer.
  * Rybinski et al. (2021) Maciej Rybinski, Sarvnaz Karimi, and Aleney Khoo. 2021.  Science2Cure: A clinical trial search prototype.  In _Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval_ , pages 2620–2624.
  * Rybinski et al. (2020) Maciej Rybinski, Jerry Xu, and Sarvnaz Karimi. 2020.  Clinical trial search: Using biomedical language understanding models for re-ranking.  _Journal of Biomedical Informatics_ , 109:103530.
  * Salton and Buckley (1988) Gerard Salton and Christopher Buckley. 1988.  Term-weighting approaches in automatic text retrieval.  _Information Processing & Management_, 24(5):513–523.
  * Salton et al. (1983) Gerard Salton, Edward A Fox, and Harry Wu. 1983.  Extended boolean information retrieval.  _Communications of the ACM_ , 26(11):1022–1036.
  * Su et al. (2021) Jianlin Su, Jiarun Cao, Weijie Liu, and Yangyiwen Ou. 2021.  Whitening sentence representations for better semantics and faster retrieval.  _arXiv preprint arXiv:2103.15316_.
  * Tasneem et al. (2012) Asba Tasneem, Laura Aberle, Hari Ananth, Swati Chakraborty, Karen Chiswell, Brian J McCourt, and Ricardo Pietrobon. 2012.  The database for aggregate analysis of clinicaltrials. gov (aact) and subsequent regrouping by clinical specialty.  _PloS one_ , 7(3):e33677.
  * Trotman et al. (2014) Andrew Trotman, Antti Puurula, and Blake Burgess. 2014.  Improvements to bm25 and language models examined.  In _Australasian Document Computing Symposium_ , pages 58–65.
  * Tsatsaronis et al. (2012) George Tsatsaronis, Konstantinos Mourtzoukos, Vassiliki Andronikou, Tassos Tagaris, Iraklis Varlamis, Michael Schroeder, Theodora Varvarigou, Dimitris Koutsouris, and Nikolaos Matskanis. 2012.  PONTE: a context-aware approach for automated clinical trial protocol design.  In _proceedings of the 6th International Workshop on Personalized Access, Profile Management, and Context Awareness in Databases in conjunction with VLDB_.
  * Van der Maaten and Hinton (2008) Laurens Van der Maaten and Geoffrey Hinton. 2008.  Visualizing data using t-SNE.  _Journal of Machine Learning Research_ , 9(11).
  * Van Gysel et al. (2016) Christophe Van Gysel, Maarten de Rijke, and Evangelos Kanoulas. 2016.  Learning latent vector spaces for product search.  In _ACM International on Conference on Information and Knowledge Management_ , pages 165–174.
  * Wang et al. (2022a) Hao Wang, Yangguang Li, Zhen Huang, Yong Dou, Lingpeng Kong, and Jing Shao. 2022a.  SNCSE: Contrastive learning for unsupervised sentence embedding with soft negative samples.  _arXiv preprint arXiv:2201.05979_.
  * Wang et al. (2020a) Shuohang Wang, Yuwei Fang, Siqi Sun, Zhe Gan, Yu Cheng, Jingjing Liu, and Jing Jiang. 2020a.  Cross-thought for sentence encoder pre-training.  In _Conference on Empirical Methods in Natural Language Processing_ , pages 412–421.
  * Wang et al. (2020b) Zifeng Wang, Xi Chen, Rui Wen, Shao-Lun Huang, Ercan Kuruoglu, and Yefeng Zheng. 2020b.  Information theoretic counterfactual learning from missing-not-at-random feedback.  _Advances in Neural Information Processing Systems_ , 33:1854–1864.
  * Wang et al. (2022b) Zifeng Wang, Chufan Gao, Lucas M Glass, and Jimeng Sun. 2022b.  Artificial intelligence for in silico clinical trials: A review.  _arXiv preprint arXiv:2209.09023_.
  * Wang and Sun (2022) Zifeng Wang and Jimeng Sun. 2022.  Transtab: Learning transferable tabular transformers across tables.  _arXiv preprint arXiv:2205.09328_.
  * Wang et al. (2021) Zifeng Wang, Rui Wen, Xi Chen, Shilei Cao, Shao-Lun Huang, Buyue Qian, and Yefeng Zheng. 2021.  Online disease diagnosis with inductive heterogeneous graph convolutional networks.  In _Proceedings of the Web Conference_ , pages 3349–3358.
  * Wang et al. (2022c) Zifeng Wang, Zhenbang Wu, Dinesh Agarwal, and Jimeng Sun. 2022c.  MedCLIP: Contrastive learning from unpaired medical images and text.  In _EMNLP_.
  * Wu et al. (2021) Xing Wu, Chaochen Gao, Liangjun Zang, Jizhong Han, Zhongyuan Wang, and Songlin Hu. 2021.  ESimCSE: Enhanced sample building method for contrastive learning of unsupervised sentence embedding.  _arXiv preprint arXiv:2109.04380_.
  * Wu et al. (2020) Zhuofeng Wu, Sinong Wang, Jiatao Gu, Madian Khabsa, Fei Sun, and Hao Ma. 2020.  CLEAR: Contrastive learning for sentence representation.  _arXiv preprint arXiv:2012.15466_.
  * Xiong et al. (2020) Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul N Bennett, Junaid Ahmed, and Arnold Overwijk. 2020.  Approximate nearest neighbor negative contrastive learning for dense text retrieval.  In _International Conference on Learning Representations_.
  * Yan et al. (2021) Yuanmeng Yan, Rumei Li, Sirui Wang, Fuzheng Zhang, Wei Wu, and Weiran Xu. 2021.  ConSERT: A contrastive framework for self-supervised sentence representation transfer.  _arXiv preprint arXiv:2105.11741_.
  * Yang et al. (2017) Peilin Yang, Hui Fang, and Jimmy Lin. 2017.  Anserini: Enabling the use of lucene for information retrieval research.  In _International ACM SIGIR Conference on Research and Development in Information Retrieval_ , pages 1253–1256.
  * Yang et al. (2021) Ziyi Yang, Yinfei Yang, Daniel Cer, Jax Law, and Eric Darve. 2021.  Universal sentence representation learning with conditional masked language model.  In _Conference on Empirical Methods in Natural Language Processing_ , pages 6216–6228.
  * Yates et al. (2021) Andrew Yates, Rodrigo Nogueira, and Jimmy Lin. 2021.  Pretrained transformers for text ranking: Bert and beyond.  In _ACM International Conference on Web Search and Data Mining_ , pages 1154–1156.
  * Zaheer et al. (2020) Manzil Zaheer, Guru Guruganesh, Kumar Avinava Dubey, Joshua Ainslie, Chris Alberti, Santiago Ontanon, Philip Pham, Anirudh Ravula, Qifan Wang, Li Yang, et al. 2020.  Big bird: Transformers for longer sequences.  _Advances in Neural Information Processing Systems_ , 33:17283–17297.
  * Zamani et al. (2018) Hamed Zamani, Mostafa Dehghani, W Bruce Croft, Erik Learned-Miller, and Jaap Kamps. 2018.  From neural re-ranking to neural ranking: Learning a sparse representation for inverted indexing.  In _ACM International Conference on Information and Knowledge Management_ , pages 497–506.
  * Zhan et al. (2021) Jingtao Zhan, Jiaxin Mao, Yiqun Liu, Jiafeng Guo, Min Zhang, and Shaoping Ma. 2021\.  Optimizing dense retrieval model training with hard negatives.  In _International ACM SIGIR Conference on Research and Development in Information Retrieval_ , pages 1503–1512.
  * Zhang et al. (2021) Hang Zhang, Yeyun Gong, Yelong Shen, Weisheng Li, Jiancheng Lv, Nan Duan, and Weizhu Chen. 2021.  Poolingformer: Long document modeling with pooling attention.  In _International Conference on Machine Learning_ , pages 12437–12446. PMLR.
  * Zhang et al. (2020) Yan Zhang, Ruidan He, Zuozhu Liu, Kwan Hui Lim, and Lidong Bing. 2020.  An unsupervised sentence embedding method by mutual information maximization.  In _Conference on Empirical Methods in Natural Language Processing_ , pages 1601–1610.
  * Zhang et al. (2022) Yu Zhang, Zhihong Shen, Chieh-Han Wu, Boya Xie, Junheng Hao, Ye-Yi Wang, Kuansan Wang, and Jiawei Han. 2022.  Metadata-induced contrastive learning for zero-shot multi-label text classification.  _arXiv preprint arXiv:2202.05932_.

Table 6: List of text corpora used for continual pretraining of TrialBERT.

Corpus | Number of words
---|---
ClinicalTrials.gov | 240M
Medical Encyclopedia | 3M
Wikipedia Articles | 11M

##  Appendix A Baselines for clinical trial similarity search

  * •

TF-IDF Salton et al. (1983); Salton and Buckley (1988). It is short for term
frequency–inverse document frequency that has been widely used for information
retrieval systems for decades. One can use TF-IDF for document retrieval by
concatenating scores of all words in this document then computing cosine
distance between document vectors.

  * •

BM25 Trotman et al. (2014). A bag-of-words retrieval method commonly used in
practice. We run it based on the rank-bm25 package
888https://pypi.org/project/rank-bm25/ with its default hyperparameters.

  * •

Word2Vec Mikolov et al. (2013). It is a classic dense retrieval method by
building distributed word representations by self-supervised learning methods
(CBOW). We take an average pooling of word representations in a document for
retrieval by cosine distance. We use gensim
999https://radimrehurek.com/gensim/models/doc2vec.html to run this method.

  * •

BERT. We take an average pooling over all token embeddings at the last layer
of it for similarity computation. We take the TrialBERT pretrained on all the
clinical trial documents.

  * •

BERT-Whitening Huang et al. (2021); Su et al. (2021). This is an unsupervised
post-processing method that uses anisotropic BERT embeddings Ethayarajh
(2019); Li et al. (2020) to improve semantic search. We take the average of
last and first layer of its BERT embeddings following Su et al. (2021).

  * •

BERT-SimCSE Gao et al. (2021). It is a contrastive sentence representation
learning method stemming from InfoNCE loss. It simply takes other samples in
batch as negative samples.

  * •

MonoT5-Med Pradeep et al. (2022). It was proposed in Roberts et al. (2021) for
matching patient descriptive texts and clinical trial documents via T5 model
Raffel et al. (2020) based on prompts. We use its version finetuned on Med
Marco dataset Koopman and Zuccon (2016).

##  Appendix B Embedding space visualization

From Fig. 5, trial embeddings are clearly clustered into topics with self-
supervised learning, which provides a great help for topic mining and
discovery for the existing clinical trials. For instance, we can find that
cancers that happen on different body parts are near to each other on the
bottom of the embedding space (Prostate Cancer, Breast Cancer, Pancreatic
Cancer, Colorectal Cancer, etc.). Also, the diseases which are related to
brain function, e.g., Alzheimer’s Disease, Parkinson’s Disease, Major
Depressive Disorder, etc. Other examples include Covid19, Influenza, Pulmonary
Disease, etc.

The reason is that we explicitly utilize the knowledge from attributes of
trials for negative sample building, which endows the embedding space the
ability to discriminate trials’ similarity. These similar trials can also have
similar characteristics like having similar recruiting criteria or targeting
similar outcome measures, which are captured by Trial2Vec by refining the
embeddings of attributes by detailed descriptions. Based on this observation,
we can infer that such medically meaningful trial embeddings would be
beneficial to downstream tasks on clinical trials, e.g., trial outcome
prediction.

##  Appendix C Case Study

For the first case, the query trial is [NCT02972294], which studies using
Tranexamic acid and Iron Isomaltoside to reduce the occurrence of Anemia and
blood transfusion in hip fracture cases. We show the top-1 retrieved by three
methods on the right. Trial found by TF-IDF studies the efficiency of plasma
in patients with Hemorrhagic shock; BioBERT finds a trial about patients
undergoing heart surgery who have Anaemia to test if a correction of iron
reduces red blood cell transfusion requirements. Trial2Vec finds a trial that
studies Tranexamic acid effect in blood loss in hip fracture operations.
Trial2Vec result is highly relevant to the query trial as it has the identical
drug on blood loss of the same type of operation.

In the second example, the query trial tries to investigate the benefits of
Diclofenac for Normotensive patients with acute symptomatic Pulmonary Embolism
and Right Ventricular Dysfunction. TF-IDF finds an irrelevant study on the
efficacy and safety of Elobixibat for adults with NAFLD or NASH. TrialBERT
also retrieves an irrelevant study on Intravascular Volume Expansion to
Neuroendocrine-Renal Function Profiles in Chronic Heart Failure. On the other
hand, Trial2Vec digs out a trial that studies the same type of drug with a
similar purpose as the target’s: evaluating the efficiency of NSAID
(Diclofenac) to the evolution of postoperative (cardiac surgery) pericardial
effusion.