#  Augmenting LLMs with Knowledge
A survey on hallucination prevention
— student project —

Konstantinos Andriopoulos  Delft University of Technology
Delft, The Netherlands
<EMAIL_ADDRESS>Johan Pouwelse  Delft University of
Technology
Delft, The Netherlands
<EMAIL_ADDRESS>
###### Abstract

Large pre-trained language models have demonstrated their proficiency in
storing factual knowledge within their parameters and achieving remarkable
results when fine-tuned for downstream natural language processing tasks.
Nonetheless, their capacity to access and manipulate knowledge with precision
remains constrained, resulting in performance disparities on knowledge-
intensive tasks when compared to task-specific architectures. Additionally,
the challenges of providing provenance for model decisions and maintaining up-
to-date world knowledge persist as open research frontiers. To address these
limitations, the integration of pre-trained models with differentiable access
mechanisms to explicit non-parametric memory emerges as a promising solution.
This survey delves into the realm of language models (LMs) augmented with the
ability to tap into external knowledge sources, including external knowledge
bases and search engines. While adhering to the standard objective of
predicting missing tokens, these augmented LMs leverage diverse, possibly non-
parametric external modules to augment their contextual processing
capabilities, departing from the conventional language modeling paradigm.
Through an exploration of current advancements in augmenting large language
models with knowledge, this work concludes that this emerging research
direction holds the potential to address prevalent issues in traditional LMs,
such as hallucinations, un-grounded responses, and scalability challenges.

##  I Introduction

Large Language Models (LLMs) have brought about remarkable advancements in
Natural Language Processing (NLP) and are now integral to various widely-used
products, including Copilot [1], Google’s search engine, and more recently,
Chat-GPT, a chatbot built upon GPT3 [2]. These models, characterized by their
memorization capabilities as well as their compositional prowess, have
demonstrated unprecedented performance in tasks ranging from language
understanding to text generation, paving the way for more sophisticated human-
computer interactions.

However, LLMs are not without their limitations. They often produce seemingly
plausible yet incorrect predictions, a phenomenon known as hallucinations [3],
leading to avoidable errors in various contexts. Furthermore, many of the
groundbreaking capabilities of LLMs appear to scale with the model’s size in
terms of trainable parameters. While recent efforts have produced smaller LLMs
with retained capabilities [4], the practicality of training and maintaining
large models remains a challenge, with continual learning for such models
posing an ongoing research question [5].

These limitations are rooted in a fundamental issue with LLMs: they are
primarily trained for statistical language modeling, relying on a single
parametric model and a relatively limited context, typically the preceding ”n”
tokens. Despite advancements in hardware and software, most models still
employ relatively small context sizes compared to the expansive context
required for accurate language modeling in all scenarios. Consequently,
achieving the necessary scale to store the knowledge beyond the immediate
context has become a necessity.

In response, a growing research trend has emerged, moving away from the
traditional statistical language modeling paradigm. One approach addresses the
limited context size of LLMs by enhancing its relevance through the
incorporation of information extracted from external documents [6] [7]. By
equipping language models with modules that retrieve relevant documents from
databases based on the context, it becomes possible to replicate certain
capabilities of larger LLMs while using fewer parameters [8] [9].

Moreover, in this evolving landscape, pioneering models [10] [11] that
leverage structured knowledge stand out. These models leverage knowledge
graphs along with a corpus of supporting documents, which can be jointly
processed by Graph Convolutional Neural Networks (CNNs). By harnessing graph-
based representations, these structured-knowledge augmented models excel in
generating precise responses to open-domain questions. This innovative use of
structured knowledge marks a significant advancement in enhancing language
models, demonstrating the diverse strategies researchers are adopting to
address the limitations of contemporary LLMs.

It is worth noting that such approaches transform the resulting models into a
non-parametric ones, as they can now effectively query external data sources.

Another strategy involves enabling LLMs to leverage external tools [12], such
as search engines [13] [14] [12], allowing them to augment the current context
with crucial missing information not contained within the model’s weights.
Although most of these efforts aim to address individual shortcomings of LLMs,
it is evident that a more comprehensive integration of knowledge tools has the
potential to significantly enhance the capabilities of these models.

In light of these recent developments in NLP, there is a pressing need for a
comprehensive taxonomy of augmented language models and clear definitions of
the technical terminology used, which sometimes carry varying interpretations
and intentions.

##  II Background

As we delve into the intricacies of augmenting Large Language Models (LLMs)
with external knowledge, it is imperative to establish a foundational
understanding of the key concepts that underpin this transformative field.
Knowledge augmentation strategies, such as harnessing knowledge graphs,
employing beam search techniques, leveraging triplestore databases, and
integrating sequence-to-sequence models, form the bedrock upon which advanced
language models now stand. In this section, we embark on a comprehensive
exploration of these pivotal concepts, unraveling their significance,
methodologies, and interconnectedness. By elucidating these fundamental
building blocks, we pave the way for a profound understanding of how
contemporary LLMs harness external knowledge to achieve unprecedented
linguistic feats.

###  II-A Generative Language Models

Generative language models are trained to produce new text, given an input
sequence of tokens. They are able to perform this by learning the statistical
relationships between words and phrases in a large corpus of text. When given
a prompt, a generative model will try to produce text that is consistent with
the statistical patterns it has learned.

Some of the most popular generative models in natural language processing
include autoregressive models [15], variational autoencoders (VAEs) [16] and
Generative adversarial networks (GANs) [17]. In this literature survey, we
will mostly explore Transformers, autoregressive models, along with another
type of generative language models, sequence-to-sequence models.

###  II-B Autoregressive Models

An autoregressive model [15] is a type of neural network used for generating
sequences of data, where each element in the sequence is predicted one at a
time based on the previously generated elements. In other words, the model
generates data by conditioning its predictions on the data it has generated so
far. Autoregressive models are typically used for tasks like text generation,
time series forecasting, and speech synthesis.

One of the most well-known autoregressive models in NLP is the GPT (Generative
Pre-trained Transformer) series, such as GPT-2 [18] and GPT-3 [2]. These
models generate text by predicting the next word in a sentence based on the
preceding words. They use self-attention [19] mechanisms to capture
dependencies between words at different positions in the sequence, making them
capable of generating coherent and contextually relevant text.

###  II-C Sequence-to-sequence Models

A sequence-to-sequence (seq2seq) model [20] predicts the probability of a
token being the next token in a given sequence of words.

It consists of an encoder and a decoder. The encoder reads the input sequence,
one step at a time and produces a fixed-dimensional vector representation of
the entire sequence. This vector is called a context vector and it is a
representation of all the meaningful information of the input sequence. The
context vector is then passed to the decoder, which generates an output
sequence.

Sequence-to-sequence models are typically trained using a maximum likelihood
objective, which means that they are trained to produce the output sequence
that is most likely to follow the input sequence. In summary, seq2seq models
are designed for tasks involving the transformation of one sequence into
another, often with different lengths and structures. They are typically
applied to tasks such as: machine translation, text summarization, and
question-answering, where the relationship between the input and output
sequences is not purely linear or where the lengths of input and output
sequences can vary significantly.

From this point and onwards, we will refer to sequence-to-sequence models as
just seq2seq.

###  II-D Transformers

The Transformer architecture [19] marked a groundbreaking advancement in the
field of NLP. Since its inception, Transformers have become the backbone of
various state-of-the-art language models, underpinning many of the recent
developments in the realm of augmented language models.

At its core, the Transformer architecture revolutionized sequence-to-sequence
modeling through the introduction of the attention mechanism. Unlike earlier
recurrent neural networks (RNNs) [21] [22] and convolutional neural networks
(CNNs) [23], Transformers rely on self-attention mechanisms to capture
dependencies between elements in a sequence, making them highly parallelizable
and efficient for processing long-range dependencies.

The architecture consists of two main components: the encoder and the decoder.
The encoder processes the input sequence, while the decoder generates the
output sequence. Each component comprises multiple layers, with each layer
containing a multi-head self-attention mechanism and feed-forward neural
networks. These self-attention mechanisms enable Transformers to capture
contextual information efficiently, making them ideal for tasks that involve
understanding and generating sequences of data.

In the context of language modeling, Transformers can be adapted to function
as decoder-only models. In decoder-only Transformers, the encoder component,
which is used for encoding input sequences, is removed. These models retain
the core Transformer architecture but focus exclusively on generating
sequences of tokens, making them particularly suitable for autoregressive
language modeling tasks.

Decoder-only Transformers operate in an autoregressive manner. They generate
sequences one token at a time, with each token’s prediction conditioned on the
previously generated tokens. This autoregressive approach allows them to
produce coherent and contextually relevant text. Decoder-only Transformers
have been instrumental in various text generation tasks, including machine
translation, text summarization, and text completion.

Since the introduction of the Transformer architecture, numerous variants and
extensions have emerged, each tailored to address specific challenges in NLP.
These variants include models such as BERT (Bidirectional Encoder
Representations from Transformers) [24], GPT (Generative Pre-trained
Transformer) [18] [2], and T5 (Text-to-Text Transfer Transformer) [25], among
others. Many of these models have laid the foundation for augmenting language
models with external knowledge, a topic of great interest in recent NLP
research.

###  II-E Beam Search

Beam Search is a heuristic search algorithm that explores a graph, G, by
expanding only the K (beam width) most promising nodes at each step. Beam
Search simulates the behavior of Breadth-First Search. More specifically, it
uses BFS to create a search tree. At each level of the tree, it checks all the
successors of the current level and keeps only the top K ones, while pruning
the others. The process repeats until K leaves are found. Beam search will
return the leaf that maximizes some given score function.

In the context of NLP, when using a generative model, Beam Search is utilized
to find the sequence $y=(y_{1},...,y_{n})$ that is most likely to come after
an input sequence $x$. In mathematic notation, the probability to maximize is:

$\begin{split}p(y|x)&=p(y_{n}|x,y_{1...n-1})\cdot p(y{1...n-1}|x)\\\
&=p(y_{n}|x,y_{1...n-1})\cdot p(y_{n-1}|x,y_{1...n-2})\cdot...\cdot
p(y_{1}|x)\\\ \end{split}$ (1)

Instead of choosing only the output token with the highest probability each
time, beam search chooses the top K tokens with the highest probability and
explores the generated sequences recursively until we reach an $<EOS>$ (end-
of-sequence) token. Then, it returns sequence $y$ (out of the K sequences)
that maximizes $p(y|x)$.

In the following sections, we will explore some concepts that are pivotal to
the understanding of state-of-the-art augmentation of LLMs.

###  II-F Text Corpus

A text corpus, $D$ is a set of documents: ${d_{1},...,d_{|D|}}$ where each
document is a sequence of words: $d_{i}=(w_{1},...,w_{|d_{i}|})$.
Specifically, in the context of this paper, a document is essentially a
sentence, and an article is a collection of documents.

As we will see later on in this survey, text corpora are considered an
unstructured knowledge base and are usually organized in vector databases.

###  II-G Vector Database

In a vector database, a document can correspond to one vector or many vectors,
depending on the specific implementation of the database. A single vector
captures the overall meaning of the document. This is often done by averaging
the vectors of the words in the document. In other cases, a document may be
represented by a vector for each word in the document. This is often done when
it is important to be able to track the individual words in the document.

When a language model retrieves information from a vector database, it
essentially has access to knowledge that is not stored in its parameters
(weights). Therefore, a vector database is a form of non-parametric memory for
LLMs.

###  II-H Dense Vector Index

Indexing in a vector database is the process of organizing the vectors in the
database in a way that makes it efficient to search for and retrieve similar
vectors (vectors with a high inner product). This is accomplished by creating
a data structure that maps each vector to a set of other vectors that are
similar to it.

Maximum Inner Product Search (MIPS) is a specific type of vector search that
aims to find the vector in the database with the highest inner product with a
given query vector. MIPS is used in a variety of applications, such as
recommendation systems, machine learning, and image retrieval.

FAISS [26] is a popular open-source library for efficient similarity search
and clustering of dense vectors. FAISS contains a variety of algorithms for
MIPS, as well as other types of vector search. FAISS is used by many companies
and organizations, including Google, Facebook, and Microsoft.

###  II-I Triplestore Knowledge Bases

A Triplestore knowledge base is a database that consists of subject-predicate-
object triples. An example of such a triple is: (Subject: Albert Einstein,
Predicate: was born in, Object: Ulm, Germany). Triples are a great form of
representing factual knowledge because they capture the nature of the
relationship between a subject and an object and can be easily processed by
LLMs. One can visualize this knowledge base as a graph whose vertices are the
various subjects and objects (entities) and the predicates are the edges
between these entities.

Each edge has a type (e.g: ”was born in”) that describes the kind of the
relation between the connected entities. Triplestore knowledge bases with more
than one types of relations are called heterogeneous.

Triplestores are an excellent example of what we call structured knowledge
bases. They can be merged with unstructured knowledge bases through a set of
entity links: $(v,d_{p})$, connecting entity $v$ with a word at position $p$,
in document $d$.

###  II-J Graph Convolutional Networks

Graph convolutional networks (GCNs) are a type of neural network that can be
used to learn representations of nodes in a structured knowledge base, such as
a graph. GCNs are particularly well-suited for node classification tasks,
where the goal is to predict the label of each node in the graph (e.g: whether
the node contains an answer to a given question or not).

GCNs work by iteratively aggregating information from the neighbors of each
node. At each layer, the GCN collects the embeddings of all of a node’s
neighbors, averages them, and then applies a linear transformation and a
nonlinear activation function. The output of this layer is then used as the
input to the next layer.

The more layers the GCN has, the more multi-hop reasoning the model will be
able to perform, because it will gather information from more far away
neighbors. This makes GCNs well-suited for tasks where the labels of nodes
depend on the labels of their neighbors, such as social network analysis and
fraud detection.

Here is a high-level overview of how a GCN works for node classification:

  1. 1.

Initialize the embeddings of all nodes in the graph.

  2. 2.

For each node in the graph:

    1. (a)

Collect the embeddings of all of the node’s neighbors.

    2. (b)

Average the embeddings of the node’s neighbors.

    3. (c)

Apply a linear transformation and a nonlinear activation function to the
average embedding.

    4. (d)

The output of this function is the new embedding for the node.

  3. 3.

Repeat step 2 for a fixed number of layers.

  4. 4.

The final embedding of each node is used as the input to a classifier to
predict the node’s label.

###  II-K Relational Graph Convolutional Networks

One problem that arises when the knowledge-base graph heterogeneous is that,
in that case, we want to take into consideration the type of relation that a
node has with its neighbors before we average their embeddings.

A relational GCN [27] is similar to a regular GCN, but it uses a separate
matrix for each type of relation. Therefore, when using a relational GCN, we
aggregate the embeddings from all neighbors with a specific relation and we
pass the averaged embedding into a separate CNN layer for each relation.

##  III Knowledge Base Augmented Generation

Language models have the ability to store knowledge in their parameters.
Alternatively, knowledge in the form of natural language can be offloaded
completely from the LM by retrieving from an external knowledge base. Memory
augmentation strategies help the language model to avoid producing non-factual
information as well as reducing the number of parameters required to achieve
comparable performance to significantly larger LMs. Based on their structure,
knowledge bases can be either unstructured (text-based) or structured (graph-
based). In this literature survey, we are going to explore work from both
worlds.

Figure 1: Overview of knowledge augmentation of language models from the paper
by Izacard et al.[7]. The input query (light yellow), along with a number of
retrieved relevant documents (light blue), passes through the generative
seq2seq model to produce an output response.

###  III-A Retrieval-Augmented Generation (RAG)

RAG [6] uses both parametric and non-parametric memory to generate more
accurate and informative responses to an input query.

Specifically, the RAG architecture entails:

  * •

a generator: a BART-large [28] sequence-to-sequence language model, pre-
trained on a massive dataset of text and code (parametric memory).

  * •

a knowledge base: a dense vector index of the Wikipedia database (non-
parametric memory). All documents in the knowledge base are also encoded as
vectors using a $BERT_{BASE}$ [24] document encoder, $BERT_{d}$.

  * •

a retriever: a component that is responsible for retrieving the documents of
the knowledge base that are most relevant to the input query. It follows the
DPR (dense passage retrieval) architecture [29] and it consists of a document
encoder, $BERT_{d}$ and a query encoder, $BERT_{q}$. The retriever

    * –

calculates the embedding of the input query, using the $BERT_{q}$ encoder.

    * –

conducts Maximum Inner Product Search (MIPS) in the indexed knowledge base to
find the K most similar documents to the input query

According to the authors of RAG, training and fine-tuning the parameters of
the $BERT_{d}$ encoder is extremely computationally expensive, and not very
effective accuracy-wise. Specifically, if they were to train the parameters of
$BERT_{d}$, then for each training iteration, the embeddings of each document
in the $BERT_{BASE}$ knowledge base would have to be updated as well, so that
they are in-sync with the new $BERT_{d}$ encoder.

Therefore, they use a completely pre-trained $BERT_{d}$ encoder, and during
the fine-tuning stage, they only fine-tune the parameters of the query encoder
$BERT_{q}$.

One interesting aspect of RAG is how it implements the fusion of knowledge
from all retrieved documents to produce a final response. In both proposed
versions of RAG, RAG-token and RAG-sequence, fusion is performed right after
the decoder.

Specifically, RAG-token:

  * •

for each retrieved document $z$, calculates the probability for each token
$y_{i}$ in the vocabulary to be the next token in the sequence:

$p_{\theta}(y_{i}|x,z,y_{1:i-1})\\\ $ (2)

  * •

sums the probabilities over all retrieved documents (marginalization):

$p_{\theta}^{{}^{\prime}}(y_{i}|x,y_{1:i-1})=\sum_{z}{p_{\eta}(z|x)\cdot
p_{\theta}(y_{i}|x,z,y_{1:i-1})}\\\ $ (3)

  * •

runs Beam Search to find the K most likely next tokens

  * •

chooses the token, $y_{i}$ with the highest transition probability

The RAG-sequence model is quite easier to grasp. It takes into account only
one retrieved document per sequence that it generates. Specifically, for each
retrieved document, it conducts Beam Search to generate K sequences. Then, it
simply returns the sequence with the highest probability.

###  III-B REALM [30]

REALM was the first method that managed to pre-train jointly the retriever and
the generator. The authors of REALM propose three stages of training for the
given architecture:

  * •

initialization

  * •

pre-training

  * •

fine-tuning

One significant challenge that REALM faced was the fact that, at the beginning
of training, the query and document encoders, $Embed_{input}$ and
$Embed_{doc}$ respectively contain completely random parameters. Hence, the
retrieved documents, z, will likely be unrelated to the input query, x. As a
result, the Generator learns to ignore the retrieved documents. Once this
occurs, during training, the Retriever no longer receives a meaningful
gradient and cannot improve, creating a vicious cycle that does not result in
an accurate end model.

To avoid this cold-start problem, the authors warm-start (initialization) the
Retriever ($Embed_{input}$ \+ $Embed_{doc}$) using a training objective known
as the Inverse Cloze Task (ICT) [31] where, given a sentence, the model is
trained to retrieve the document where that sentence came from.

In the case of the Generator, the authors warm-start it with BERT pre-training
[24] and they use the uncased BERT-base model (12 layers, 768 hidden units, 12
attention heads).

After the initialization stage, the REALM proposes an unsupervised pre-
training method. During the pre-training iteration, REALM:

  1. 1.

randomly selects sentences from the text corpus and masks specific tokens from
each sentence

  2. 2.

receives a masked query, q, as input. An example of that query would be: ”The
[MASK] at the top of the pyramid”

  3. 3.

outputs its token prediction (correct answer is ”pyramidion”)

  4. 4.

back-propagates through the parameters, $\theta$ of the the retriever
$p_{\theta}(z|x)$, and $\phi$, of the generator $p_{\phi}(z|x)$ (joint pre-
training of the models).

During pre-training, both the $Embed_{doc}$ and the $Embed_{input}$ components
of the Retriever are updated. Because the parameters of $Embed_{doc}$ are
updated during pre-training, in order for the document embeddings in the
Wikipedia knowledge base to stay in-sync with the updated Retriever, after
each back-propagation step, REALM needs to:

  1. 1.

re-compute the document embeddings

  2. 2.

re-calculate the document index (in order to perform MIPS)

This is a computationally expensive task, especially for really large
databases, such as Wikipedia. Therefore, REALM was designed such that the
embedding updates happen every 100 back-propagation steps, as an asynchronous
process.

The supervised fine-tuning method that the authors used in order to evaluate
REALM on Open-domain Question Answering (Open-QA) goes as follows: 1\. they
collect question-answer tuples, such as: (”What’s the angle of an equilateral
triangle”, ”60 degrees”). 4\. REALM receives the question as input. 5\. it
outputs its prediction. 6\. similar to the pre-training phase, REALM back-
propagates through the parameters of the the retriever $p_{\theta}(z|x)$, and
$\phi$, of the generator $p_{\phi}(z|x)$, but this time $Embed_{doc}$ stays
untouched. Therefore, fine-tuning is much less computationally expensive.

###  III-C Fusion in Decoder (FiD)

FiD [7] employs a similar but quite simpler idea to RAG. Their main
difference, however, lies in the way they perform the fusion of the retrieved
knowledge.

Similar to RAG, in FiD, we have two main models:

  * •

the retriever which has access to a $BERT_{BASE}$ where documents are
represented as dense vectors and retrieves the most relevant documents by
running Maximum Inner Product Search (MIPS) using the FAISS library [26]

  * •

the generator which is a sequence-to-sequence model that receives the input
query concatenated with a retrieved passage and is trained to produce an
answer. For their experiments, they used a pre-trained T5 [25] seq2seq model.

In FiD, fusion of the knowledge in the retrieved documents is performed right
before the decoder. Specifically, similar to RAG, they concatenate the input
query with each retrieved passage and they separately feed each concatenation
to the encoder (in parallel). However, after that, all the produced encoded
vectors are concatenated together (fusion) and are passed a single-vector
input to the decoder, which performs attention across all retrieved documents
(cross-attention).

Figure 2: Overview of the Fusion-in-Decoder (FiD)[7] technique. The input
question gets concatenated with each relevant passage and all concatenations
get encoded in parallel. The embeddings that are produced are concatenated
together (fusion) and are passed as input to the decoder.

###  III-D Atlas

Atlas [9] is essentially the next generation of RAG and FiD, but it
specializes in few-shot learning tasks. Atlas builds upon REALM [30] and
proposes jointly pre-training both the retriever and the generator model,
unlike RAG which uses pre-trained models and jointly trains them only during
the fine-tuning stage.

When performing a task, from question answering to generating Wikipedia
articles, Atlas starts by retrieving the top-k relevant documents from a large
corpus of text. Then, these documents are fed to the language model, along
with the query, which in turn generates the output. Both the retriever and the
language model are based on pre-trained transformer networks.

Atlas, similar to FiD, follows the retriever-generator architecture:

  * •

the retriever is based on the Contriever model [32] which entails a $BERT_{q}$
and a $BERT_{d}$ encoder and returns the K most relevant documents based on
their similarity with the query.

  * •

the generator uses a T5 seq2seq model [25] and applies the FiD technique that
processes each document separately in the encoder and concatenates the
embeddings before they enter the decoder.

Atlas, in contrast with RAG, trains both $BERT_{q}$ and $BERT_{d}$ (not only
$BERT_{q}$). Hence, the $BERT_{q}$ embeddings for each document in the
knowledge base need to be regularly updated so that they are in-sync with the
updated $BERT_{d}$ encoder. This is a computationally expensive task.

###  III-E RETRO

The creators of RETRO [8] managed to implement an augmented language model at
an unprecedented scale. This work’s breakthrough is that it managed to pre-
train and augment a relatively small Transformer model (25×fewer parameters
than GPT-3 [2]) with a database that is 2 trillion tokens large
($10^{3}$×larger than similar retrieval-augmented LLMs).

As we saw in previous work, such as RAG, REALM and Atlas, one main difficulty
of augmenting LLMs with external knowledge-bases is that training the
Retriever can be computationally expensive, because while the document encoder
becomes better, the embeddings for each passage in the database need to be
recomputed.

In this paper, they completely bypassed that challenge by using a frozen BERT
retriever [24] which contains a pre-trained document encoder. Hence, in RETRO
they calculate the document embeddings once, in the beginning, and do not
update them again. As a result, the main bottleneck that accessing the
external database entails is to retrieve the K most-relevant documents to the
input query, which they implemented using the SCaNN library [33]. This is a
task of sub-linear complexity, which means that we can query their 2 trillion
token database in 10ms.

One main difference of RETRO with previous work is that in RETRO they don’t
retrieve single documents (sentences), but chunks (a retrieved sentence along
with the following sentence). This enables the generator model to acquire more
context around the retrieved information and produce more accurate answers.

Here is an overview of how RETRO produces an answer to an input query, q:

  1. 1.

it splits the input query into chunks of 4 tokens

  2. 2.

For each chunk, cq of q, RETRO:

    1. (a)

calculates the embedding of the chunk

    2. (b)

finds the 2 nearest neighbors (most relevant documents) in its knowledge base

    3. (c)

encodes cq through the encoder

    4. (d)

encodes the 2 nearest neighbors through the encoder

    5. (e)

interleaves the encodings of the nearest neighbors with the query-chunk
embeddings to perform cross-attention. Neighbors of the first chunk only
affect the last token of the first chunk and the first tokens of the second
chunk.

Through this technique, RETRO manages to perform attention in complexity that
is linear to the number of retrieved passages.

###  III-F GRAFT-Net

GRAFT-Net [10] is a novel model designed for enhancing Question Answering (QA)
in scenarios where there is a structured, graph-like knowledge base
(triplestore) along with a substantial text corpus. GRAFT-Net leverages
advancements in graph representation learning to extract answers by creating
question-specific sub-graphs containing both text and knowledge-base entities
and relations.

Results in a range of benchmarks demonstrate that GRAFT-Net exhibits
competitive performance compared to state-of-the-art methods when tested on
either structured knowledge bases or text corpora in isolation.

Graft-Net consists of the following stages:

  1. 1.

the question sub-graph ($G_{q}$) retrieval stage: This is a characteristic of
early fusion, the process of combining information from the triplestore
knowledge-base and text early in the model, i.e., before a graph neural
network is used.

  2. 2.

the answer selection stage, where GRAFT-Net use a Graph Convolutional Network
(GCN) variant [34] [35] [27] to do binary classification (answer, not-answer)
on the nodes of $G_{q}$.

The question sub-graph $G_{q}$ essentially is a copy of the entire knowledge-
base graph, in which the nodes and edges that are irrelevant to a given
question, $q$, are pruned. In addition, the question sub-graph contains text
documents as well, but only the ones that are likely to contain the answer to
question $q$.

The retrieval of the question sub-graph, $G_{q}$ happens in two parallel
pipelines:

  1. 1.

Knowledge Base Retrieval

  2. 2.

Text Retrieval

During the knowledge base retrieval, a sub-graph of the triplestore knowledge
base is retrieved. Specifically, GRAFT-Net:

  1. 1.

retrieves a set of seed entities, $Sq$, that are relevant to the question $q$

  2. 2.

runs the Personalized PageRank (PPR) method [36] around these seeds to
identify other entities which might be an answer to the question. During PPR,
we assign weights to edges around the seed entities. Each edge weight is
essentially the cosine similarity between:

     * •

the question vector, v(q): average of all word vectors in the question

     * •

the relation vector, v(r): average of all word vectors in the relation
corresponding to that edge

  3. 3.

retains the top E entities $v_{1}$, …, $v_{E}$ by PPR score, along with any
edges between them, and adds them to the question sub-graph, $G_{q}$

During the text retrieval phase, GRAFT-Net retrieves documents (sentences)
relevant to the question , $q$, from the Wikipedia database. The text
retrieval phase entails the steps that are described below. GRAFT-Net:

  1. 1.

retrieves the top 5 most relevant Wikipedia articles (collection of
documents), by using a weighted bag-of-words model [37].

  2. 2.

populates a Lucene index [38] (facilitates data search in a large corpus of
text) with sentences from these articles, and retrieves the top ranking ones:
$d_{1}$, …, $d_{D}$.

The final question graph $G_{q}$ consists of:

  * •

$V_{q}$: all retrieved entities and documents

  * •

$E_{q}$: all relations between the retrieved entities and all entity links
between entities and documents

Because the vertices of the graphs can be either entities or documents, the
graph is considered: heterogeneous.

###  III-G PullNet [11]

PullNet builds upon the advancements made by GRAFT-Net and uses the text
corpus to supplement information extracted from the triplestore knowledge base
in order to answer multi-hop questions. The subjects and objects in the
triples contain links to relevant documents in the text corpus and PullNet
uses these links to produce more factually-based answers.

Like GRAFT-Net, PullNet has an initial phase where it retrieves a question
sub-graph $G_{q}$. However, PullNet learns how to construct the sub-graph,
rather than using an ad-hoc subgraph-building strategy. More specifically,
PullNet relies on a small set of retrieval operations, each of which expands a
graph node by retrieving new information from the knowledge base or the
corpus. It learns when and where to apply these “pull” operations with another
graph CNN classifier. The “pull” classifier is weakly supervised, using
question-answer pairs.

The end result is a learned iterative process for sub-graph construction,
which begins with a small sub-graph containing only the question text and the
entities which it contains, and gradually expands the sub-graph to contain
information from the knowledge base and corpus that are likely to be useful.
The process is especially effective for multi-hop questions.

##  IV Search-Engine Augmented Generation

Augmenting large language models with search engines represents the next step
in the evolution of AI-driven natural language processing. Search engines
empower models with a gateway to an expansive universe of knowledge that far
surpasses what external knowledge bases can access. By harnessing the prowess
of search engines, these models gain the ability to tap into the vast and
ever-expanding repository of information on the World Wide Web. This dynamic
access not only provides a wealth of information but also ensures that text
generation remains current and up-to-date with the latest developments, a feat
that external knowledge bases often struggle to achieve as they require
continuous updates.

However, it is crucial to acknowledge that this newfound access to the open
web through search engines carries potential risks. The information landscape
of the internet is diverse, encompassing both valuable knowledge and,
regrettably, harmful or malicious content. When integrated with augmented
large language models, there exists the possibility of inadvertently exposing
the model to inappropriate or unsafe content. This introduces concerns
regarding the reliability and safety of the generated responses, as the model
may unintentionally incorporate harmful information into its outputs.

As we will see in the following sections, the use of search engine-based
queries has the benefit that these queries are inherently designed to be
understood by humans, enhancing both the interpretability of the model’s
responses and its potential for continuous improvement through direct
annotation or feedback. However, to harness the immense potential of this
symbiotic fusion of AI-driven language models and the vast knowledge landscape
facilitated by search engines, it is imperative to develop robust safeguards
and mechanisms to mitigate the risks associated with accessing potentially
harmful or malicious content. This will ensure that the augmentation of
language models with search engines not only broadens their horizons but also
maintains the integrity and safety of their outputs, ushering in a new era of
responsible and informed natural language understanding and interaction.

###  IV-A Internet Augmented Dialogue Generation (IADG)

Previously described FAISS-based approaches, such as RAG (III-A) and FiD
(III-C), can take advantage of many existing methods developed for QA and
dialogue tasks, as we saw, but have several disadvantages. First, they may be
difficult to update to real-time web documents. On top of that, there may be a
limit to the number of documents that can be stored in local FAISS
deployments. Finally, such methods will not take advantage of the high quality
ranking that has been finely tuned in Internet Search engines over decades of
use. Thus, the authors of this paper by Facebook AI Research consider using
Internet search engines directly for knowledge retrieval.

IADG [13] consists of two main components:

  * •

a search query generator: an encoder-decoder Transformer that takes in the
dialogue context as input, and generates a search query. This is given to the
black-box search engine API, and N documents are returned.

  * •

a FiD-style generator: an encoder-decoder model that encodes each document
individually (along with the dialog context), concatenates the embeddings
before they enter the encoder, and finally generates the next response.

Each of these components can be trained separately, given supervised data for
both tasks. The query generator requires: (context, search query) pairs, and
the response generator requires: (context, response) pairs.

The search engine is a black box in this system (similar to LaMDA), and could
potentially be swapped out for any method. In IADG, they use the Bing Search
API [39] for their experiments to generate a list of URLs for each query.
Then, they use these URLs as keys to find their page content.

###  IV-B SeeKeR

SeeKeR [14] (Search-engine $\rightarrow$ Knowledge $\rightarrow$ Response)
introduces an innovative approach that employs a single language model to
tackle three distinct modular tasks consecutively: searching for information,
generating knowledge, and crafting a final response. In this research
endeavor, SeeKeR explores a modular framework that builds upon the foundations
of IADG [13] while amalgamating the most effective elements from various
existing solutions.

The SeeKeR model adheres to the foundational architecture of the standard
transformer [19], but it distinguishes itself by employing the same model in a
modular fashion, iteratively for multiple tasks. Within each module, the
encoder (or decoder) incorporates distinct special tokens to signal the
specific module being activated. The output generated by each module is
subsequently fed into the next one, along with the original context. SeeKeR
comprises a trio of specialized modules, each dedicated to unique
functionalities, namely:

  * •

Search module: generates a search query from the encoded input context.
Subsequently, this query is channeled into the Bing Web Search API [39],
initiating a retrieval process that yields the 5 most relevant documents as
outcomes.

  * •

Knowledge module: utilizes the encoded input context and a pool of retrieved
documents to generate meaningful responses. This response comprises one or
more pertinent phrases or sentences extracted directly from the retrieved
documents. Notably, the FiD [7] method is employed to encode both the context
and the documents.

  * •

Response module: operates on the encoded input context merged with the
knowledge response and crafts a coherent and contextually relevant
continuation to the input.

It is essential to highlight that the knowledge module essentially involves a
”copy” mechanism, as it does not entail the creation of new tokens; rather,
its complexity lies in the precise selection of the relevant knowledge to
replicate.

The authors of SeeKeR consider the GPT2 transformer [18] as a base model, and
fine-tune it to become a SeeKeR model. Therefore, they did not perform any
pre-training of their own in this case. For their experiments, they considered
medium, large and XL (345M, 762M and 1.5B parameters) models.

###  IV-C LaMDA

In this paper by Google, the authors of LaMDA [12] manage to augment a
language generation model with what they call a Toolset (TS), a black-box
external knowledge source. The Toolset consists of:

  1. 1.

a calculator

  2. 2.

a translator

  3. 3.

an information retrieval system (similar to a search engine)

The TS takes a single string as input and outputs a list of one or more
strings. Each tool in TS expects a string and returns a list of strings. For
example, the information retrieval system can receive “How old is Rafael
Nadal?” as input, and output [“Rafael Nadal / Age / 35”].

The information retrieval system is also capable of returning snippets of
content from the open web, with their corresponding URLs. The TS tries an
input string on all of its tools, and produces a final output list of strings
by concatenating the output lists from every tool in the following order:
calculator, translator, and information retrieval system. A tool will return
an empty list of results if it can’t parse the input (e.g., the calculator
cannot parse “How old is Rafael Nadal?”), and therefore does not contribute to
the final output list.

It is essential to note that only little information is given on how the
information retrieval system works, in the LaMDA paper, apart from the fact
that it entails a database, but also can provide web snippets along with their
URLs.

LaMDA entails two main sub-models that follow the decoder-only Transformer
architecture:

  1. 1.

LaMDA-Base: A regular generative model that is pre-trained on a large dataset.
LaMDA-Base is the first model to receive a query from the user. It then
generates a response that is checked and refined by LaMDA-Research.

  2. 2.

LaMDA-Research: A generative model that usually receives the output of LaMDA-
Base as input and is fine-tuned to choose the recipient of its output (the TS
or the user). In general, LaMDA-Research queries the TS in a loop, until it
has sufficient information to generate a final response to the user.

Year | ALM | Source of Knowledge | Retriever | Generator
---|---|---|---|---
2018 | GRAFT-Net | Graph + Text | Personalized PageRank + DrQA | GCNN
2019 | PullNet | Graph + Text | Pull | GCNN
2020 | RAG | Text | BERT | seq2seq
2020 | REALM | Text | BERT | seq2seq
2021 | FiD | Text | BERT | seq2seq
2021 | IADG | Internet | seq2seq + Search Engine | Encoder-Decoder Transformer
2022 | LaMDA | Internet | Black Box Information Retrieval System | Decoder-only Transformer
2022 | Atlas | Text | Contriever | seq2seq
2022 | RETRO | Text | BERT | Encoder-Decoder Transformer
2022 | SeeKeR | Text | Encoder-Decoder Transformer | Encoder-Decoder Transformer
TABLE I: Overview of mentioned augmented language model (ALM) architectures

##  V Limitations and Discussion

Augmented large language models grapple with a set of recurring challenges.
These issues encompass occasional inconsistencies, contradictions, factual
inaccuracies, potential repetition, and a limited depth of reasoning, among
others [40] [41].

Furthermore, concerns emerge regarding the generation of content imbued with
toxic language and bias, especially in specific contexts and topics [42] [43].
Another noteworthy concern is the influence of internet-sourced documents on
model outputs, potentially leading to the retrieval of undesirable content.
Many research experiments lean on externally developed search engines,
offering advantages in terms of optimization and reliability. However,
building one’s retrieval system, as is often the case in question-answering
(QA) and language modeling (LM) research, necessitates starting from scratch.

While search engines are adept at crawling and indexing the latest news and
documents, this process demands significant engineering effort and is vital
for various applications. Conversely, methods in the literature using their
retrieval setups often rely on fixed document databases, which become outdated
over time. Additionally, search engines are designed for human interaction,
using natural language queries with limited context. In contrast, machine-
generated queries, as exemplified by models like RAG [6], can potentially
encode more context or adopt vector-encoded queries, albeit at the cost of
human interpretability. A benefit of search engine-based queries is their
human readability, offering both interpretability and the potential for
improvement through direct annotation or feedback.

Language models employing augmentation address the challenge of hallucination
but do not guarantee factual grounding. Instances of conflicting retrievals
can lead to mixed responses. To enhance reliability, the introduction of trust
mechanisms, assigning different weights to retrievals, is a potential avenue.
Another concern is the generation of generic responses that may overlook the
incorporated knowledge.

In this survey, we have highlighted these common challenges and limitations
faced by augmented large language models, shedding light on the evolving
landscape of language generation and the pressing need for innovative
solutions.

##  VI Conclusion

In this literature survey, we have explored a multitude of works in which
Language Models (LMs) have been enriched with external knowledge, enabling
them to generate more contextually grounded and up-to-date responses.
Throughout these studies, LMs have demonstrated their capacity to enhance
context by incorporating relevant information, thereby fostering the
production of informative answers to various questions. This augmentation
often involves the integration of non-parametric modules, marking a departure
from the conventional language modeling paradigm and categorizing these models
as augmented language models.

However, it is essential to acknowledge certain limitations within this
paradigm shift. While LMs augmented with external knowledge exhibit reduced
hallucination, they do not offer an ironclad guarantee of factual grounding.
Instances arise where conflicting retrievals result in mixed answers,
underscoring the need for continued refinement in this domain. Moreover, the
limited exploration of the interplay between reasoning augmentation and
knowledge integration in current research highlights a promising avenue for
future endeavors.

As we reflect on the landscape of augmented language models, it becomes
evident that this field holds immense promise and excitement. It represents a
vital step towards ushering in the next generation of deep learning systems
that can engage in complex and meaningful human-machine interactions while
minimizing the parameter footprint. The journey towards fully realizing the
potential of augmented LMs is ongoing, with opportunities for further
innovation and investigation awaiting those who seek to shape the future of
this dynamic field.

## References

  * [1] M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. de Oliveira Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, A. Ray, R. Puri, G. Krueger, M. Petrov, H. Khlaaf, G. Sastry, P. Mishkin, B. Chan, S. Gray, N. Ryder, M. Pavlov, A. Power, L. Kaiser, M. Bavarian, C. Winter, P. Tillet, F. P. Such, D. Cummings, M. Plappert, F. Chantzis, E. Barnes, A. Herbert-Voss, W. H. Guss, A. Nichol, A. Paino, N. Tezak, J. Tang, I. Babuschkin, S. Balaji, S. Jain, W. Saunders, C. Hesse, A. N. Carr, J. Leike, J. Achiam, V. Misra, E. Morikawa, A. Radford, M. Knight, M. Brundage, M. Murati, K. Mayer, P. Welinder, B. McGrew, D. Amodei, S. McCandlish, I. Sutskever, and W. Zaremba, “Evaluating large language models trained on code,” 2021.
  * [2] T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei, “Language models are few-shot learners,” 2020.
  * [3] S. Welleck, I. Kulikov, S. Roller, E. Dinan, K. Cho, and J. Weston, “Neural text generation with unlikelihood training,” 2019.
  * [4] J. Hoffmann, S. Borgeaud, A. Mensch, E. Buchatskaya, T. Cai, E. Rutherford, D. de Las Casas, L. A. Hendricks, J. Welbl, A. Clark, T. Hennigan, E. Noland, K. Millican, G. van den Driessche, B. Damoc, A. Guy, S. Osindero, K. Simonyan, E. Elsen, J. W. Rae, O. Vinyals, and L. Sifre, “Training compute-optimal large language models,” 2022.
  * [5] T. Scialom, T. Chakrabarty, and S. Muresan, “Fine-tuned language models are continual learners,” 2022.
  * [6] P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W.-t. Yih, T. Rocktäschel _et al._ , “Retrieval-augmented generation for knowledge-intensive nlp tasks,” _Advances in Neural Information Processing Systems_ , vol. 33, pp. 9459–9474, 2020.
  * [7] G. Izacard and E. Grave, “Leveraging passage retrieval with generative models for open domain question answering,” 2021.
  * [8] S. Borgeaud, A. Mensch, J. Hoffmann, T. Cai, E. Rutherford, K. Millican, G. van den Driessche, J.-B. Lespiau, B. Damoc, A. Clark, D. de Las Casas, A. Guy, J. Menick, R. Ring, T. Hennigan, S. Huang, L. Maggiore, C. Jones, A. Cassirer, A. Brock, M. Paganini, G. Irving, O. Vinyals, S. Osindero, K. Simonyan, J. W. Rae, E. Elsen, and L. Sifre, “Improving language models by retrieving from trillions of tokens,” 2022.
  * [9] G. Izacard, P. Lewis, M. Lomeli, L. Hosseini, F. Petroni, T. Schick, J. Dwivedi-Yu, A. Joulin, S. Riedel, and E. Grave, “Atlas: Few-shot learning with retrieval augmented language models,” 2022.
  * [10] H. Sun, B. Dhingra, M. Zaheer, K. Mazaitis, R. Salakhutdinov, and W. Cohen, “Open domain question answering using early fusion of knowledge bases and text,” in _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_. Brussels, Belgium: Association for Computational Linguistics, Oct.-Nov. 2018, pp. 4231–4242. [Online]. Available: https://aclanthology.org/D18-1455
  * [11] H. Sun, T. Bedrax-Weiss, and W. W. Cohen, “Pullnet: Open domain question answering with iterative retrieval on knowledge bases and text,” 2019.
  * [12] R. Thoppilan, D. D. Freitas, J. Hall, N. Shazeer, A. Kulshreshtha, H.-T. Cheng, A. Jin, T. Bos, L. Baker, Y. Du, Y. Li, H. Lee, H. S. Zheng, A. Ghafouri, M. Menegali, Y. Huang, M. Krikun, D. Lepikhin, J. Qin, D. Chen, Y. Xu, Z. Chen, A. Roberts, M. Bosma, V. Zhao, Y. Zhou, C.-C. Chang, I. Krivokon, W. Rusch, M. Pickett, P. Srinivasan, L. Man, K. Meier-Hellstern, M. R. Morris, T. Doshi, R. D. Santos, T. Duke, J. Soraker, B. Zevenbergen, V. Prabhakaran, M. Diaz, B. Hutchinson, K. Olson, A. Molina, E. Hoffman-John, J. Lee, L. Aroyo, R. Rajakumar, A. Butryna, M. Lamm, V. Kuzmina, J. Fenton, A. Cohen, R. Bernstein, R. Kurzweil, B. Aguera-Arcas, C. Cui, M. Croak, E. Chi, and Q. Le, “Lamda: Language models for dialog applications,” 2022.
  * [13] M. Komeili, K. Shuster, and J. Weston, “Internet-augmented dialogue generation,” 2021.
  * [14] K. Shuster, M. Komeili, L. Adolphs, S. Roller, A. Szlam, and J. Weston, “Language models that seek for knowledge: Modular search and generation for dialogue and prompt completion,” 2022.
  * [15] Y. Bengio, R. Ducharme, and P. Vincent, “A neural probabilistic language model,” in _Advances in Neural Information Processing Systems_ , T. Leen, T. Dietterich, and V. Tresp, Eds., vol. 13. MIT Press, 2000. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2000/file/728f206c2a01bf572b5940d7d9a8fa4c-Paper.pdf
  * [16] D. P. Kingma and M. Welling, “An introduction to variational autoencoders,” _Foundations and Trends® in Machine Learning_ , vol. 12, no. 4, pp. 307–392, 2019. [Online]. Available: https://doi.org/10.1561%2F2200000056
  * [17] I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial networks,” 2014.
  * [18] A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever _et al._ , “Language models are unsupervised multitask learners,” _OpenAI blog_ , vol. 1, no. 8, p. 9, 2019.
  * [19] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” 2023.
  * [20] I. Sutskever, O. Vinyals, and Q. V. Le, “Sequence to sequence learning with neural networks,” 2014.
  * [21] D. E. Rumelhart, G. E. Hinton, R. J. Williams _et al._ , “Learning internal representations by error propagation,” 1985.
  * [22] M. I. Jordan, “Serial order: A parallel distributed processing approach,” in _Advances in psychology_. Elsevier, 1997, vol. 121, pp. 471–495.
  * [23] Y. LeCun, B. Boser, J. Denker, D. Henderson, R. Howard, W. Hubbard, and L. Jackel, “Handwritten digit recognition with a back-propagation network,” _Advances in neural information processing systems_ , vol. 2, 1989.
  * [24] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” 2019.
  * [25] C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu, “Exploring the limits of transfer learning with a unified text-to-text transformer,” 2020.
  * [26] J. Johnson, M. Douze, and H. Jégou, “Billion-scale similarity search with gpus,” _IEEE Transactions on Big Data_ , vol. 7, no. 3, pp. 535–547, 2021.
  * [27] M. Schlichtkrull, T. N. Kipf, P. Bloem, R. Van Den Berg, I. Titov, and M. Welling, “Modeling relational data with graph convolutional networks,” in _The Semantic Web: 15th International Conference, ESWC 2018, Heraklion, Crete, Greece, June 3–7, 2018, Proceedings 15_. Springer, 2018, pp. 593–607.
  * [28] M. Lewis, Y. Liu, N. Goyal, M. Ghazvininejad, A. Mohamed, O. Levy, V. Stoyanov, and L. Zettlemoyer, “Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension,” 2019.
  * [29] V. Karpukhin, B. Oğuz, S. Min, P. Lewis, L. Wu, S. Edunov, D. Chen, and W. tau Yih, “Dense passage retrieval for open-domain question answering,” 2020.
  * [30] K. Guu, K. Lee, Z. Tung, P. Pasupat, and M.-W. Chang, “Realm: Retrieval-augmented language model pre-training,” 2020.
  * [31] K. Lee, M.-W. Chang, and K. Toutanova, “Latent retrieval for weakly supervised open domain question answering,” 2019.
  * [32] G. Izacard, M. Caron, L. Hosseini, S. Riedel, P. Bojanowski, A. Joulin, and E. Grave, “Unsupervised dense information retrieval with contrastive learning,” 2022.
  * [33] R. Guo, P. Sun, E. Lindgren, Q. Geng, D. Simcha, F. Chern, and S. Kumar, “Accelerating large-scale inference with anisotropic vector quantization,” 2020.
  * [34] T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” 2017.
  * [35] Y. Li, D. Tarlow, M. Brockschmidt, and R. Zemel, “Gated graph sequence neural networks,” 2017.
  * [36] T. H. Haveliwala, “Topic-sensitive pagerank,” in _Proceedings of the 11th International Conference on World Wide Web_ , ser. WWW ’02. New York, NY, USA: Association for Computing Machinery, 2002, p. 517–526. [Online]. Available: https://doi.org/10.1145/511446.511513
  * [37] D. Chen, A. Fisch, J. Weston, and A. Bordes, “Reading wikipedia to answer open-domain questions,” 2017.
  * [38] A. S. Foundation. (2011) Apache lucene - scoring. Letzter Zugriff: 20. Oktober 2011. [Online]. Available: http://lucene.apache.org/java/3_4_0/scoring.html
  * [39] Microsoft, “Bing web search api,” 2023. [Online]. Available: https://www.microsoft.com/en-us/bing/apis/bing-web-search-api
  * [40] S. Roller, E. Dinan, N. Goyal, D. Ju, M. Williamson, Y. Liu, J. Xu, M. Ott, K. Shuster, E. M. Smith, Y.-L. Boureau, and J. Weston, “Recipes for building an open-domain chatbot,” 2020.
  * [41] L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. L. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, J. Schulman, J. Hilton, F. Kelton, L. Miller, M. Simens, A. Askell, P. Welinder, P. Christiano, J. Leike, and R. Lowe, “Training language models to follow instructions with human feedback,” 2022.
  * [42] J. Xu, D. Ju, M. Li, Y.-L. Boureau, J. Weston, and E. Dinan, “Recipes for safety in open-domain chatbots,” 2021.
  * [43] E. Dinan, A. Fan, A. Williams, J. Urbanek, D. Kiela, and J. Weston, “Queens are powerful too: Mitigating gender bias in dialogue generation,” in _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_. Online: Association for Computational Linguistics, Nov. 2020, pp. 8173–8188. [Online]. Available: https://aclanthology.org/2020.emnlp-main.656


# SIAN: Style-Guided Instance-Adaptive Normalization for Multi-Organ
Histopathology Image Synthesis

###### Abstract

Existing deep neural networks for histopathology image synthesis cannot
generate image styles that align with different organs, and cannot produce
accurate boundaries of clustered nuclei. To address these issues, we propose a
style-guided instance-adaptive normalization (SIAN) approach to synthesize
realistic color distributions and textures for histopathology images from
different organs. SIAN contains four phases, semantization, stylization,
instantiation, and modulation. The first two phases synthesize image semantics
and styles by using semantic maps and learned image style vectors. The
instantiation module integrates geometrical and topological information and
generates accurate nuclei boundaries. We validate the proposed approach on a
multiple-organ dataset, Extensive experimental results demonstrate that the
proposed method generates more realistic histopathology images than four
state-of-the-art approaches for five organs. By incorporating synthetic images
from the proposed approach to model training, an instance segmentation network
can achieve state-of-the-art performance.

Index Terms—  Histopathology image synthesis, style manipulation, nuclei
annotation

##  1 Introduction

Histopathology image analysis has achieved great success in automatic tissue
segmentation [1, 2] and cancer grading [3]. Existing deep learning-based
methods require large fully-annotated datasets during the training stage, but
current annotated datasets are relatively small. For example, only tens of
image patches were used in [1, 4, 5, 6]. With large annotated datasets, we
could train more accurate and reliable models. However, it is expensive to
annotate large datasets for histopathology images, because each image may
contain more than tens of thousands of nuclei.

(a) Breast

(b) Prostate

(c) Liver

(d) Bladder

(e) Kidley

(f) Colon

Fig. 1: Examples of image synthesis for multiple organs. The top row shows
real histopathology images from six organs. The second and third rows are
synthesized images generated using the proposed approach (SIAN).

To overcome the challenge, image synthesis is adopted. Recent works have
demonstrated that high-quality synthetic images could improve the overall
performance in histopathology image analysis [7, 8, 9]. However, these methods
generated images only for single cancer or cancers with shared similarity,
e.g., colorectal cancer [7, 8], lymph node [9]; thus their models cannot
generate different image styles for different cancer types. In practice,
H&E-stained images for cancers from different organs could have large color
and texture variances both in foreground nuclei and background stroma (first
row of Fig. 1). Therefore, it is important to have the network to generate
histopathology images well in various stain distribution across multiple
organs.

Recently, neural style transfer (NSF) methods have been widely exploited in
many natural image synthesis tasks for manipulating image styles [10, 11, 12],
they aim to learn the style from a reference image and apply it to the target
image. SPADE [11] extended the AdaIN norm [10] into the spatially-adaptive
manner for obtaining semantic alignments and used the encoded style vector at
the beginning of a network, which enabled simultaneously style manipulation
and semantic image synthesis. However, most existing histopathology image
synthesis methods applied semantic layouts as the network input to learn
object-level image appearance [11, 7, 8]. In histopathology image analysis, a
large amount of clustered and overlapped objects may have the same semantic
class label, which makes it difficult to generate accurate boundaries among
clustered objects.

Fig. 2: Architecture of the proposed method. The encoder learns style vectors
from a referred image; and the SIAN blocks integrate image style
($\mathbf{S}$), semantic map ($\mathbf{M}$), directional map ($\mathbf{P}$),
and distance map ($\mathbf{Q}$) into a generator network.

To alleviate the above issues, we proposed a style-guided instance-adaptive
normalization (SIAN) to combine image style vector with instance layout for
modulating the GAN generator. The learned transformation can effectively
propagate the network to learn style factors for synthesizing histopathology
images across various color distributions; the captured instance features can
preserve geometrical and topological information for generating accurate
densely-clustered nuclei. SIAN can generate style images from a specific organ
and synthesize histopathology images with a similar style.

##  2 Proposed Method

###  2.1 Architecture and learning objectives

Fig. 2 shows the overall architecture of the proposed generator. The proposed
generator has seven SIAN residual blocks (SIAN ResBlk), and each SIAN ResBlk
is followed by a up-sampling layer. Each SIAN ResBlk contains two consecutive
SIAN blocks, and each is followed by ReLU and convolutional layers. The skip
connection has a SIAN block, a ReLU, and a convolutional layer. All input maps
are down-sampled to the same height and width with the corresponding feature
maps in the generator. We follow the same encoder and discriminator
architectures described in [11]. The overall loss function contains five loss
components, and is defined by [11]

$\mathcal{L}_{SIAN}=\mathcal{L}_{GAN}+\lambda_{1}\mathcal{L}_{F}+\lambda_{2}\mathcal{L}_{P}+\\\
\lambda_{3}\mathcal{L}_{KLD}+\lambda_{4}\mathcal{L}_{reg}$ (1)

where $\mathcal{L}_{GAN}$ is the hinge-based conditional adversarial loss
[13], $\mathcal{L}_{F}$ is the feature matching loss in the multi-scale
discriminator [14], $\mathcal{L}_{P}$ is the perceptual loss [15] for
minimizing the features between real and synthetic images, and
$\mathcal{L}_{KLD}$ is the KL divergence loss [16] for the encoder to
constrain the style vector to the standard Gaussian distribution.
$\lambda_{1}$, $\lambda_{2}$, $\lambda_{3}$, and $\lambda_{4}$ controls the
contributions of different loss terms.

###  2.2 Style-guided instance-adaptive normalization

We propose a new conditional normalization block, namely, the Style-guided
Instance-Adaptive Normalization (SIAN) to learn instance-level features and
integrate image styles for cancers from different organs. Fig. 3 shows the
details of the SIAN block. The block has four phases: semantization,
stylization, instantiation, and modulation. The block takes four inputs
besides image feature maps, i.e., semantic mask $\mathbf{M}$, style vector
$\mathbf{S}$, direction mask $\mathbf{P}$, and distance mask $\mathbf{Q}$. The
semantization phase embeds image semantics from the input mask; the
stylization creates a style matrix from a referred image and integrates image
semantics and style. The instantiation phase uses direction and distance maps
to distinguish individual nuclei. The modulation phase learns the scale and
bias and integrates them into the network.

Fig. 3: SIAN normalization. SIAN block takes four inputs: semantic masks
$\mathbf{M}$, style vector $\mathbf{S}$, directional mask $\mathbf{P}$, and
distance mask $\mathbf{Q}$ for combining different features at multiple phases
in the block. $\otimes$ denotes element-wise multiplication, and $\oplus$ is
element-wise addition.

Let $\mathbf{h}$ denote the input activation of the current layer of the
proposed neural network with a batch size of $N$. Let $H$, $W$ and $C$ denote
the height, width, and channels of an activation map in $i$th layer. The final
modulated activation value ($n\in N$, $c\in C$, $y\in H$, $x\in W$) is defined
as

$\gamma_{c,y,x}(\mathbf{M},\mathbf{S},\mathbf{P},\mathbf{Q})\frac{h_{n,c,y,x}-\mu_{c}}{\sigma_{c}}+\beta_{c,y,x}(\mathbf{M},\mathbf{S},\mathbf{P},\mathbf{Q})\vspace*{-1mm}$
(2)

where $h_{n,c,y,x}$ is the activation output before normalization; the
modulation parameter $\gamma_{c,y,x}$ and $\beta_{c,y,x}$ are the element-wise
summation of modulation parameters of two branches, i.e.,
$\gamma_{c,y,x}^{i}+\gamma_{c,y,x}^{j}$ and
$\beta_{c,y,x}^{i}+\beta_{c,y,x}^{j}$. $\mu_{c}$ and $\sigma_{c}$ are the mean
and standard deviation of the activation of the channel $c$, respectively. In
the SIAN block, the semantic layout first passes to two convolutional layers,
which split the semantic information into two separate branches to learn the
directional features and distance features separately. The two branches have
the same architecture. In each branch, the convolutional kernel first
multiplies with the reshaped style vector, which combines style factors in the
block. After that, the instance layouts (direction or distance) are fed
through a 1 $\times$ 1 convolutional layer and multiplied with the previous
convolutional layer. The next convolutional layer learns the compensation of
semantic, style, and instance features and then split into two convolutional
layers to learn the modulation parameters ($\gamma$ and $\beta$) spatially.
Finally, those modulation parameters and the output of batch normalization are
integrated for accurate histopathology image synthesis. All convolutional
layers in SIAN use the 3$\times$3 kernel size with 128 filters.

Instance masks are applied to generate the semantic mask, and nuclei
directional and distance maps. The semantic map is used to separate nuclei and
stroma, and the directional and distance maps are useful to demonstrate the
boundaries and centroids between two or more touching nuclei. We employed the
2-bin direction mask [17] and Medial Axis (MA) distance mask [18] as the
instance descriptors. Direction map provides important centroid and
directional information of nuclei. MA distance mask shows the distance between
the nucleus boundary to its skeleton while providing nuclei topological and
geometrical features.

##  3 Experimental Results

###  3.1 Dataset, metrics, and setting

Dataset. The experiments are conducted on the multi-organ nuclei segmentation
dataset (MoNuSeg) [4] which has 44 H&E stained histopathology image patches.
Both the training and test sets contain images from six organs including
breast, liver, kidney, prostate, bladder, and colon; while the training set
includes stomach as the seventh organ, and the testing set has brain images.

Evaluation metrics. We employed five metrics to evaluate the method
performance for image synthesis, e.g., FID [19], SSIM [20], DQ [21], SQ [21],
and PQ [21]. We used two metrics FID and SSIM to measure the distribution
distance and structural similarity between real images and synthetic images,
respectively; and used DQ, SQ, and PQ are utilized to assess the nuclei
segmentation performance. Specifically, we run a pre-trained segmentation
model (SegNet [22]) which is trained on real images, and then test and
evaluate using the synthetic images. In addition, we show the visual
comparison of our synthetic images compared to other methods.

Implementation details. The input image size of our approach is
256$\times$256\. We used random flip, rotation and median blur for data
augmentation. We use the ADAM optimizer with the total training epochs of 50
and batch size of 8 to train the network. The experiments are conducted on a
NVIDIA RTX 8000 GPU.

During inference, for the style encoder, we take an arbitrary histopathology
image as input and output the encoded style vectors. Then, the encoded style
vectors together with an arbitrary instance mask are input into the trained
generator network to produce histopathology images.

###  3.2 Image quality assessment

The proposed method is compared with four state-of-the-art image synthesis
models: pix2pix GAN [23], Sharp-GAN [24], pix2pixHD [25], and SPADE [11] using
FID, SSIM, DQ, SQ, and PQ metrics. The quantitative results of different
approaches on the MoNuSeg test set are shown in Table 1. The proposed method
outperforms the state-of-the-art methods both in image reconstruction quality
using SSIM and FID, and segmentation quality using PQ, SQ and DQ. In addition,
we integrated instantiation phase (INST) only, and the SIAN block with the
style vectors (STYLE), all the evaluation metrics are improved from the
baseline SPADE. Overall, we can conclude that our SIAN achieved the best
quantitative performance among other methods.

Fig. 4: Visual comparison of histopathology image synthesis for the MoNuSeg test set. (a) Image patches, (b) nuclei masks, (c-e) results of Sharp-GAN, SPADE, and ours, respectively Methods |  FID$\downarrow$ |  SSIM$\uparrow$ |  DQ$\uparrow$ |  SQ$\uparrow$ |  PQ$\uparrow$
---|---|---|---|---|---
pix2pix [23] | 170.1 | 0.467 | 0.687 | 0.717 | 0.493
Sharp-GAN [24] | 155.2 | 0.483 | 0.721 | 0.745 | 0.538
pix2pixHD [25] | 186.3 | 0.479 | 0.750 | 0.753 | 0.566
SPADE [11] | 134.6 | 0.488 | 0.705 | 0.738 | 0.552
INST | 125.4 | 0.491 | 0.748 | 0.768 | 0.575
STYLE | 116.5 | 0.506 | 0.743 | 0.769 | 0.571
SIAN | 115.7 | 0.515 | 0.757 | 0.761 | 0.586
Table 1: Overall performance on MoNuSeg datasets with reconstruction metrics
and segmentation metrics.

Fig. 4 compares the proposed method, Sharp-GAN [24], and SPADE [11] using
three examples. We noted that Sharp-GAN cannot recover the texture and color
distributions of nuclei and stroma in real images, especially in the first and
second row. I.e., the synthetic nuclei have different appearances from real
nuclei, and their background stroma lacks meaningful texture and color. SPADE
achieved better performance compared to Sharp-GAN, but the generated images
were not realistic. Our SIAN generates more realistic images than SPADE and
Sharp-GAN. SPADE used the semantic layout as input, while our method used the
instance layouts. As shown in Figure 5, SPADE tends to generate blur and
incorrect nuclei in the clustered region. Our approach produces more accurate
boundaries for clustered nuclei.

Fig. 5: Synthesis for clustered nuclei. First row: instance masks (different colors represent different nuclei). Second row: results of SPADE [11]. Third row: results of SIAN. Meth. | Bre. | Kid. | Pro. | Bla. | Col. | Lun. | Bra.
---|---|---|---|---|---|---|---
[23] | 211.0 | 197.3 | 237.5 | 246.9 | 239.3 | 187.5 | 304.4
[24] | 198.0 | 190.4 | 235.3 | 211.4 | 208.6 | 195.4 | 260.3
[25] | 260.0 | 222.1 | 260.1 | 278.6 | 221.8 | 252.8 | 278.6
[11] | 212.4 | 187.9 | 200.4 | 212.5 | 222.3 | 207.2 | 245.5
SIAN | 200.4 | 173.9 | 190.2 | 196.9 | 204.1 | 194.6 | 239.6
Table 2: Performance comparison of image synthesis for multiple organs using
the FID score.

###  3.3 Multi-organ image synthesis

To evaluate synthetic images across multiple organs, we compare the generation
performance of four methods for each organ using FID scores. The results are
shown in Table 2. The proposed method outperforms other state-of-the-art
methods in kidney, prostate, bladder, colon, and brain images, and achieved
the second-best results for synthesizing breast and lung images. Fig. 1 shows
the results of SIAN across multiple organs, the color and texture distribution
of foreground nuclei and background stroma are close to the reals.

###  3.4 Nuclei segmentation using synthetic images

In this experiment, we evaluate the effectiveness of synthetic augmentation
for training segmentation networks. We train SegNet [22] with different input
configurations (as shown in Table 3). In experiments, nucleus-like polygons
are generated as the synthetic nuclei instance masks [26], in total 5,000
synthetic instance masks are generated and applied to produce corresponding
semantic, directional, and distance masks. Then, the pre-trained SIAN is used
to apply seven different style vectors encoded from seven different organs
(around 700 synthetic images per organ) and generate realistic histopathology
images. Finally, we test and evaluate the segmentation performance with the
MoNuSeg test set using DQ, SQ, PQ metrics. We compared the proposed approach
to other methods on synthetic augmentation. Synthetic images generated from
other methods follow their design. As shown in Table 3, with synthetic
training images from pix2pixHD, Sharp-GAN, and SPADE, the performance of
nuclei segmentation could be significantly improved. The proposed SIAN help
generate the best segmentation performance.

Training Set | DQ | SQ | PQ
---|---|---|---
MoNuSeg training set | 0.704 | 0.737 | 0.521
MoNuSeg training set∗ | 0.732 | 0.739 | 0.538
+pix2pixHD | 0.742 | 0.737 | 0.544
+Sharp-GAN | 0.740 | 0.739 | 0.547
+SPADE | 0.743 | 0.738 | 0.549
+SIAN | 0.748 | 0.742 | 0.555
Table 3: Performance of SegNet using different training sets. ∗ denotes the
training set augmented using traditional augmentation techniques, e.g., flip,
rotate, blur. ’+method’ denotes applying synthetic augmentation∗ with 5,000
synthetic images generated from ’method’ to the training set.

##  4 Conclusion

In this paper, we propose the style-guided instance-adaptive normalization
(SIAN) for multi-organ histopathology image synthesis, which integrates
instance layouts and encodes style vectors into a generative network. SIAN
synthesizes histopathology images with styles that align with the image styles
of multiple organs. SIAN utilizes the directional and distance masks from the
nuclei instance maps and generates clear boundaries for densely-clustered
nuclei. With the integration of the stylization phase, SIAN allows style
editing for synthesizing images of multiple organs. In addition, SIAN
demonstrates its effectiveness in augmenting the training set and improving
the overall performance of a deep learning model for nuclei segmentation.

## References

  * [1] Simon Graham, Quoc Dang Vu, Shan E Ahmed Raza, et al.,  “Hover-net: Simultaneous segmentation and classification of nuclei in multi-tissue histology images,”  Medical Image Analysis, vol. 58, pp. 101563, 2019.
  * [2] Haotian Wang, Min Xian, and Aleksandar Vakanski,  “Bending loss regularized network for nuclei segmentation in histopathology images,”  in 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI). IEEE, 2020, pp. 1–5.
  * [3] Muhammad Shaban, Ruqayya Awan, Muhammad Moazam Fraz, et al.,  “Context-aware convolutional neural network for grading of colorectal cancer histology images,”  IEEE transactions on medical imaging, vol. 39, no. 7, pp. 2395–2405, 2020.
  * [4] Neeraj Kumar, Ruchika Verma, Deepak Anand, et al.,  “A multi-organ nucleus segmentation challenge,”  IEEE Transactions on Medical Imaging, vol. 39, no. 5, pp. 1380–1391, 2020.
  * [5] Quoc Dang Vu, Simon Graham, Tahsin Kurc, Minh Nguyen Nhat To, et al.,  “Methods for segmentation and classification of digital microscopy tissue images,”  Frontiers in bioengineering and biotechnology, p. 53, 2019.
  * [6] Peter Naylor, Marick Laé, Fabien Reyal, and Thomas Walter,  “Segmentation of nuclei in histopathology images by deep regression of the distance map,”  IEEE transactions on medical imaging, vol. 38, no. 2, pp. 448–459, 2018.
  * [7] Srijay Deshpande, Fayyaz Minhas, Simon Graham, and Nasir Rajpoot,  “Safron: Stitching across the frontier network for generating colorectal cancer histology images,”  Medical Image Analysis, vol. 77, pp. 102337, 2022.
  * [8] Jerry Wei, Arief Suriawinata, Louis Vaickus, et al.,  “Generative image translation for data augmentation in colorectal histopathology images,”  Proceedings of machine learning research, vol. 116, pp. 10, 2019\.
  * [9] Yuan Xue, Jiarong Ye, Qianying Zhou, et al.,  “Selective synthetic augmentation with histogan for improved histopathology image classification,”  Medical image analysis, vol. 67, pp. 101816, 2021.
  * [10] Xun Huang and Serge Belongie,  “Arbitrary style transfer in real-time with adaptive instance normalization,”  in Proceedings of the IEEE international conference on computer vision, 2017, pp. 1501–1510.
  * [11] Taesung Park, Ming-Yu Liu, Ting-Chun Wang, et al.,  “Semantic image synthesis with spatially-adaptive normalization,”  in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 2337–2346.
  * [12] Tero Karras, Samuli Laine, and Timo Aila,  “A style-based generator architecture for generative adversarial networks,”  in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 4401–4410.
  * [13] Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida,  “Spectral normalization for generative adversarial networks,”  in International Conference on Learning Representations, 2018.
  * [14] Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Andrew Tao, et al.,  “High-resolution image synthesis and semantic manipulation with conditional gans,”  in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 8798–8807.
  * [15] Justin Johnson, Alexandre Alahi, and Li Fei-Fei,  “Perceptual losses for real-time style transfer and super-resolution,”  in European conference on computer vision. Springer, 2016, pp. 694–711.
  * [16] Diederik P. Kingma and Max Welling,  “Auto-encoding variational bayes,”  in 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings, Yoshua Bengio and Yann LeCun, Eds., 2014.
  * [17] Liang-Chieh Chen, Alexander Hermans, George Papandreou, et al.,  “Masklab: Instance segmentation by refining object detection with semantic and direction features,”  in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 4013–4022.
  * [18] Haotian Wang, Min Xian, and Aleksandar Vakanski,  “Ta-net: Topology-aware network for gland segmentation,”  in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2022, pp. 1556–1564.
  * [19] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, et al.,  “Gans trained by a two time-scale update rule converge to a local nash equilibrium,”  Advances in neural information processing systems, vol. 30, 2017\.
  * [20] Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli,  “Image quality assessment: from error visibility to structural similarity,”  IEEE transactions on image processing, vol. 13, no. 4, pp. 600–612, 2004.
  * [21] Alexander Kirillov, Kaiming He, Ross Girshick, et al.,  “Panoptic segmentation,”  in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 9404–9413.
  * [22] Vijay Badrinarayanan, Alex Kendall, and Roberto Cipolla,  “Segnet: A deep convolutional encoder-decoder architecture for image segmentation,”  IEEE transactions on pattern analysis and machine intelligence, vol. 39, no. 12, pp. 2481–2495, 2017.
  * [23] Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros,  “Image-to-image translation with conditional adversarial networks,”  in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 1125–1134.
  * [24] Sujata Butte, Haotian Wang, Min Xian, and Aleksandar Vakanski,  “Sharp-gan: Sharpness loss regularized gan for histopathology image synthesis,”  in 2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI). IEEE, 2022, pp. 1–5.
  * [25] Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Andrew Tao, et al.,  “High-resolution image synthesis and semantic manipulation with conditional gans,”  in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 8798–8807.
  * [26] Le Hou, Ayush Agarwal, Dimitris Samaras, et al.,  “Robust histopathology image analysis: To label or to synthesize?,”  in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 8533–8542.