# Neural Program Repair with Execution-based Backpropagation

He Ye<EMAIL_ADDRESS>KTH Royal Institute of TechnologySweden ,  Matias Martinez
<EMAIL_ADDRESS>Université Polytechnique Hauts-de-France, France & KTH
Royal Institute of Technology, Sweden  and  Martin Monperrus
<EMAIL_ADDRESS>KTH Royal Institute of TechnologySweden

###### Abstract.

Neural machine translation (NMT) architectures have achieved promising results
for automatic program repair. Yet, they have the limitation of generating low-
quality patches (e.g., not compilable patches). This is because the existing
works only optimize a purely syntactic loss function based on characters and
tokens without incorporating program-specific information during neural
network weight optimization. In this paper, we propose a novel program repair
model called RewardRepair. The core novelty of RewardRepair is to improve NMT-
based program repair with a loss function based on program compilation and
test execution information, rewarding the network to produce patches that
compile and that do not overfit. We conduct several experiments to evaluate
RewardRepair showing that it is feasible and effective to use compilation and
test execution results to optimize the underlying neural repair model.
RewardRepair correctly repairs 207 bugs over four benchmarks. we report on
repair success for 121 bugs that are fixed for the first time in the
literature. Also, RewardRepair produces up to 45.3% of compilable patches, an
improvement over the 39% by the state-of-the-art.

##  1\. Introduction

Automatic program repair (APR) aims to reduce manual work related to bug
localization and bug fixing (Monperrus, 2017; Gazzola et al., 2017). With
recent advances in deep learning, research has been proposed to use neural
networks for program repair, a subarea of a trend on using machine learning on
code (Chen and Monperrus, 2019). This line of work, put here under the
umbrella term “neural program repair”, mostly uses neural machine translation
(NMT) approaches (Jiang et al., 2021; Chen et al., 2019; Lutellier et al.,
2020; Tufano et al., 2019a; Chakraborty et al., 2020; Tufano et al., 2019b;
Gupta et al., 2017).

Program repair systems based on neural machine translation treat the repair
task as a translation from buggy code to correct code, both represented as a
sequence of tokens (Chen et al., 2019). Given sufficient training data, NMT-
based repair has achieved promising performance (Jiang et al., 2021; Chen et
al., 2019; Lutellier et al., 2020; Tufano et al., 2019a; Chakraborty et al.,
2020; Tufano et al., 2019b; Gupta et al., 2017). All the prior works on
program repair based on neural machine translation use the static loss
function: cross-entropy loss based on token similarity.

Despite early successes, NMT-based program repair approaches suffer from two
major drawbacks. First, they often generate patches that do not compile (Chen
et al., 2019). The reason is that a lower cross-entropy value does not
necessarily lead to a compilable patch. Second, the loss function under
optimization forces the neural network to learn to produce absolutely
identical patches, thus missing the opportunity to explore semantically
equivalent. However, Rabin et al. (Rabin et al., 2021) made the case that
generalizability in machine learning on code relates to generalization over
equivalent programs. Both problems (uncompilability and being stuck with
syntactic identity) share the same root: there is a discrepancy between the
core training objective, learning to generate compilable and correct patches,
and the loss function that is being optimized. The cross-entropy loss used in
related work requires a strict pairwise matching between the generated patch
and the human-written ground truth patch, and not more (Zhang et al., 2019).
Nothing in cross-entropy loss encourages the neural network to produce
compilable or syntactically different but semantically equivalent patches.
This is the problem we address in this paper.

We introduce a novel neural repair model, called RewardRepair based on a mixed
learning objective. The key insight in the design of RewardRepair is the
combination of a syntactic training objective at the token-level and a
semantic training objective based on program execution. RewardRepair defines a
discriminator to discriminate good patches from low-quality ones during the
training of the neural network. The discriminator is based on executing the
compiler and running the test cases on the generated patches, providing high
qualified execution feedback on their quality. This feedback is transformed as
a quantified reward signal that modulates the cross-entropy loss. Then, the
neural network’s weights are updated based on this novel discriminator. In
other words, in RewardRepair, backpropagation embeds essential compilation and
execution information.

We conduct large experiments to evaluate RewardRepair based on four well-
accepted datasets from the literature, including Defects4J version 1.2 (Just
et al., 2014), Defects4J version 2.0 (Just et al., 2014), Bugs.jar (Saha et
al., 2018) and QuixBugs (Lin et al., 2017). First, we show that RewardRepair
produces more correct patches than the recent related work. In total,
RewardRepair repairs 207 on the four benchmarks. RewardRepair achieves an
improvement in two benchmarks Defects4J(v2.0) and Bugs.jar, and achieves the
top-2 performance in the other two benchmarks Defects4J(v1.2) and QuixBugs.
Second, we demonstrate that RewardRepair outperforms the state-of-the-art on
addressing the compilability problem in neural program repair, CURE (Jiang et
al., 2021), by producing a higher ratio of compilable patches in all
considered beam size configuration (45.3% versus 39% in top-30, and 37.5%
versus 28% in top-100 candidate patches).

To sum up, our contributions are:

  * •

We devise, RewardRepair, a neural program repair approach with execution-based
backpropagation. RewardRepair defines a novel training objective that employs
compilation and test execution information to optimize the neural network.

  * •

We perform an original series of experiments to show that RewardRepair’s
training objective outperforms the cross-entropy loss used in related work.
Our experimental results demonstrate that embedding execution information in
backpropagation improves the quality of generated patches (more compilable
patches and correct patches).

  * •

We provide evidence that RewardRepair can correctly fix 45 bugs for
Defects4J(v1.2), 45 bugs for Defects4J(v2.0), 97 bugs for Bugs.jar and 20 bugs
for QuixBugs. We report that ReparRepair can correctly fix 121 bugs that were
never repaired in previous literature, including 5 unique Defects4J(v1.2)
bugs.

  * •

We make all our data and code publicly available for future research
(Experiment, 2022).

##  2\. Background

###  2.1. Neural Program Repair

Neural Machine Translation (NMT) systems have recently achieved state-of-the-
art performance on program repair tasks, forming a field called “neural
program repair” (Chen et al., 2019; Lutellier et al., 2020; Chakraborty et
al., 2020; Tufano et al., 2019a; Gupta et al., 2017; Tufano et al., 2019b; Zhu
et al., 2021; Jiang et al., 2021). Despite the difference in their inputs and
neural models, those works are similar in the sense that they are all based on
a typical NMT model formulated as an encoder-decoder-attention architecture
optimized with cross-entropy loss function (Bahdanau et al., 2014; Vaswani et
al., 2017a, b; Zhang et al., 2019; Lu et al., 2020). All the prior works on
program repair are based on the NMT architecture with a cross-entropy loss
function to update the neural network weights (Chen et al., 2019; Lutellier et
al., 2020; Chakraborty et al., 2020; Tufano et al., 2019a; Gupta et al., 2017;
Tufano et al., 2019b; Zhu et al., 2021; Jiang et al., 2021). The cross-entropy
loss (a.k.a. log loss) is a measure from information theory, building upon
entropy and calculating the difference between two probability distributions
(Bahdanau et al., 2014; Ghazvininejad et al., 2020). During the training of a
neural repair model, the cross-entropy loss calculates the difference between
the generated tokens and the human-written patch tokens in a strict pairwise
matching manner (Bahdanau et al., 2014; Zhang et al., 2019; Lu et al., 2020),
and is used to update the neural network weights by backpropagation. In
program repair patches, a low cross-entropy value means that the generated
patch is syntactically close to the ground truth patch at the token-level.

###  2.2. Limitations of Current Neural Repair

The token-based cross-entropy loss optimization is effective at guiding the
model to generate patches syntactically identical or close to the human-
written patches given as input during training. However, it suffers from two
major limitations. Firstly, a major goal in patch generation is to generate
well-formed patches that compile. Unfortunately, the cross-entropy loss does
not favor patches that compile over non-compilable patches. Even worse, if a
generated patch has a significantly lower loss per the token-based cross-
entropy, although being non-compilable, it would be favored by the model at
training time. Secondly, the cross-entropy loss function fails to learn from
semantically equivalent patches: a syntactically different but semantically
equivalent patch could potentially have a high cross-entropy loss value. This
means that the cross-entropy loss discourages the network to explore
equivalent solutions. In the field of neural machine translation (NMT), this
problem is known as the overcorrection problem of cross-entropy loss (Zhang et
al., 2019; Lu et al., 2020): cross-entropy based models tend to learn strictly
identical translations and to overcorrect synonymous tokens which would be
acceptable. Overall, the problem we address in this paper is that the cross-
entropy loss used in previous research cannot learn programming knowledge
beyond tokens.

###  2.3. Motivating Example

⬇

\- return FastMath.pow(2 * FastMath.PI, -dim / 2 ) *

\+ return FastMath.pow(2 * FastMath.PI, -0.5 * dim ) *

The human-written patch

⬇

\+ return FastMath.pow(2 * FastMath.PI, -d * dim ) *  //loss value 0.1157

A generated non-compilable patch receives a smaller loss score

⬇

\+ return FastMath.pow(2 * FastMath.PI, -dim / 2d ) * //loss value 0.4224

A generated semantically-equivalent patch receives a bigger loss score

List of listings 1 Motivating example: NMT based repair models based on cross-
entropy loss may favor non-compilable patches.

1 is a motivating example to show the drawbacks of a neural repair model based
on cross-entropy loss optimization. 1a presents the buggy code and the human-
written patch for bug Math-11 from Defects4J(v1.2). 1b gives one generated
non-compilable patch because of the undefined variable $d$. The network
generates this patch with maximum likelihood estimation because its cross-
entropy loss value is low (0.1157). 1c is a semantically equivalent patch
compared to the human-written patch and its computed loss is 0.4224, which is
higher than the non-compilable patch.

With cross-entropy loss, the NMT-based repair model penalizes the semantically
equivalent patch and favors the non-compilable patch. This is because there is
only one token difference for the generated non-compilable patches, where the
wrong token d is not the expected token 0.5 from the ground truth patch.
However, the token $d$ is an undefined variable in the buggy program. On the
contrary, there exist three token differences in the semantically equivalent
patch: ($dim\rightarrow\texttt{0.5}$), ($/\rightarrow*$), ($2d\rightarrow
dim$). Consequently, the NMT-based repair model considers the generated non-
compilable patch is closer to the human-written patch, and thus should be
favored during backpropagation. However, in the context of program repair, the
semantically equivalent patch should have a loss close to zero, because it is
a valid solution to the bug. The generated non-compilable patch has a lower
loss, but still cannot satisfy the compiler, which is inconsistent with our
goal. This example shows the fundamental limitation of optimizing neural
networks with the traditional token-based cross-entropy loss for the program
repair task.

Figure 1. An Overview of RewardRepair.

##  3\. RewardRepair

###  3.1. Overview

In this paper, we propose a novel neural repair model called RewardRepair. The
core idea of RewardRepair is to improve the learning process of neural program
repair and in particular to address the limitations of using the cross-entropy
loss function at the token-level. Figure 1 gives an overview of RewardRepair.
The top, middle and bottom parts represent three stages of RewardRepair:
syntactic training, semantic training and inference. RewardRepair is trained
with two datasets respectively, a syntactic training dataset with pairs of
buggy and patch code, and a semantic training dataset. They are fundamentally
different. The syntactic training dataset only consists of textual patches as
in the related work (Jiang et al., 2021; Lutellier et al., 2020; Chen et al.,
2019). However, the requirements for the semantic training dataset are full
execution: each sample in the dataset comes with a compiler configuration and
test cases. Achieving full execution enables us to derive execution-based
information to be used for optimizing the neural network weights with
programming knowledge during backpropagation.

Syntactic training is our first training phase. We train RewardRepair to
optimize the cross-entropy loss based on a large bug fix corpus per the
related work (Jiang et al., 2021; Lutellier et al., 2020; Chen et al., 2019;
Gupta et al., 2017; Li et al., 2020; Tufano et al., 2019a; Chakraborty et al.,
2020). Syntactic training is meant to provide a good initial model for
semantic training. Semantic training is our second phase after syntactic
training. Semantic training is based on a discriminative model. The
discriminative model of RewardRepair analyzes the candidate patches with four
discriminators (difference discriminator, compilability discriminator,
plausibility discriminator and regression discriminator), and modulates the
cross-entropy loss before the start of backpropagation on training. The
inference is the final phase. Once RewardRepair has been trained, it can
generate patches for new and unseen bugs based on the trained patch generator.

###  3.2. Code Representation

As code representation, we follow Lutellier et al. (Lutellier et al., 2020) to
represent the buggy code and context code as two separate token sequences. In
RewardRepair, the context code is considered as 10 lines of code surrounding
the buggy code. In addition, the context code is enriched with a summary of
the buggy class as proposed by Chen et al. (Chen et al., 2019) as follows: we
keep all the instance variables and their initializers, along with the
signature of the constructors and methods. We follow the existing work (Jiang
et al., 2021) to use subword tokenization with SentencePiece (Kudo and
Richardson, 2018), as demonstrated useful by Karampatsis et al. (Karampatsis
et al., 2020).

###  3.3. Patch Generator

In RewardRepair, the patch generator is trained in a supervised manner based
on an encoder-decoder architecture (Raffel et al., 2020; Vaswani et al.,
2017a). Syntactic training takes as input buggy code tokens
$B=(b_{0},b_{1}\ldots b_{n})$, context code tokens $C=(c_{0},c_{1}\ldots
c_{m})$, and tokens from the ground truth fix patch $F=(f_{0},f_{1}\ldots
f_{t})$. RewardRepair transforms the $B$ and $C$ into a predicted patch
$F^{\prime}=(f^{\prime}_{0},f^{\prime}_{1}\ldots f^{\prime}_{k})$. Note that
size of the buggy code, context code, ground truth patch and predicted patch,
i.e., $n$, $m$, $t$ and $k$, can be different. In RewardRepair, patch
generation is shared by both syntactic training and semantic training.

###  3.4. Syntactic Training of RewardRepair

RewardRepair initially trains the patch generator with cross-entropy loss
function per the state-of-the-art of NMT for program repair (Lutellier et al.,
2020; Chen et al., 2019; Gupta et al., 2017; Li et al., 2020; Tufano et al.,
2019a; Chakraborty et al., 2020; Jiang et al., 2021). For each training point,
the optimization target is to minimize the loss between ground truth fix patch
$F$ and the predicted patch $F^{\prime}$. As shown in the previous study
(Jiang et al., 2021; Lutellier et al., 2020), syntactic training is trained
with a large corpus of buggy code and fixed code pairs. Syntactic training
could be trained for multiple epochs to achieve convergence and obtain the
best combination of weights. By the end of syntactic training, the patch
generator’s weights between the connections of the networks are optimized.

###  3.5. Semantic Training of RewardRepair

The goal of semantic training is to let the patch generator be aware of
program-specific knowledge (compilation and execution) beyond the syntactic
loss computation at the token-level. For that, we propose a mixed learning
objective (Lu et al., 2020), where “mixed” means that the core learning
objective is combined with two or more sub-learning objectives. Our mixed
learning objective combines the core cross-entropy objective with compilation
and execution information. Once the RewardRepair has been sufficiently trained
with syntactic training, we start the semantic training process alternately
with syntactic training.

For semantic training, RewardRepair employs a discriminative model to assess
the quality of generated patches based on compilation and test oracles. As a
result, the discriminative model outputs a reward value that quantifies the
patch quality, which is used to adjust the weights of the patch generator
during backpropagation. A higher reward means better quality for the generated
patch, e.g., the patch is compilable and passes all test cases. On the
contrary, a lower reward means that the quality of the generated patch may be
unsatisfying, e.g., the patch is non-compilable.

Precisely, the patch reward value modulates the token-level loss value before
the start of the backward pass (Micikevicius et al., 2018; Zhao et al., 2019;
Ryou et al., 2019), so that the updated loss can then be properly represented
with program-specific knowledge. In such a way, the discriminator guides the
neural network to generate high-quality patches. To our knowledge, we are the
first to introduce semantic training based on a patch discriminative model for
neural program repair.

####  3.5.1. Discriminative model

In machine learning, a discriminator is a model for identifying “good” and
“bad” predictions (Goodfellow et al., 2014). The discriminator of RewardRepair
is the key component of the semantic training phase. Its goal is to measure
the quality of the generated patches, which is then quantified as a reward
signal. In RewardRepair, the discriminative model is composed of four
discriminators, each of them specialized in one aspect of patch quality. These
four discriminators are serially executed. Each discriminator does a binary
classification: whether a patch fulfills the discriminator’s criterion or not.
When a discriminator is affirmative, it means the patch passes the quality
criterion, and it is passed to the next discriminator. In the rest of this
section, we present the four discriminators of RewardRepair.

##### Difference discriminator

The difference discriminator validates whether the generated patches are
different from the given buggy code. It is required because we found that
neural repair models regularly generate a patch that is identical to the given
buggy code, i.e., the output of the model is identical to the input of the
model (called a “no-change” patch in this paper). This happens when the neural
nets discover that the buggy code achieves the maximum likelihood estimation.
This is explained by previous research (Tian et al., 2020), which has shown
that the many buggy codes are similar to correct code, with only minor
transformations and few changed tokens. Consequently, the generator tends to
copy the buggy code because it is the maximum likelihood prediction, per the
data seen at training time. We design the difference discriminator to
syntactically compare the generated patch code with the input buggy code, with
a token sequence comparison. If the buggy code and generated patched code are
the same, the difference discriminator assigns the generated patch a penalty,
called in this paper the no-change penalty ($R_{no-change}$), i.e., a negative
reward. This penalty signal modulates the RewardRepair loss to avoid the
generation of no-change patches.

##### Compilability discriminator

The compilability discriminator validates whether the generated patches are
compilable. As shown in previous research (Jiang et al., 2021), neural repair
models suffer from outputting non-compilable patches. For example, the golden
sequenceR model produces 79.7% of patches that are not compilable in top-50
candidate patches. To force the generator towards producing compilable
patches, we introduce a compilability discriminator in the semantic training
phase. The compilability discriminator employs the compiler to compile the
generated patched program. If the patched program is compilable, RewardRepair
assigns a compilable reward ($R_{compilable}$) and passes the patch to the
next discriminator for further assessment. Otherwise, a non-compilable penalty
($R_{non-compilable}$), i.e., a negative reward, is returned to the
RewardRepair model.

##### Plausibility discriminator

The plausibility discriminator aims at encouraging the generator to produce
patches that pass the human-written test cases provided by developers. Recall
that, per the seminal work on program repair (Le Goues et al., 2012), test
cases can be used as an executable program specification. In other words, if a
patch passes the human-written tests, it is considered as a plausible patch.
During semantic training, RewardRepair is trained on buggy projects with
executable human-written tests. Each candidate patch is executed against the
human-written tests. If a patch makes all human-written tests pass, a
plausible reward ($R_{plausible}$) is assigned. By doing so, the plausibility
discriminator leverages the human-written tests to drive the network to
produce plausible patches that pass all human-written tests.

##### Regression discriminator

The last discriminator used in RewardRepair’s discriminative model is the
regression discriminator. The goal of this discriminator is to minimize the
behavioral changes introduced by the patch. This discriminator complements the
plausible patch discriminator by specifying behavior outside the human-written
tests, in order to avoid patch overfitting (Yu et al., 2018; Le et al., 2018,
2019; Smith et al., 2015). The RewardRepair regression discriminator employs
automatically generated tests, per the RGT technique of Ye et al. (Ye et al.,
2021b). The effectiveness of this technique to identify correct patches from
plausible patches has been shown in recent work (Wang et al., 2020; Ye et al.,
2021a, c). The idea of the RGT technique is to automatically generate test
cases based on the ground truth patched program to expose program execution
behavior differences between a generated patch and a ground truth patch (Le et
al., 2019; Wang et al., 2020; Ye et al., 2021b). If a candidate patch makes
all RGT tests pass, i.e., it does not contradict the ground truth program
behavior, it is considered as likely-correct.

During semantic training, all patches are executed against the RGT tests. If a
candidate patch makes all automatically generated tests pass, meaning the same
program execution behavior with ground truth program, then a likely-correct
reward signal ($R_{l-correct}$) is assigned to this patch. This
discriminator’s reward is used to encourage the RewardRepair to avoid
regressions. _This means we encourage RewardRepair to generate non-overfitting
patches beyond the existing test cases._

####  3.5.2. Defining reward values from discriminators

As shown previously, RewardRepair defines five reward signals for patch
quality assessment. The discriminators are executed serially, that is, if a
patch _does not satisfy_ one discriminator, then the reward $R$ obtained up to
that moment is returned immediately and other discriminators are not executed.
Consequently, the $R$ is the maximum of the five reward values as follows:

(1) $\footnotesize R=max\begin{cases}R_{no-change}=s_{0}\\\ R_{non-
compilable}=s_{1}\\\ R_{compilable}=s_{2}\\\ R_{plausible}=s_{3}\\\
R_{l-correct}=s_{4}\\\ \end{cases}$

where $s_{i}$ ($i\in\\{0,1,2,3,4\\}$) are five scaling parameters that control
the range of the reward values. The scaling parameters of $s_{i}$ define a
configuration space that is controlled by end-users of RewardRepair.
Additionally, those reward values must fulfill the following constraint:

(2) $\footnotesize R_{no-change}<R_{non-
compilable}<R_{compilable}<R_{plausible}<R_{l-correct}$

where the higher reward value represents the better quality of the generated
patch.

Given the cross-entropy $\mathcal{L}$, the loss function of RewardRepair
$\mathcal{L}_{RewardRepair}$ dynamically reweighs $\mathcal{L}$ with respect
to the discriminator’s reward signal $R$, which encodes the quality of the
generated patch. We follow (Yu et al., 2017; Ryou et al., 2019) to formulate
RewardRepair loss function as a scaling of the cross-entropy, as follows:

(3) $\footnotesize\mathcal{L}_{RewardRepair}=(1-R)*\mathcal{L}\quad(where\quad
R<1)$

The reward modulator $(1-R)$ constrains the domain of $R\in(-\infty,1)$, as it
is meaningless for the objective function to minimize a negative loss. In this
formulation, the syntactic cross-entropy at the token-level is combined with
the semantic reward at patch-level, embedding compilation and execution
knowledge deep into the neural model. It mitigates the limitations of only
considering cross-entropy loss in program repair tasks and solves the problem
discussed in Section 2.3.

For low-quality patches, RewardRepair assigns a penalty, i.e., negative reward
value, to increase the original cross-entropy loss $\mathcal{L}$. Thus the
domain for $R_{no-change},R_{non-compilable}\in(-\infty,0)$. On the contrary,
for the high-quality patches, RewardRepair scales down the original
$\mathcal{L}$ by assigning positive reward values for $R_{compilable}$,
$R_{plausible}$, and $R_{l-correct}$ to encourage the model. The domain for
these three reward values is $[0,1)$. Based on Equation 3, the higher the
reward, the smaller computed loss is back-propagated into the neural model.
The extreme case is the highest reward value approximate to 1, where the
RewardRepair loss goes close to 0. This indicates that the generated patch is
likely correct and the network should not be changed.

Combining the domain constraints discussed above, the range of scaling
parameters $s_{i}$ must meet the following criteria:

(4) $\footnotesize\begin{cases}s_{0}\in(-\infty,0)\\\ s_{1}\in(s_{0},0)\\\
s_{i}\in[0,1),s_{i-1}<s_{i},i\in\\{2,3,4\\}\\\ \end{cases}$

Algorithm 1 One step of semantic training in RewardRepair

1: Input: buggy code b, context code c, ground-truth patch code p, human-
written test cases $t_{h}$, RGT test cases $t_{m}$, learning rate $\alpha$, G
is the RewardRepair patch generator, D is the discriminator function

2: $\widetilde{b}\leftarrow\varphi(b)$ {Encode buggy code}

3: $\widetilde{c}\leftarrow\varphi(c)$ {Encode context code}

4: $\widetilde{p}\leftarrow\varphi(p)$ {Encode ground-truth patch code}

5: $\widetilde{q}\leftarrow G(\widetilde{b},\widetilde{c})$ {Generate
candidate patch }

6: $\mathcal{L}=\sum_{x\in X}\widetilde{p}(x)log\,\widetilde{q}(x)$

7: $R\leftarrow D(\widetilde{b},\widetilde{c},\widetilde{q},t_{h},t_{m})$

8: $\mathcal{L}_{Reward\\-Repair}=(1-R)*\mathcal{L}$

9: $G\leftarrow G-\alpha\partial L_{Reward\\-Repair}/\partial G$ {update patch
generator with backpropagation}

####  3.5.3. Algorithm

Algorithm 1 presents one step of semantic training. Given the encoded buggy
$\widetilde{b}$ and context code $\widetilde{c}$ (line 2 and 3), the patch
generator $G$ generates a candidate patch (line 5). Then, the cross-entropy
loss $\mathcal{L}$ is computed by comparing the token distribution between the
ground truth patch $\widetilde{p}$ and the generated patch $\widetilde{q}$
(line 6), where the $x$ indicates the index of tokens. The discriminator
provides a reward $R$ based on generated patch $\widetilde{q}$ and the
corresponding program compilability and test execution information (line 7).
Lastly, RewardRepair combines the cross-entropy loss at token-level and reward
value at patch-level to form RewardRepair loss (line 8), which encodes the
program-specific knowledge.

###  3.6. Inference

At inference phase, for a given suspicious statement found by fault
localization tools (e.g., Ochiai (Abreu et al., 2007)), RewardRepair
represents it with two sequences of tokens: one for the suspicious statement,
the other one for its context (see Section 3.3). Those tokens are given to the
patch generator of RewardRepair, previously trained as explained in Sections
3.4 and 3.5. As RewardRepair is configured by the inference beam size $n$ (see
(Jiang et al., 2021; Chen et al., 2019)), it outputs the $n$ best patches for
that suspicious statement. RewardRepair can be used with any fault
localization technique in real-world bug repair tasks, as shown by (Baudry et
al., 2021).

###  3.7. Implementation

We implement RewardRepair’s patch generator with the state-of-the-art
Transformer based architecture (Raffel et al., 2020) from Hugging Face.
RewardRepair is trained with 15 syntactic training epochs and 4 semantic
training epochs. For hyper-parameters configuration, we use a vocabulary size
of 32 128. We configure RewardRepair to take a maximum of $512$ input tokens
from buggy and context code, and generate a patch with a maximum of $100$
tokens. The learning rate sets to $1\mathrm{e}{-4}$ for both syntactic and
semantic training. We configure reward scaling values
$s_{i},i\in\\{0,1,2,3,4\\}$ respectively to {-0.4, -0.2, 0.2, 0.4, 0.6} for
the best experiment result. The encoder and decoder consist of 6 layers.
RewardRepair is configured by a beam size of $200$ and outputs the $200$ best
patches per bug. We consider the beam size of $200$ rather than $1000$ used in
CURE (Jiang et al., 2021) and CoCoNuT (Lutellier et al., 2020) due to the
limitations of our available GPUs.

##  4\. Experimental Methodology

In this section, we describe our methodology for evaluating RewardRepair by
defining three research questions and how we propose to answer them.

###  4.1. Research Questions

  * •

RQ1 (comparison with other tools): To what extent is RewardRepair effective at
repairing bugs compared with the state-of-the-art repair approaches?

  * •

RQ2 (compilable rate): To what extent does RewardRepair improve the
compilability of generated patches?

  * •

RQ3 (impact of semantic training): To what extent does semantic training
improve the effectiveness of RewardRepair?

###  4.2. Dataset

Phases |  Requirements |  Name & Source |  #Patches
---|---|---|---
Syntactic training |  Tokenization |  CoCoNuT (Lutellier et al., 2020) |  3 241 966
MegaDiff (Monperrus et al., 2021) |  240 306
CodRep (Chen and Monperrus, 2018) |  24 969
Semantic training |  Tokenization, Compilation |  Bears (Madeiral et al., 2019) |  123
Developer and RGT Tests
Testing |  Tokenization, Compilation, Developer Tests |  Defects4J(v1.2) (Just et al., 2014) |  120
Defects4J(v2.0) (Just et al., 2014) |  257
Bugs.jar (Saha et al., 2018) |  490
QuixBugs (Lin et al., 2017) |  34
Table 1. Datasets used for the different steps of our experiment.

Recall that we need three datasets for syntactic training, semantic training
and testing.

We have common criteria for both training and testing datasets:  1) all
datasets are composed of bug-fix patches;  2) we focus on single-file patches,
per previous work (Chen et al., 2019; Li et al., 2020; Liu et al., 2019);  3)
we focus on single-hunk patches, where the patch is confined to a single
contiguous chunk of code, at a single location, per previous work (Chen et
al., 2019; Li et al., 2020; Liu et al., 2019);  4) we discard patches that do
not make program behavior differences, e.g., those with only changes in
comments or logging.

Next, we have specific requirements per dataset. Table 1 shows the dataset of
patches that we use for training and evaluating RewardRepair. The first column
indicates the phase where each dataset is used. The second column gives the
requirements for each dataset. The third column gives the source of the
dataset and the fourth column indicates the number of patches in this dataset.
For example, as shown in the first row, RewardRepair is syntactically trained
with data from three different sources, CoCoNuT (Lutellier et al., 2020),
Megadiff (Monperrus et al., 2021) and CodRep (Chen and Monperrus, 2018).

As aforementioned in Section 3, the selection criteria for semantic training
dataset are:  1) to be able to compile the patched program;  2) to run the
test cases on the patched program;  3) to be able to automatically generate
tests to specify the expected program behavior.  All criteria are met for 123
single-hunk bugs of the Bears dataset (Madeiral et al., 2019), for which some
available RGT tests were generated by previous research (Ye et al., 2021a). We
use the evaluation mode of RewardRepair to create more semantic training
points based on beam search, as done in (Kommrusch et al., 2021). It is to be
noted that those criteria are very strong, and neither CoCoNuT (Lutellier et
al., 2020), MegaDiff (Monperrus et al., 2021) nor CodRep (Chen and Monperrus,
2018) meets them, in particular, the patched program cannot be compiled.

To test RewardRepair, we use well-accepted datasets from program repair
research (Liu et al., 2020; Martinez et al., 2017; Durieux et al., 2019; Ye et
al., 2021c): Defects4J (Just et al., 2014), Bugs.jar (Saha et al., 2018), and
QuixBugs (Lin et al., 2017). For all those bug datasets, the requirements of
compilation and test execution are met. In line with the most recent work (Zhu
et al., 2021), we also consider the bugs of Defects4J version 2.0. After
filtering single-hunk bugs, we use 120 bugs from Defects4J(v1.2) and 257
additional new bugs from Defects4J version 2.0 denoted as Defects4J(v2.0) in
our paper. We use the same single-hunk bugs criteria for Bugs.jar and
QuixBugs.

###  4.3. Methodology for RQ1

In RQ1, we compare RewardRepair against the state-of-the-art neural repair
approaches: CURE (Jiang et al., 2021), Recoder (Zhu et al., 2021), CoCoNuT
(Lutellier et al., 2020), and other approaches (Saha et al., 2019; Liu et al.,
2019; Jiang et al., 2018; Wen et al., 2018; Yuan and Banzhaf, 2018; Xin and
Reiss, 2019; Saha et al., 2017; Xuan et al., 2016; Martinez and Monperrus,
2016). Per previous studies (Lutellier et al., 2020; Jiang et al., 2021), we
take the quantitative results from the literature.  We run RewardRepair under
two fault localization modes. First, we use spectrum-based fault localization
with Gzoltar (Riboira and Abreu, 2010) per previous work (Le Goues et al.,
2012; Xuan et al., 2016). Second, we assume that the fault has been localized,
an evaluation technique known as perfect fault localization and extensively
used in recent work (Liu et al., 2020; Jiang et al., 2021; Lutellier et al.,
2020).

We compute the two traditional APR performance metrics for each testing
dataset:  1) the number of bugs that are correctly repaired. In our paper, a
patch is deemed correctly repaired if it meets either of the two following
criteria: it is identical to the developer patch, or it is considered as
correct by manual analysis done by at least two authors;  2) the number of
bugs that can be uniquely repaired by individual repair approaches.

###  4.4. Methodology for RQ2

In RQ2, we calculate the compilable rate of RewardRepair. As compilable rates
were reported in SequenceR (Chen et al., 2019), CoCoNuT (Lutellier et al.,
2020) and CURE (Jiang et al., 2021), we use the same benchmarks
Defects4J(v1.2) and QuixBugs as they do, and compare against the numbers
reported in the original papers. We also follow the existing work (Jiang et
al., 2021) and compute the compilable rate depending on the beam size. We
report on beam sizes in 30, 100 and 200. We do not consider larger beams due
to the limitations of our available GPUs.

###  4.5. Methodology for RQ3

In RQ3, we conduct an ablation study with the goal of measuring the effect of
semantic training. To understand the contribution of semantic training, we
compare the effectiveness of RewardRepair considering:  1) only syntactic
training;  2) both syntactic and semantic training.  For doing this study, we
apply the same protocol as the one used for responding to the RQ1. We conduct
manual analysis on the unique bugs that are only repaired by including
semantic training and present the most interesting categories on repair action
changes.

##  5\. Experimental Results

Approaches | D4J(v1.2) | D4J(v2.0) | Bugs.jar | QuixBugs
---|---|---|---|---
| 120 bugs | 257 bugs | 490 bugs | 34 bugs
Using Spectrum-based Fault Localization
jGenProg (Martinez and Monperrus, 2016) | 5 | - | - | 1
Nopol (Xuan et al., 2016) | 5 | - | - | 2
Elixir (Saha et al., 2017) | 26 | - | 22 | -
sharpFix (Xin and Reiss, 2019) | 27 | - | 15 | -
SimFix (Jiang et al., 2018) | 27 | 2 | - | -
Hercules (Saha et al., 2019) | 33 | - | - | -
Recoder (Zhu et al., 2021) | 39 | 19 | - | 17
RewardRepair (this paper) | 29 | 24 | 42 | 19
Assuming Perfectly Localized Fault
SequenceR (Chen et al., 2019) | 14 | - | - | -
DLFix (Li et al., 2020) | 33 | - | - | -
TBar (Liu et al., 2020) | 33 | 8 | - | -
CoCoNuT (Lutellier et al., 2020) | 33 | - | - | 12
CURE (Jiang et al., 2021) | 45 | - | - | 24
Recoder (Zhu et al., 2021) | 52 | - | - | -
RewardRepair (this paper) | 45 | 45 | 97 | 20
RewardRepair Unique | 5 | 34 | 78 | 4
Table 2. Comparison of RewardRepair against the related work, the numbers from
the related work are filtered by single-hunk bugs. Across all benchmarks,
RewardRepair correctly fixes 207 bugs and uniquely fixes 121 ones. We use 4
testing benchmarks to maximize generalizability.

###  5.1. RQ1: Comparative Study with Other Repair Approaches

⬇

if (_dataFormatReaders != null) {

return _detectBindAndReadValues(_dataFormatReaders.findFormat(

src,offset,length), false);}

\- return _bindAndReadValues(_considerFilter(_parserFactory.createParser(src),

\+ return _bindAndReadValues(_considerFilter(_parserFactory.createParser

\+ (src,offset,length),

true));

List of listings 2 RewardRepair correct patch for Defects4J(v2.0)
JacksonDatabind_57

Table 2 shows the patch generation results of RewardRepair and 12 other APR
approaches on four benchmarks: the two versions of Defects4J, Bugs.jar and
QuixBugs. The numbers are the correctly repaired bugs by each APR approach.
The results are those reported in the literature, by the authors of the tool
or by subsequent comparative experiments (Liu et al., 2020; Ye et al., 2021c).
A ‘-’ indicates that the APR approach has not been evaluated on the considered
benchmark, to the best of our knowledge.  Note that the first seven APR
approaches were executed with spectrum-based fault localization (FL), where
the later approaches assumed perfect FL. We measure RewardRepair’s
effectiveness with both spectrum-based and perfect FL. Next, we focus on the
comparison under perfect FL as the most state-of-the-art techniques only
report effectiveness with perfect FL.

Repaired bugs. Overall, RewardRepair is able to correctly repair 45 of 120
bugs on Defects4J(v1.2), 45 of 257 bugs on Defects4J(v2.0), 97 of 490 bugs on
Bugs.jar and 20 of 34 bugs on QuixBugs benchmark. From these results, we make
the following observations.

RewardRepair outperforms all APR approaches in two benchmarks: Defects4J(v2.0)
and Bugs.jar. RewardRepair sets new baselines of repaired bugs for these two
benchmarks. While the majority of APR papers showcase bugs from version 1.0 or
1.2 of Defects4J, 2 gives the correct RewardRepair patch for a Defects4J(v2.0)
bug: JacksonDatabind_57. As shown, RewardRepair succeeds in reusing the
surrounding variables offset and length to construct the parameter list for
overridden method $createParser$. Recoder (Zhu et al., 2021), which is so far
the best tool evaluated on Defects4J(v2.0) fails at repairing this bug.

Figure 2. Uniquely repaired bugs on Defects4J(v1.2).

⬇

private void removeUnreferencedFunctionArgs(...){

\+ if (!removeGlobals) {

\+ return;

\+ }

List of listings 3 RewardRepair patch for Closure-1 from Defects4J(v1.2),
identical to the developer patch.

In addition, RewardRepair achieves the top-2 performance on Defects4J(v1.2)
and QuixBugs benchmarks. RewardRepair performs better than all APR approaches
on Defects4J(v1.2) but Recoder. Regarding Recoder’s performance on
Defects4J(v1.2), we analyze the bugs that cannot be repaired by RewardRepair.
We find that three bugs (Closure-14, Closure-104, and Closure-118) require
fixing tokens outside the considered buggy class and three bugs (Lang-26,
Lang-43 and Closure-33) require fixing tokens outside the context code scope
as implemented by RewardRepair. This analysis suggests that RewardRepair could
achieve better performance by enlarging the context code scope. Regarding
CURE’s performance on QuixBugs, the reason is likely the beam size: recall
that CURE generates 10,000 candidate patches for each bug, while RewardRepair
generates 200 patches per bug. As shown in previous research (Tufano et al.,
2019a), a larger beam size leads to more correct patches.

Uniquely repaired bugs. Let us now focus on the last row of Table 2, which
gives the number of uniquely repaired bugs by RewardRepair. RewardRepair
respectively repairs 5, 34, 78 and 4 unique bugs for Defects4J(v1.2),
Defects4J(v2.0), Bugs.jar and QuixBugs, all of which were never repaired by
any other APR approaches in the literature. This shows RewardRepair
complements all the existing works on the four considered benchmarks.

Figure 2 gives the detailed uniqueness analysis on Defects4J(v1.2) with the
state-of-the-art APR approaches. We give the exact results for the most recent
neural repair approaches (CoCoNuT, CURE and Recoder). We combine the rest of
the top-ranked related work in a unique bin for sake of readability. As shown,
RewardRepair fixes 5 unique bugs compared with the other APR approaches on
Defects4J(v1.2). Notably, they come from three different Defects4J projects
(Closure, Lang and Math) showing that the additional learned knowledge is not
specific to one single domain.

3 gives the RewardRepair patch for Closure-1, which can only be fixed by
RewardRepair. This is a patch with an addition of an if block, including the
code of the then statement. RewardRepair learns the if condition from the
given context code. While pattern-based repair is able to synthesize
conditions (e.g., TBar (Liu et al., 2019)), no pattern-based repair systems
have this complete if/then/return pattern. The recent neural repair models,
CoCoNuT and CURE, do not generate this patch, we suspect that with a strict
token-based cross-entropy optimization, Recoder does not learn such a complex
patch structure. RewardRepair is the first to produce this addition-only patch
based on a non-trivial if/then/return structure.

Model |  Top-30 |  Top-100 |  Top-200
---|---|---|---
SequenceR (Chen et al., 2019) |  33% |  - |  -
CoCoNuT (Lutellier et al., 2020) |  24% |  15% |  6%-15%
CURE (Jiang et al., 2021) |  39% |  28% |  14%-28%
RewardRepair |  45.3% |  37.5% |  33.1%
Table 3. Average compilable rates of the Top-K candidate patches in
Defects4J(v1.2) and QuixBugs. ‘-’ indicates data unavailability.

Generalizability. Durieux et al. (Durieux et al., 2019) revealed the
phenomenon of “benchmark overfitting” in program repair, meaning that
performance results reported in the APR literature do not generalize to other
benchmarks. The main reason is that APR approaches were typically evaluated in
a single dataset, in particular, Defects4J(v1.2) in Java evaluation.
RewardRepair is evaluated on four benchmarks in order to maximize the
generalizability of our claims. To our knowledge, this is one of the
experiments with the largest number of testing benchmarks used for assessing
the proposed repair approach.

Answer to RQ1: RewardRepair correctly fixes 45, 45, 97 and 20 bugs on the
considered Java benchmarks Defects4J(v1.2), Defects4J(v2.0), Bugs.jar and
QuixBugs, respectively. There are 121 unique bugs that are repaired by
RewardRepair for the first time ever w.r.t. the APR literature. The external
validity of our results is founded on 4 testing benchmarks.

Figure 3. RewardRepair compilable rate on Defects4J(v1.2) by project.

###  5.2. RQ2: Improvement of Compilable Rate

⬇

for (FormattingOption formattingOption : flags.formatting) {

formattingOption.applyToOptions(options); }

\- if (flags.process_closure_primitives) {

\- options.closurePass = true;

\- }

\+ options.closurePass = flags.process_closure_primitives;

initOptionsFromFlags(options);

return options;

}

List of listings 4 RewardRepair correct patch generated for Closure-101

Table 3 shows the average compilable rates of the top-k candidate patches
where k relates to the beam size. For sake of a fair comparison, we use the
same methodology as Jiang et al. (Jiang et al., 2021) and we combine
Defects4J(v1.2) and QuixBugs together (our appendix website gives the results
per benchmark (Experiment, 2022)). We provide ranges for CoCoNuT and CURE
regarding the top-200 results due to data unavailability: the range is the
bracket [top-100, top-1000] of compilable rates as reported in the original
paper of CURE (Jiang et al., 2021).

Notably, the compilable rate of RewardRepair outperforms the three considered
approaches CURE, CoCoNuT and SequenceR for all beam size configurations. In
the best case, RewardRepair achieves a compilable rate of up to 45.3%, which
is the highest among the three considered beam sizes. Next, we see that the
compilable rate of RewardRepair decreases when we increase the beam size, this
result is consistent with the ones of CURE and CoCoNuT. Since the beam
enumerates by decreasing probability, it suggests that the learned neural
model does capture compilability and favors it.

Recall that the key contribution of CURE (Jiang et al., 2021) is to introduce
two strategies to increase the patch compilable rate of NMT based neural
repair models: valid-identifier checker strategy and length-control strategy.
Both strategies work in the inference phase, by filtering out the invalid
tokens in the Java code or patches that are not close in length to the buggy
code. This means that CURE does not embed program-specific knowledge in the
neural network. On the contrary, RewardRepair learns this knowledge during
semantic training. That is if an invalid identifier is used during semantic
training and results in a non-compilable patch, RewardRepair punishes the
patch by increasing the loss. Beyond identifiers, RewardRepair is also able to
learn other programming knowledge during semantic training, such as structure
and typing constraints related to the domain classes and methods. To some
extent, CURE is limited to the static analysis checks devised and implemented
by its authors while RewardRepair works in a fully agnostic, data-driven way
to identify important compilation constraints.

We analyze the 1883 uncompilable patches from the top-30 patches generated for
Defects4J(v1.2). We show the most frequent 10 compilation errors in Table 4.
If there are multiple errors for one patch, we only count the first error per
patch. The first column gives the compilation error type and the second column
shows the number of patches that fail on the corresponding error type. We see
most common error types are related to semantics that very few errors are
related to syntax (the first one being ”;” expected). To overcome the typing
problems, this suggests doing some specific training on the project under
repair to capture this missing knowledge (Ye et al., 2022).

4 shows the RewardRepair patch for bug Closure-101 from Defects4J(v1.2), which
is identical to the developer patch. RewardRepair incorporates two repair
actions in this patch: First, RewardRepair removes the if/then condition.
Second, RewardRepair generates a new statement by combining the logical
expression from the if-condition and from the statement in the then block.
This is arguably a complex patch, and no repair system has reported generating
a patch for this bug. This case shows that RewardRepair generates a compilable
and correct patch with the complex structure. CURE fails to generate this
patch because of its length-control strategy that encourages patches similar
to the buggy code in length.

Compile Errors | No. Failures
---|---
cannot find symbol | 606
illegal start of expression | 329
no suitable method constructor found for… | 132
incompatible types | 123
not a statement | 102
”;” expected | 76
unreachable statement | 63
case, default,or } expected | 59
incomparable types | 56
method X in class Y cannot be applied to given types | 53
Table 4. Analysis of Top-10 reasons for uncompilable generated patches after semantic training. Model |  D4J(v1.2) |  D4J(v2.0) |  Bugs.jar |  QuixBugs |  Total
---|---|---|---|---|---
RewardRepair (Syntatic) |  42 |  40 |  93 |  18 |  193
RewardRepair (Syntatic+Semantic) |  45 |  45 |  97 |  20 |  207
Table 5. Ablation study w.r.t correct patches.

Figure 3 shows the compilable rate of RewardRepair per project of
Defects4J(v1.2) with top 30, 100 and 200 candidate patches according to beam.
We make the two observations as follows: First, for all projects, increasing
the beam size of RewardRepair decreases the compilable rate for each project.
This confirms the conclusion made in Table 3 at the level of aggregate results
over bugs and benchmarks. Second, the range of compilable rates over bugs
decreases with beam size, both the range of extreme values (whiskers) and the
range of interquartile values (boxes). We explain this by statistical sampling
(sampling 30 items yields less stable results than sampling 200). However, it
may also be that the compilable rate does change significantly for some bugs.
This latter explanation is supported by the fact that there is a clear
difference in compilable rate depending on the project (Lang patches compile
much more than Time patches). This latter phenomenon – the compilable rate
significantly varying over projects, in the worst case being 0% – is a yet
unknown limitation of neural program repair and suggests more future research
on this to increase the compilable rate in a more uniform way.

Answer to RQ2: For all considered beam sizes, RewardRepair improves the
compilable rate over the state-of-the-art. Over all benchmarks, RewardRepair
reaches up to 45.3% of compilable patches (approximately one out of two
patches compile), showing that the RewardRepair neural model has captured
important information w.r.t. compilation.

###  5.3. RQ3: Impact of Semantic Training

Table 5 shows the results of the ablation study w.r.t semantic training. Per
the same protocol as RQ1, the considered metric is the number of correct
patches. The first row shows RewardRepair’s effectiveness with only syntactic
training, the second row shows RewardRepair with both syntactic and semantic
training. For example, RewardRepair with only syntactic training generates 42
correct patches on Defects4J(v1.2), and RewardRepair with semantic training
generates 45 correct patches. Overall, the addition of semantic training after
syntactic training does yield more correct patches on all considered
benchmarks. This shows that semantic training addresses the limits of
syntactic training, and the improvement is not tied to specific benchmarks.

Actions | Syntactic Training | Semantic Training
---|---|---
Add if conditions | 1852 | 2316
Method invocation | 1040 | 1352
Add return statement | 1123 | 1554
Ternary operator for null checking | 0 | 212
Table 6. Examples of repair action differences between syntactic training and
semantic training of RewardRepair.

⬇

\- this.dataset = dataset;

Buggy line

⬇

\+ .setDataSet(dataset)  // patch 1

\+ setDataset( );  // patch 2

Two non-compilable patches by syntactic training

⬇

\+ setDataset(dataset);

Correct compilable patch by semantic training

List of listings 5 Bug Chart-12 only fixed by semantic training

To better understand the effectiveness of semantic training, we manually
analyze those unique bugs that are only generated by RewardRepair by including
semantic training (and not generated with pure syntactic training). This leads
to an analysis of 14 800 patches (7 400 candidate patches by syntactic
training and 7 400 candidate patches by semantic training). We group those
patches by category and summarize the most interesting categories in Table 6.
The first column gives the type of repair action employed in the patch, and
the numbers in the second column and the third column indicate the numbers of
patches based on those repair actions from syntactic and semantic training
respectively. For example, the first row shows there are 1 852 patches
generated by syntactic training which add if/then statements, while semantic
training yields 2 316 patches using this construct. This means that the usage
of if/then statements is increased by 25.1% with semantic training. Notably,
the unique bugs that benefit from semantic training come from different
projects, showing that adding semantic training is beneficial in general.

Recall that syntactic training is done on more than 3 million training
samples, while semantic training is done on 123 training samples, which is
much less. This suggests that the improvement obtained with semantic training
does not come from the number of additional training points, but more from the
training process described in Section 3.5. RewardRepair’s loss function is
able to better optimize the neural network, improving the quality of generated
patches, with only a few training data points.

Finally, 5 discusses the case of Chart-12. The first line is the buggy line.
Next, in part (b), two patches after syntactic training are shown, they are
both close to the correct patch but none of them compile (extraneous dot in
the first patch and missing parameter in the second patch). Finally, part (c)
shows the correct patch by RewardRepair with semantic training, which is
identical to the developer patch. In this case, it suggests that the neural
model with semantic training has understood that leading dots before method
calls is not correct per the Java grammar, and that setDataset is a method
likely to take a parameter called dataset. A subtle character may lead to a
huge difference in program execution, but this knowledge is hard to be
obtained by syntactic training with cross-entropy loss.

Answer to RQ3: Our ablation study shows semantic training of RewardRepair
contributes to improving the overall effectiveness in terms of correctly fixed
bugs.

##  6\.  Discussion

###  6.1. Impact of Inference Beam Size

We investigate the impact of beam search size in the inference time, and our
experiment shows a bigger beam size indeed leads to more correct patches
generated, which confirms the study of Tufano et al. (Tufano et al., 2019b).
We provide the results of configuring beam size as 500 in our online appendix
repository (Experiment, 2022).

###  6.2. Threats to Validity

A threat to external validity relates to whether the performance of
RewardRepair generalizes to arbitrary programming languages. Per the standards
of the field, our approach has been tested in one language (Java) and the
evaluation is carried out on established benchmarks. In principle, our
approach can be applied to other programming languages and datasets. A threat
to internal validity relates to the hyper-parameter configuration we adopted.
To ensure replicability and extension, we make all the source code and results
publicly available for future research (Experiment, 2022).

##  7\. Related Work

###  7.1. Automatic Program Repair

A decade of research has generated a rich body of work on automatic program
repair (Monperrus, 2017; Gazzola et al., 2017). We have already discussed
neural repair approaches (Chen et al., 2019; Lutellier et al., 2020; Li et
al., 2020; Tufano et al., 2019a; Chakraborty et al., 2020; Tufano et al.,
2019b; Gupta et al., 2017; Zhu et al., 2021; Jiang et al., 2021) in Section 2.
These approaches only use the syntactic cross-entropy loss objective, which
poses a discrepancy between the training objective of generating compilable
correct patches and the loss criterion. The key novelty of RewardRepair is the
discriminative model to capture the compilation and execution knowledge during
model training and backpropagation.

We mentioned generate and validate (G&V) program repair approaches in RQ1
(Section 5.1). Other notable G&V systems include (Chen et al., 2017; Yuan and
Banzhaf, 2018; Martinez and Monperrus, 2016; Ghanbari et al., 2019). Moreover,
the third line of research is about synthesis-based repair (Xiong et al.,
2017; Xuan et al., 2016; Mechtaev et al., 2016; Le et al., 2017; Mechtaev et
al., 2015; Gao et al., 2019; Shariffdeen et al., 2021) which converts the
search problem to a satisfiability problem. All these approaches often work by
extracting a repair constraint typically via symbolic execution incorporated
with human knowledge for patch generation. On the contrary, RewardRepair
automatically learns such fix operators, language grammar and semantics from
the training corpus.

In the field of APR, the recent work of Jiang et al. (Jiang et al., 2021) is
the most closely related to ours, also focusing on the non-compilable patch
problem. They address this problem by employing a valid-identifier checker in
the inference stage to filter invalid tokens. However, many reasons could lead
to a non-compilable patch, and the presence of invalid identifiers is only one
of them. Our approach is fundamentally different: 1) RewardRepair works at
training time and not at inference time; 2) RewardRepair is based on the
actual compilation and test execution of training patches; 3) RewardRepair
encourages syntactic diversity while CURE encourages patches similar to the
buggy code (the length-control strategy in CURE).

###  7.2. Discriminators for Machine Learning on Code

Several works propose deep learning on code based on a discriminator (Harer et
al., 2018; Alhefdhi et al., 2020) where the discriminator provides a loss that
solves the discrepancy between the generated and real distributions of the
object under study, as pioneered by generative adversarial networks
(GAN)(Goodfellow et al., 2014). Harer et al. (Harer et al., 2018) propose an
adversarial learning approach to solve software vulnerabilities. Alhefdhi et
al. (Alhefdhi et al., 2020) leverage a similar GAN architecture to suggest
repairs that are as close as possible to human-written repairs. RewardRepair
shares the concepts of employing a traditional NMT model as a generator, and
of replacing the cross-entropy loss with the feedback from a discriminator.
The key difference between this related work and ours is that our
discriminator uses execution information, through compilation and test
execution.

###  7.3. Improving Backpropagation

Past research has improved the cross-entropy loss based on domain-specific
knowledge. In neural machine translation, Zhang et al. (Zhang et al., 2019)
show the limitation of considering cross-entropy loss and its tendency to
overcorrect synonymous words and phrases. To relieve the problem, further
research (Zhang et al., 2019; Lu et al., 2020) proposed to combine cross-
entropy loss and add translation evaluation at the sentence level. In object
detection, Ryou et al. (Ryou et al., 2019) proposed AnchorLoss to dynamically
rescale the cross-entropy based on prediction difficulty. Loss scaling is a
technique used in floating-point optimization, consisting of scaling up the
loss value up before the start of backpropagation (Micikevicius et al., 2018;
Zhao et al., 2019).

###  7.4. Training based on Execution

Recently, semantic information has been used in program synthesis tasks. Chen
et al. (Chen et al., 2019) and Gupta et al. (Gupta et al., 2020) propose
execution-guided synthesis leveraging the semantics of the language. These
approaches execute a partial program to obtain intermediate states to guide
program synthesis. Wang et al. (Wang et al., 2018b) use dynamic information
from execution to measure semantic redundancy between student programs. Mesbah
et al. (Mesbah et al., 2019) extract compiler diagnostic information as an
input source for repairing compilation errors. As in our work, these
approaches use execution information as an additional input for the considered
model. The key difference is that none of them employ the execution
information as a reward signal to update the neural network weights through
backpropagation.

Wang and colleagues (Wang et al., 2018a; Wang and Su, 2020) leverage the full
execution traces to learn neural semantic program embeddings. These related
works improve the code representation based on semantic information. Our
novelty is not on the representation, but on the training objective
improvement, which is not addressed in (Wang et al., 2018a; Wang and Su,
2020).

##  8\. Conclusion

We have presented a novel neural program repair model RewardRepair based on
compilation and test execution. The key idea is to employ a discriminative
model to provide a reward signal on the generated patches according to the
actual execution outcome. This signal modulates the purely syntactic cross-
entropy loss function in what we call semantic training. We have conducted an
extensive empirical evaluation, including a comprehensive experiment on the
widely used benchmark Defects4J, Bugs.jar and QuixBugs. Our results show that
it is possible to embed execution information in the backpropagation process
to improve neural program repair.

##  9\. Acknowledgments

We thank the anonymous reviewers for the insightful feedback. This work was
supported by the Wallenberg AI, Autonomous Systems and Software Program (WASP)
funded by the Knut and Alice Wallenberg Foundation. Some experiments were
performed on resources provided by the Swedish National Infrastructure for
Computing.

## References

  * (1)
  * Abreu et al. (2007) Rui Abreu, Peter Zoeteweij, and Arjan J.C. van Gemund. 2007\.  On the Accuracy of Spectrum-based Fault Localization. In _Testing: Academic and Industrial Conference Practice and Research Techniques - MUTATION (TAICPART-MUTATION 2007)_. 89–98.
  * Alhefdhi et al. (2020) Abdulaziz Alhefdhi, Hoa Khanh Dam, Xuan-Bach D. Le, and Aditya Ghose. 2020.  Adversarial Patch Generation for Automatic Program Repair.  arXiv:2012.11060
  * Bahdanau et al. (2014) Dzmitry Bahdanau, Kyunghyun Cho, and Y. Bengio. 2014\.  Neural Machine Translation by Jointly Learning to Align and Translate.  _ArXiv_ 1409 (09 2014).
  * Baudry et al. (2021) B. Baudry, Zimin Chen, K. Etemadi, Han Fu, Davide Ginelli, Steve Kommrusch, Matias Martinez, Monperrus Martin, Javier Ron, He Ye, and Zhongxing Yu. 2021.  A Software-Repair Robot Based on Continual Learning.  _IEEE Software_ 38 (2021), 28–35.
  * Chakraborty et al. (2020) S. Chakraborty, Y. Ding, M. Allamanis, and B. Ray. 2020\.  CODIT: Code Editing with Tree-Based Neural Models.  _IEEE Transactions on Software Engineering_ (2020).  https://doi.org/10.1109/TSE.2020.3020502
  * Chen et al. (2017) L. Chen, Y. Pei, and C. A. Furia. 2017.  Contract-based program repair without the contracts. In _2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE)_.
  * Chen et al. (2019) Xinyun Chen, Chang Liu, and Dawn Song. 2019.  Execution-Guided Neural Program Synthesis. In _International Conference on Learning Representations_.
  * Chen et al. (2019) Z. Chen, S. J. Kommrusch, M. Tufano, L. Pouchet, D. Poshyvanyk, and M. Monperrus. 2019\.  SEQUENCER: Sequence-to-Sequence Learning for End-to-End Program Repair.  _IEEE Transactions on Software Engineering_ (2019).
  * Chen and Monperrus (2018) Zimin Chen and Martin Monperrus. 2018.  _The CodRep Machine Learning on Source Code Competition_.  Technical Report 1807.03200. arXiv.  http://arxiv.org/pdf/1807.03200
  * Chen and Monperrus (2019) Zimin Chen and Martin Monperrus. 2019.  A Literature Study of Embeddings on Source Code.  arXiv:1904.03061
  * Durieux et al. (2019) Thomas Durieux, Fernanda Madeiral, Matias Martinez, and Rui Abreu. 2019.  Empirical Review of Java Program Repair Tools: A Large-scale Experiment on 2,141 Bugs and 23,551 Repair Attempts. In _Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering_. ACM, 302–313.  https://doi.org/10.1145/3338906.3338911
  * Experiment (2022) RewardRepair Experiment. 2022\.  Appendix Repository.  https://anonymous.4open.science/r/RewardRepair
  * Gao et al. (2019) Xiang Gao, Sergey Mechtaev, and Abhik Roychoudhury. 2019\.  Crash-Avoiding Program Repair. In _Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis_ (Beijing, China) _(ISSTA 2019)_. Association for Computing Machinery, New York, NY, USA, 8–18.  https://doi.org/10.1145/3293882.3330558
  * Gazzola et al. (2017) Luca Gazzola, Daniela Micucci, and Leonardo Mariani. 2017\.  Automatic Software Repair: A Survey.  _IEEE Transactions on Software Engineering_ (2017).
  * Ghanbari et al. (2019) Ali Ghanbari, Samuel Benton, and Lingming Zhang. 2019\.  Practical Program Repair via Bytecode Mutation. In _Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis_ (Beijing, China) _(ISSTA 2019)_. Association for Computing Machinery, New York, NY, USA, 19–30.  https://doi.org/10.1145/3293882.3330559
  * Ghazvininejad et al. (2020) Marjan Ghazvininejad, Vladimir Karpukhin, Luke Zettlemoyer, and Omer Levy. 2020.  Aligned Cross Entropy for Non-Autoregressive Machine Translation.  arXiv:2004.01655 [cs.CL]
  * Goodfellow et al. (2014) Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014\.  Generative Adversarial Nets _(NIPS’14)_. MIT Press, Cambridge, MA, USA, 2672–2680.
  * Gupta et al. (2020) Kavi Gupta, Peter Ebert Christensen, Xinyun Chen, and Dawn Song. 2020.  Synthesize, Execute and Debug: Learning to Repair for Neural Program Synthesis _(NIPS’20)_.
  * Gupta et al. (2017) Rahul Gupta, Soham Pal, Aditya Kanade, and Shirish Shevade. 2017\.  DeepFix: Fixing Common C Language Errors by Deep Learning. In _Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence_ (San Francisco, California, USA) _(AAAI’17)_. AAAI Press, 1345–1351.
  * Harer et al. (2018) Jacob A. Harer, Onur Ozdemir, Tomo Lazovich, Christopher P. Reale, Rebecca L. Russell, Louis Y. Kim, and Peter Chin. 2018.  Learning to Repair Software Vulnerabilities with Generative Adversarial Networks _(NIPS’18)_. Curran Associates Inc., Red Hook, NY, USA, 7944–7954.
  * Jiang et al. (2018) Jiajun Jiang, Yingfei Xiong, Hongyu Zhang, Qing Gao, and Xiangqun Chen. 2018.  Shaping Program Repair Space with Existing Patches and Similar Code _(ISSTA)_.
  * Jiang et al. (2021) Nan Jiang, Thibaud Lutellier, and Lin Tan. 2021\.  CURE: Code-Aware Neural Machine Translation for Automatic Program Repair. In _Proceedings of the ACM/IEEE 43rd International Conference on Software Engineering_.
  * Just et al. (2014) Rene Just, Darioush Jalali, and Michael D Ernst. 2014\.  Defects4J: A database of existing faults to enable controlled testing studies for Java programs. In _Proceedings of the 2014 International Symposium on Software Testing and Analysis_. ACM, 437–440.
  * Karampatsis et al. (2020) Rafael-Michael Karampatsis, Hlib Babii, Romain Robbes, Charles Sutton, and Andrea Janes. 2020\.  Big code!= big vocabulary: Open-vocabulary models for source code. In _2020 IEEE/ACM 42nd International Conference on Software Engineering (ICSE)_. IEEE, 1073–1085.
  * Kommrusch et al. (2021) Steve Kommrusch, Martin Monperrus, and Louis-Noël Pouchet. 2021\.  Self-Supervised Learning to Prove Equivalence Between Programs via Semantics-Preserving Rewrite Rules.  arXiv:2109.10476 [cs.LG]
  * Kudo and Richardson (2018) Taku Kudo and John Richardson. 2018.  SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing. In _Empirical Methods in Natural Language Processing(EMNLP)_.
  * Le et al. (2017) Xuan-Bach D. Le, Duc-Hiep Chu, David Lo, Claire Le Goues, and Willem Visser. 2017.  S3: Syntax- and Semantic-guided Repair Synthesis via Programming by Examples. In _Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering_ _(ESEC/FSE 2017)_.
  * Le et al. (2018) Xuan-Bach D. Le, Ferdian Thung, David Lo, and Claire Le Goues. 2018\.  Overfitting in Semantics-based Automated Program Repair. In _Proceedings of the 40th International Conference on Software Engineering_ (Gothenburg, Sweden) _(ICSE ’18)_. ACM, New York, NY, USA.
  * Le et al. (2019) Xuan-Bach D. Le, Lingfeng Bao, David Lo, Xin Xia, and Shanping Li. 2019\.  On Reliability of Patch Correctness Assessment. In _Proceedings of the 41st ACM/IEEE International Conference on Software Engineering_.
  * Le Goues et al. (2012) Claire Le Goues, ThanhVu Nguyen, Stephanie Forrest, and Westley Weimer. 2012.  GenProg: A generic method for automatic software repair.  _Software Engineering, IEEE Transactions on_ 38, 1 (2012), 54–72.  https://doi.org/10.1109/TSE.2011.104
  * Li et al. (2020) Yi Li, Shaohua Wang, and Tien N. Nguyen. 2020.  DLFix: Context-Based Code Transformation Learning for Automated Program Repair. In _Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering_ (Seoul, South Korea) _(ICSE ’20)_. Association for Computing Machinery, New York, NY, USA, 602–614.  https://doi.org/10.1145/3377811.3380345
  * Lin et al. (2017) Derrick Lin, James Koppel, Angela Chen, and Armando Solar-Lezama. 2017.  QuixBugs: A Multi-Lingual Program Repair Benchmark Set Based on the Quixey Challenge. In _Proceedings Companion of the 2017 ACM SIGPLAN International Conference on Systems, Programming, Languages, and Applications: Software for Humanity_ (Vancouver, BC, Canada) _(SPLASH Companion 2017)_. Association for Computing Machinery, New York, NY, USA, 55–56.  https://doi.org/10.1145/3135932.3135941
  * Liu et al. (2019) Kui Liu, Anil Koyuncu, Dongsun Kim, and Tegawendé F. Bissyandé. 2019.  TBar: Revisiting Template-based Automated Program Repair. In _Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis_. ACM, 31–42.  https://doi.org/10.1145/3293882.3330577
  * Liu et al. (2020) Kui Liu, Shangwen Wang, Anil Koyuncu, Kisub Kim, Tegawendé F. Bissyandé, Dongsun Kim, Peng Wu, Jacques Klein, Xiaoguang Mao, and Yves Le Traon. 2020\.  On the Efficiency of Test Suite Based Program Repair: A Systematic Assessment of 16 Automated Repair Systems for Java Programs. In _Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering_ (Seoul, South Korea) _(ICSE ’20)_. Association for Computing Machinery, New York, NY, USA, 615–627.  https://doi.org/10.1145/3377811.3380338
  * Lu et al. (2020) Wenjie Lu, Leiying Zhou, Gongshen Liu, and Quanhai Zhang. 2020\.  A Mixed Learning Objective for Neural Machine Translation. In _Chinese Computational Linguistics_ , Maosong Sun, Sujian Li, Yue Zhang, Yang Liu, Shizhu He, and Gaoqi Rao (Eds.). Springer International Publishing, Cham, 201–213.
  * Lutellier et al. (2020) Thibaud Lutellier, Hung Viet Pham, Lawrence Pang, Yitong Li, Moshi Wei, and Lin Tan. 2020.  CoCoNuT: Combining Context-Aware Neural Translation Models Using Ensemble for Program Repair _(ISSTA 2020)_.
  * Madeiral et al. (2019) Fernanda Madeiral, Simon Urli, Marcelo Maia, and Martin Monperrus. 2019.  Bears: An Extensible Java Bug Benchmark for Automatic Program Repair Studies. In _Proceedings of the 26th IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER ’19)_.  https://arxiv.org/abs/1901.06024
  * Martinez et al. (2017) Matias Martinez, Thomas Durieux, Romain Sommerard, Jifeng Xuan, and Martin Monperrus. 2017.  Automatic Repair of Real Bugs in Java: A Large-scale Experiment on the Defects4J Dataset.  _Empirical Software Engineering_ 22, 4 (2017), 1936–1964.  https://doi.org/10.1007/s10664-016-9470-4
  * Martinez and Monperrus (2016) Matias Martinez and Martin Monperrus. 2016.  ASTOR: A Program Repair Library for Java. In _Proceedings of ISSTA_.
  * Mechtaev et al. (2015) S. Mechtaev, J. Yi, and A. Roychoudhury. 2015.  DirectFix: Looking for Simple Program Repairs. In _2015 IEEE/ACM 37th IEEE International Conference on Software Engineering_ , Vol. 1. 448–458.  https://doi.org/10.1109/ICSE.2015.63
  * Mechtaev et al. (2016) Sergey Mechtaev, Jooyong Yi, and Abhik Roychoudhury. 2016\.  Angelix: Scalable Multiline Program Patch Synthesis via Symbolic Analysis. In _2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE)_.
  * Mesbah et al. (2019) Ali Mesbah, Andrew Rice, Emily Johnston, Nick Glorioso, and Edward Aftandilian. 2019.  DeepDelta: Learning to Repair Compilation Errors. In _Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering_ (Tallinn, Estonia) _(ESEC/FSE 2019)_. Association for Computing Machinery, New York, NY, USA, 925–936.  https://doi.org/10.1145/3338906.3340455
  * Micikevicius et al. (2018) Paulius Micikevicius, Sharan Narang, Jonah Alben, Gregory Diamos, Erich Elsen, David Garcia, Boris Ginsburg, Michael Houston, Oleksii Kuchaiev, Ganesh Venkatesh, and Hao Wu. 2018\.  Mixed Precision Training. In _International Conference on Learning Representations_.  https://openreview.net/forum?id=r1gs9JgRZ
  * Monperrus (2017) Martin Monperrus. 2017\.  Automatic Software Repair: a Bibliography.  _ACM Computing Surveys_ 51 (2017), 1–24.  https://doi.org/10.1145/3105906
  * Monperrus et al. (2021) Martin Monperrus, Matias Martinez, He Ye, Fernanda Madeiral, Thomas Durieux, and Zhongxing Yu. 2021\.  Megadiff: A Dataset of 600k Java Source Code Changes Categorized by Diff Size.  arXiv:2108.04631 [cs.SE]
  * Rabin et al. (2021) Md Rafiqul Islam Rabin, Nghi D.Q. Bui, Ke Wang, Yijun Yu, Lingxiao Jiang, and Mohammad Amin Alipour. 2021.  On the generalizability of Neural Program Models with respect to semantic-preserving program transformations.  _Information and Software Technology_ 135 (2021), 106552.
  * Raffel et al. (2020) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020.  Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer.  _Journal of Machine Learning Research_ 21, 140 (2020), 1–67.  http://jmlr.org/papers/v21/20-074.html
  * Riboira and Abreu (2010) André Riboira and Rui Abreu. 2010.  The GZoltar Project: A Graphical Debugger Interface _(TAIC PART’10)_. Springer-Verlag, Berlin, Heidelberg.
  * Ryou et al. (2019) Serim Ryou, Seong-Gyun Jeong, and P. Perona. 2019.  Anchor Loss: Modulating Loss Scale Based on Prediction Difficulty.  _2019 IEEE/CVF International Conference on Computer Vision (ICCV)_ (2019), 5991–6000.
  * Saha et al. (2018) Ripon K. Saha, Yingjun Lyu, Wing Lam, Hiroaki Yoshida, and Mukul R. Prasad. 2018.  Bugs.Jar: A Large-Scale, Diverse Dataset of Real-World Java Bugs _(MSR ’18)_. Association for Computing Machinery, New York, NY, USA, 10–13.  https://doi.org/10.1145/3196398.3196473
  * Saha et al. (2017) Ripon K. Saha, Yingjun Lyu, Hiroaki Yoshida, and Mukul R. Prasad. 2017.  ELIXIR: Effective Object Oriented Program Repair. In _Proceedings of the 32Nd IEEE/ACM International Conference on Automated Software Engineering_ (Urbana-Champaign, IL, USA) _(ASE 2017)_. IEEE Press, Piscataway, NJ, USA, 648–659.
  * Saha et al. (2019) Seemanta Saha, Ripon K. Saha, and Mukul R. Prasad. 2019\.  Harnessing Evolution for Multi-Hunk Program Repair. In _Proceedings of the 41st International Conference on Software Engineering_ (Montreal, Quebec, Canada) _(ICSE ’19)_. IEEE Press, 13–24.  https://doi.org/10.1109/ICSE.2019.00020
  * Shariffdeen et al. (2021) Ridwan Shariffdeen, Yannic Noller, Lars Grunske, and Abhik Roychoudhury. 2021.  Concolic Program Repair. In _42nd ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI)_.
  * Smith et al. (2015) Edward K. Smith, Earl T. Barr, Claire Le Goues, and Yuriy Brun. 2015.  Is the Cure Worse Than the Disease? Overfitting in Automated Program Repair. In _Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering_ _(ESEC/FSE 2015)_.
  * Tian et al. (2020) Haoye Tian, Kui Liu, Abdoul Kader Kaboré, Anil Koyuncu, Li Li, Jacques Klein, and Tegawendé F. Bissyandé. 2020\.  Evaluating Representation Learning of Code Changes for Predicting Patch Correctness in Program Repair. In _Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering_. IEEE, 981–992.  https://doi.org/10.1145/3324884.3416532
  * Tufano et al. (2019a) Michele Tufano, Jevgenija Pantiuchina, Cody Watson, Gabriele Bavota, and Denys Poshyvanyk. 2019a.  On Learning Meaningful Code Changes via Neural Machine Translation. In _Proceedings of the 41st International Conference on Software Engineering_ (Montreal, Quebec, Canada) _(ICSE ’19)_. IEEE Press, 25–36.  https://doi.org/10.1109/ICSE.2019.00021
  * Tufano et al. (2019b) Michele Tufano, Cody Watson, Gabriele Bavota, Massimiliano Di Penta, Martin White, and Denys Poshyvanyk. 2019b.  An Empirical Study on Learning Bug-Fixing Patches in the Wild via Neural Machine Translation.  _ACM Trans. Softw. Eng. Methodol._ 28, 4, Article 19 (Sept. 2019), 29 pages.  https://doi.org/10.1145/3340544
  * Vaswani et al. (2017a) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017a.  Attention is All you Need. In _Advances in Neural Information Processing Systems_ , Vol. 30.
  * Vaswani et al. (2017b) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, undefinedukasz Kaiser, and Illia Polosukhin. 2017b.  Attention is All You Need. In _Proceedings of the 31st International Conference on Neural Information Processing Systems_ (Long Beach, California, USA) _(NIPS’17)_. Curran Associates Inc., Red Hook, NY, USA, 6000–6010.
  * Wang et al. (2018a) Ke Wang, Rishabh Singh, and Zhendong Su. 2018a.  Dynamic Neural Program Embedding for Program Repair _(ICLR)_.
  * Wang et al. (2018b) Ke Wang, Rishabh Singh, and Zhendong Su. 2018b.  Search, Align, and Repair: Data-Driven Feedback Generation for Introductory Programming Exercises.  _SIGPLAN Not._ 53, 4 (June 2018), 481–495.  https://doi.org/10.1145/3296979.3192384
  * Wang and Su (2020) Ke Wang and Zhendong Su. 2020.  Blended, Precise Semantic Program Embeddings. In _Proceedings of the 41st ACM SIGPLAN Conference on Programming Language Design and Implementation_ (London, UK) _(PLDI 2020)_. Association for Computing Machinery, New York, NY, USA, 121–134.  https://doi.org/10.1145/3385412.3385999
  * Wang et al. (2020) Shangwen Wang, Ming Wen, Bo Lin, Hongjun Wu, Yihao Qin, Deqing Zou, Xiaoguang Mao, and Hai Jin. 2020\.  Automated Patch Correctness Assessment: How Far are We?. In _Proceedings of the 35th International Conference on Automated Software Engineering (ASE)_. ACM.
  * Wen et al. (2018) Ming Wen, Junjie Chen, Rongxin Wu, Dan Hao, and Shing-Chi Cheung. 2018.  Context-Aware Patch Generation for Better Automated Program Repair. In _Proceedings of the 40th International Conference on Software Engineering_ _(ICSE ’18)_.
  * Xin and Reiss (2019) Qi Xin and Steven Reiss. 2019.  Better Code Search and Reuse for Better Program Repair. In _2019 IEEE/ACM International Workshop on Genetic Improvement (GI)_. 10–17.  https://doi.org/10.1109/GI.2019.00012
  * Xiong et al. (2017) Yingfei Xiong, Jie Wang, Runfa Yan, Jiachen Zhang, Shi Han, Gang Huang, and Lu Zhang. 2017.  Precise Condition Synthesis for Program Repair. In _Proceedings of the 39th International Conference on Software Engineering_ (Buenos Aires, Argentina) _(ICSE ’17)_. IEEE Press, 416–426.  https://doi.org/10.1109/ICSE.2017.45
  * Xuan et al. (2016) Jifeng Xuan, Matias Martinez, Favio Demarco, Maxime Clément, Sebastian Lamelas, Thomas Durieux, Daniel Le Berre, and Martin Monperrus. 2016.  Nopol: Automatic Repair of Conditional Statement Bugs in Java Programs.  _IEEE Transactions on Software Engineering_ (2016).
  * Ye et al. (2021a) He Ye, Jian Gu, Matias Martinez, Thomas Durieux, and Martin Monperrus. 2021a.  Automated Classification of Overfitting Patches with Statically Extracted Code Features.  _IEEE Transactions on Software Engineering_ (2021).  https://doi.org/10.1109/tse.2021.3071750
  * Ye et al. (2021c) He Ye, Matias Martinez, Thomas Durieux, and Martin Monperrus. 2021c.  A comprehensive study of automatic program repair on the QuixBugs benchmark.  _Journal of Systems and Software_ 171 (2021), 110825.  https://doi.org/10.1016/j.jss.2020.110825
  * Ye et al. (2022) He Ye, Matias Martinez, Xiapu Luo, Tao Zhang, and Martin Monperrus. 2022.  SelfAPR: Self-supervised Program Repair with Test Execution Diagnostics. In _arXiv_.
  * Ye et al. (2021b) He Ye, Matias Martinez, and Martin Monperrus. 2021b.  Automated patch assessment for program repair at scale.  _Empirical Software Engineering_ 26, 2 (2021), 20\.  https://doi.org/10.1007/s10664-020-09920-w
  * Yu et al. (2017) Lantao Yu, Weinan Zhang, Jun Wang, and Yong Yu. 2017\.  SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient. In _Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence_ (San Francisco, California, USA) _(AAAI’17)_. AAAI Press, 2852–2858.
  * Yu et al. (2018) Zhongxing Yu, Matias Martinez, Benjamin Danglot, Thomas Durieux, and Martin Monperrus. 2018\.  Alleviating patch overfitting with automatic test generation: a study of feasibility and effectiveness for the Nopol repair system.  _Empirical Software Engineering_ (2018).
  * Yuan and Banzhaf (2018) Yuan Yuan and Wolfgang Banzhaf. 2018.  ARJA: Automated Repair of Java Programs via Multi-Objective Genetic Programming. In _IEEE Transactions on Software Engineering_.
  * Zhang et al. (2019) Wen Zhang, Yang Feng, Fandong Meng, Di You, and Qun Liu. 2019.  Bridging the Gap between Training and Inference for Neural Machine Translation. In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_. Association for Computational Linguistics, Florence, Italy, 4334–4343.  https://doi.org/10.18653/v1/P19-1426
  * Zhao et al. (2019) Ruizhe Zhao, Brian Vogel, and Tanvir Ahmed. 2019.  Adaptive Loss Scaling for Mixed Precision Training.  arXiv:1910.12385
  * Zhu et al. (2021) Qihao Zhu, Zeyu Sun, Yuan-an Xiao, Wenjie Zhang, Kang Yuan, Yingfei Xiong, and Lu Zhang. 2021.  A Syntax-Guided Edit Decoder for Neural Program Repair. In _Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering_ (Athens, Greece) _(ESEC/FSE 2021)_. Association for Computing Machinery, New York, NY, USA, 341–353.  https://doi.org/10.1145/3468264.3468544