# Improving the Learning of Code Review Successive Tasks with Cross-Task
Knowledge Distillation

Oussama Ben Sghaier  0000-0003-2737-0952 Université de MontréalCanada
<EMAIL_ADDRESS>and  Houari Sahraoui  0000-0001-6304-9926
Université de MontréalCanada<EMAIL_ADDRESS>
(2024; 2023-09-29; 2024-01-23)

###### Abstract.

Code review is a fundamental process in software development that plays a
pivotal role in ensuring code quality and reducing the likelihood of errors
and bugs. However, code review can be complex, subjective, and time-consuming.
_Quality estimation_ , _comment generation_ , and _code refinement_ constitute
the three key tasks of this process, and their automation has traditionally
been addressed separately in the literature using different approaches. In
particular, recent efforts have focused on fine-tuning pre-trained language
models to aid in code review tasks, with each task being considered in
isolation. We believe that these tasks are interconnected, and their fine-
tuning should consider this interconnection. In this paper, we introduce a
novel deep-learning architecture, named DISCOREV, which employs cross-task
knowledge distillation to address these tasks simultaneously. In our approach,
we utilize a cascade of models to enhance both _comment generation_ and _code
refinement_ models. The fine-tuning of the _comment generation_ model is
guided by the _code refinement_ model, while the fine-tuning of the _code
refinement_ model is guided by the _quality estimation_ model. We implement
this guidance using two strategies: a feedback-based learning objective and an
embedding alignment objective. We evaluate DISCOREV by comparing it to state-
of-the-art methods based on independent training and fine-tuning. Our results
show that our approach generates better review comments, as measured by the
_BLEU_ score, as well as more accurate _code refinement_ according to the
_CodeBLEU_ score.

Natural language processing, deep learning, knowledge distillation, code
review, code analysis, software maintenance.

††booktitle: Proceedings of the 32nd ACM Symposium on the Foundations of
Software Engineering (FSE ’24), July 15–19, 2024, Porto de Galinhas,
Brazil††copyright: acmlicensed††doi: 10.1145/3643775††journalyear:
2024††submissionid: fse24main-p1237-p††journalvolume: 1††journalnumber:
FSE††publicationmonth: 7

##  1\. Introduction

Code review is a pivotal process of the software development life cycle. Its
primary purpose is to find issues, suboptimal code, and identify bugs
(McIntosh et al., 2014, 2016), all while ensuring the overall quality of the
source code (Ackerman et al., 1989, 1984; Morales et al., 2015). This process
primarily involves manual examination by one or more developers of code
written by their peers (Fagan, 2002; Bavota and Russo, 2015). Code review
encompasses several successive tasks, with the three most important being:
_quality estimation_ of submitted code, description of potential issues (in
the form of review comments), and _code refinement_ to address these issues.

_Quality estimation_ consists of assessing the quality of newly written code,
with reviewers deciding whether to accept or reject a submitted pull request.
Issue identification centers on pinpointing defects or problems within the
source code, while review descriptions entail composing comments that not only
highlight specific changes that introduced issues but also suggest potential
solutions to address them (McIntosh et al., 2014, 2016). _Code refinement_ ,
on the other hand, denotes the correction phase wherein developers address the
identified issues to fix them, in accordance with reviewer comments (Bacchelli
and Bird, 2013).

The code review process is often regarded as arduous, time-consuming, and
intricate, especially in the context of large-scale projects (Eick et al.,
2001; Avgeriou et al., 2016). Moreover, it is a highly subjective process
influenced by various human and social factors (e.g., developers’ experience,
personal relationships, etc.). These factors can introduce biases in reviews,
potentially leading to inefficiencies and inconsistencies in the code review
process, ultimately impacting the quality and reliability of the codebase.

To address these challenges, there has been a growing interest in assistance
and automation approaches to code review (Li et al., 2022a; Tufano et al.,
2022; Tufan et al., 2021; Siow et al., 2020; Gupta and Sundaresan, 2018;
Hovemeyer and Pugh, 2004). One research area investigates the initial stages
of the software development process by leveraging static analysis to detect
potential issues (Wichmann et al., 1995). These tools, known as linters,
define sets of manual rules representing various issues and flag sections of
code that violate these predefined rules. However, the effectiveness of
linters is limited due to their reliance on manually defined rules, which must
be continually adapted to cover a range of issues. Additionally, software
issues are subject to change over time and can be influenced by multiple
factors such as software architecture, team culture, and project nature.
Consequently, the rigid nature of static analysis tools limits their
applicability and effectiveness in software projects (Bielik et al., 2017;
Sadowski et al., 2015).

Other approaches have employed similarity techniques (Siow et al., 2020; Gupta
and Sundaresan, 2018) to recommend relevant comments from a pre-defined
dataset of review comments based on similarities to code changes. While such
suggestions may prove useful, review comments are rarely generic and more
often context-specific.

With recent advances in deep learning and natural language processing, there
has been significant interest in using pre-trained language models to address
downstream tasks in software engineering. As a result, recent works (Tufano et
al., 2022; Tufan et al., 2021; Li et al., 2022a; Sghaier and Sahraoui, 2023)
have focused on using generative deep learning models to automate code review
tasks, resulting in significant improvements in code review _comment
generation_. Despite the utility and promising results of these approaches,
code review tasks have traditionally been considered separately, addressing
them independently despite their significant interdependencies.

In this paper, we explicitly consider the interconnections among the three
primary code review tasks: _quality estimation_ , _comment generation_ , and
_code refinement_. These tasks share knowledge, as the output of one task
serves as input to another. The comment generated by the comment generation
task feeds into the _code refinement_ task, while the revised code generated
by the latter is used in the _quality estimation_ task to determine whether
further review is necessary. In essence, _comment generation_ involves
describing an issue to guide the subsequent _code refinement_ phase. _Code
refinement_ , in turn, encompasses making the necessary code adjustments to
align with the review comment and pass the _quality estimation_ phase.

Building upon this concept, we introduce a novel deep-learning architecture
called DISCOREV (_cross-task knowledge DIStillation for COde REView_), which
incorporates two models jointly trained on two related tasks. We apply this
architecture to pairs of code review tasks, specifically _comment
generation/code refinement_ and _code refinement/quality estimation_. Our
proposed architecture is founded on cross-task knowledge distillation, where
one model benefits from the feedback of another to achieve improved
performance. DISCOREV builds upon recent advancements in deep learning for
code review, which have yielded promising results (Li et al., 2022a; Tufano et
al., 2022). In addition to leveraging pre-trained models, our approach
facilitates the joint modeling of code review tasks, enhancing overall
effectiveness. The output of the second model informs the first model,
providing feedback that gauges the relevance and informativeness of the
generated output. Through the joint training of these two models, we can
capture complex interactions, exchange knowledge and feedback, and ensure
consistency between the two tasks. Furthermore, we introduce an embedding
alignment objective to enforce closer semantic representations for the comment
and code.

In evaluating DISCOREV, we conducted experiments using the same dataset of
code reviews as employed in the literature (Li et al., 2022a). Our results
demonstrate that our architecture outperforms state-of-the-art approaches for
both _comment generation_ and _code refinement_ tasks, as measured by the
_BLEU_ and _CodeBLEU_ scores. Additionally, we investigated the impact of the
embedding alignment objective on model performance and found that enforcing
closer semantic representations between comments and code edits enhances the
_comment generation_ model’s performance, resulting in more accurate and
relevant comments.

The rest of this paper is structured as follows. Section 2 provides a concise
overview of the background. Section 3 details the different components of our
proposed approach. Section 4 describes the evaluation results and discusses
some threats to the validity of our results. Section 5 discusses the related
work. Finally, Section 6 provides concluding remarks and outlines some ideas
to improve our approach.

##  2\. Background

###  2.1. Code review

In a continuous integration setting, software developers utilize version
control systems, such as GitHub, to collaborate and oversee software versions.
As developers collaborate and persistently make changes to the codebase (i.e.,
write new code or modify existing code), the use of version control systems
enables development teams to efficiently manage and monitor the software
codebase across time (Shahin et al., 2017; Fowler and Foemmel, 2006).

In a continuous integration context, the conventional software development
workflow is as follows. A developer, who is working on implementing a new
feature, makes local changes to the codebase. Then, she commits and pushes her
changes to a shared repository to make them available to collaborators. She
can create a pull request to ask that her changes be merged into the main
branch. Some reviewers are assigned to the pull request to inspect these code
changes. Reviewers identify and locate issues that need to be solved by the
code author. The revised version of the code, submitted by developers, is
reviewed again. Once the changes are approved by reviewers, they are merged
with the main code version to be available to other collaborators. If
necessary, the project can be deployed so that changes become available to
end-users.

Code review is an essential task in the software development life cycle that
aims to preserve the high quality of the software’s codebase. It entails
regularly inspecting the source code written by fellow developers with the
objective of identifying bugs, potential issues, sub-optimal code fragments,
violations of code style, etc. It is mainly composed of three tasks: _quality
estimation_ , _code review comments_ , _code refinement_.

#### Quality estimation

The nomenclature _quality estimation_ is adopted herein, in harmony with the
work elucidated in (Li et al., 2022a), to maintain terminological consistency.
It is a pivotal step of the code review process that represents the assessment
of a pull request, ultimately culminating in the consequential decision of
either acceptance or rejection. During this process, the reviewer carefully
checks the code to ensure it follows the rules, works correctly, and won’t
cause problems in the project. _Quality estimation_ helps maintain the overall
quality of the software by only accepting changes that meet certain criteria.
If the pull request is accepted, the code changes are merged with the main
branch of the codebase. Otherwise, the reviewer needs to explain the problems
or the required improvements that the developer should consider. After making
the necessary changes by the developer, the new code changes go through the
process of _quality estimation_ again.

#### Code review comments

If the pull request is not accepted by the reviewer, she should describe the
issues or improvements that need to be addressed. Issues might be related to
security vulnerabilities, code style, code standards violations,
inconsistencies, bugs, sub-optimal code fragments, etc. Reviewers can also
suggest improvements that involve code complexity, software libraries, best
practices, refactoring, code quality, code optimization, documentation, etc.
The identified issues and improvements are expressed by means of comments. The
comments may be descriptive where the issues are detailed. However, comments
may also be actionable where the reviewers recommend solutions to resolve the
issues. The automation of this process helps to produce objective feedback
without being influenced by personal preferences, relationships, health
conditions, etc.

#### Code refinement

The comments that are produced by the previous step are fed as input to this
phase. The developers, and authors of the code changes, should consider the
feedback of the reviewers and perform the necessary code changes. Developers
should solve the issues described in the comments and apply suggested
solutions. New code changes are pushed again to the project repository and
should satisfy the reviewer’s comments. The refined version of the source code
goes through the _quality estimation_ process again to check whether the
comments are properly addressed. Proper _code refinement_ requires a good
understanding of the reviewer’s comments. That is, the reviews provided should
be clear, specific, and comprehensible.

###  2.2. Pre-trained language models

In recent years, the Natural Language Processing (NLP) domain has dramatically
evolved. State-of-the-art language models, that are based on the transformer
architecture (Vaswani et al., 2017), have shown outstanding performance in
solving natural language problems. Transformers have several advantages over
recurrent neural networks. They enable parallel computations which reduces
training time and captures long-range dependencies efficiently. _BERT_(Devlin
et al., 2018), _GPT-3_(Radford et al., 2019), _XLNet_ (Yang et al., 2019),
_RoBERTa_ (Liu et al., 2019) and _T5_ (Raffel et al., 2019) are examples of
general-purpose transformers that were pre-trained on tons of data to provide
general and high-level representations of text. _CodeBERT_ (Feng et al., 2020)
and _CodeT5_ (Wang et al., 2021) are examples of transformers that were pre-
trained on source code data. These pre-trained models could then be adapted
(i.e., fine-tuned) on downstream tasks using specific datasets (e.g., fine-
tune a language model to predict if a code fragment contains bugs).

Transformers are sequence-to-sequence models based on the attention mechanism
that takes into account the relationship between all the words in the sentence
and not only the contextual words (e.g., previous and next words). The
attention mechanism learns a weighting function indicating how much each
element contributes to the prediction (i.e., importance of each input element
in predicting the target). A transformer is composed of encoders and decoders.
The encoder transforms the input sequence into contextual embeddings. These
latter encode the meaning of each word of the input based on its context in
low-dimension and high-level representations. The decoder uses these
contextual representations to predict the output sequence.

_CodeT5_ (Wang et al., 2021) is a pre-trained encoder-decoder transformer for
code that is pre-trained on $8.35M$ functions in 8 programming languages.
There are several versions of _codeT5_. The small version has $60$ million
parameters and the base version has $220$ million parameters.

###  2.3. Knowledge distillation

Knowledge distillation is a popular technique that was first introduced by
Hinton et al. (Hinton et al., 2015) as a method to transfer knowledge from a
complex model, also known as the teacher, to a smaller and faster model,
called the student. The main goal of knowledge distillation is to transfer the
learned knowledge of the teacher model to the student model so that it can
achieve comparable or even better performance on a target task.

The process of knowledge distillation consists of training a teacher model on
a large dataset to accomplish a specific task. Then, the student model is
trained to mimic the behavior of the teacher model by minimizing the distance
between their output distributions on the same dataset, typically using the
Kullback-Leibler divergence (KL-divergence) as the distance metric (Joyce,
2011; Kim et al., 2021).

For instance, this concept is employed in model compression where the
knowledge is transferred from a large and complex neural network (i.e., the
teacher) to a smaller and simpler neural network (i.e., the student). The goal
of knowledge distillation is to make the student network learn the same
function as the teacher network but with fewer parameters.

Cross-task knowledge distillation is a recent extension of knowledge
distillation that enables the transfer of knowledge from one task to
another(Ye et al., 2020a; Yuan and Peng, 2019; Yang et al., 2022; Li et al.,
2022b). Instead of transferring knowledge from a complex model to a smaller
one for the same task, cross-task knowledge distillation transfers knowledge
from a model trained on one task, referred to as the source task, to a model
trained on a different task, referred to as the target task. This allows the
target model to improve its performance, even if the two tasks are not
directly related.

##  3\. Proposed approach

In this section, we introduce our proposed architecture DISCOREV in a broad
and abstract context, elucidating its relevance for software engineering
tasks. Subsequently, we will delve into its specific instantiation for code
review tasks.

###  3.1. A learning architecture using cross-task knowledge distillation

Most deep learning models are trained using their own feedback, which involves
comparing their predictions to the true labels. Nevertheless, some approaches
utilize feedback from another neural network to enhance training. In this way,
the training signal becomes more informative than a simple comparison with the
ground truth.

Jointly training two models can take the form of competition or collaboration.
Competitive models compete to outperform each other based on certain metrics.
Conversely, collaborative models cooperate through a feedback mechanism to
achieve better performance. For example, generative adversarial networks
(GANs) are employed for generative tasks where the objective is to create new
data resembling the original data distribution (Goodfellow et al., 2020). The
typical GAN architecture consists of two neural networks: a generator and a
discriminator. The generator learns to create synthetic data, while the
discriminator learns to distinguish between the generated and real data. GANs
are trained in an adversarial and competitive manner, with the generator
attempting to fool the discriminator, while the discriminator strives to
improve its ability to differentiate between real and fake data. Consequently,
both models improve with training over time. In contrast, knowledge
distillation is a collaborative learning technique that involves transferring
knowledge from a larger model to a smaller one, as elucidated in Section 2.3.

In software engineering, tasks are traditionally addressed independently.
Thus, to handle each task, a deep learning model is either fine-tuned or
trained from scratch. However, many software engineering tasks are
interconnected, as the output of one task may serve as the input for another.
Instead of approaching these tasks in isolation, we propose a paradigm shift
that capitalizes on the inherent interplay between them. The essence of our
proposition lies in the relationship between sequentially executed software
engineering tasks, which can be leveraged as a potent force for enhanced task
automation. By orchestrating these tasks harmoniously within a shared
framework, we aim to harness common knowledge and feedback between tasks. We
conjecture that this approach will lead to more robust, coherent, and
efficient trained models that inherently capture the dependencies and
intricacies among the constituent tasks.

We introduce DISCOREV, a novel deep learning architecture based on cross-task
knowledge distillation. DISCOREV is designed to address distinct but
interconnected tasks, particularly within the software development domain. Our
effort involves the cohesive modeling of these tasks, as abstractly
illustrated in Figure 1. Within this architecture, we have conceived two
interconnected models that operate symbiotically, informed by feedback
mechanisms to accomplish both tasks. Through joint training, our architectural
design facilitates an efficient learning process, allowing the first model to
be guided by the second model.

Figure 1. Overview of the proposed architecture DISCOREV.

More specifically, let’s consider the scenario where we are dealing with two
interdependent tasks $T_{1}$ and $T_{2}$, where the output of $T_{1}$ serves
as the input to $T_{2}$. In this context, $Model_{s}$ is the student model
responsible for automating $T_{1}$, while $Model_{t}$ is the teacher model
that addresses $T2$. The underlying intuition is that, in addition to
comparing predictions to the ground truth, the model for the subsequent task
provides guidance to the first model through feedback. This feedback serves as
an informative signal emanating from the second model, offering the first
model insight into the relevance of its output and its contribution to
fulfilling the second task.

$Model_{s}$ takes some inputs $IN_{1}$ and generates a prediction $PR_{1}$
which is then passed, along with other potential inputs $IN_{2}$ to the second
model $Model_{t}$. The teacher model subsequently generates its own prediction
$PR_{2}$. The loss function of $Model_{t}$ is defined based on the comparison
between its prediction $PR_{2}$ and the ground truth $GT_{2}$. The loss
function of the student model $Model_{s}$ consists of two components: a
regular loss function that compares its prediction $PR_{1}$ to the ground
truth $GT_{1}$, and the loss of the teacher model. Consequently, the
optimization objective for the student model extends beyond the singular
minimization of its own loss. It takes into account the loss incurred by the
teacher model, aiming to generate predictions that not only approximate the
ground truth but also facilitate the teacher model in achieving precision in
its predictions. Therefore, during backpropagation, the weights of the student
model are adjusted to optimize both tasks simultaneously.

As mentioned earlier, in the context of code review, tasks have traditionally
been treated separately in the literature (Li et al., 2022a; Tufano et al.,
2022; Tufan et al., 2021), with a predominant focus on the local scope, i.e.,
comparing the generated output to the true labels, despite their clear and
inherent interdependence. In our work, we postulate that code review tasks are
closely intertwined, as producing accurate code edits should depend on the
output reviews, and estimating the code quality relies on the generated code
edits. Therefore, we apply DISCOREV to enhance the performance of deep
learning models in automating code review tasks. In the following sections, we
illustrate this application by following the chain of dependency from teacher
to student. That is, we begin with _quality estimation_ guiding _code
refinement_ , and then proceed with _code refinement_ guiding _comment
generation_.

###  3.2. Code refinement guided by quality estimation using DISCOREV

In this part, we rely on the interdependence between the _code refinement_ and
_quality estimation_ tasks. During the _code refinement_ task, a developer
tries to make appropriate changes to the code base with the aim of getting the
pull request accepted in the next task, i.e.,  _quality estimation_. As shown
in Figure 2(a), we instantiated DISCOREV to address these tasks jointly.

We denote $\mathcal{D}_{1}$ our dataset that is composed of pairs $(c,d)$,
where $c$ is a code snippet and $d$ is a binary value that indicates whether
the code should be accepted or rejected. Given a review comment $r$ and a code
snippet $c$ from $\mathcal{D}_{1}$, the student model $\mathcal{M}_{s1}$
generates the code edits $c_{rp}$ (i.e., refined version of the code), an
approximation of $c_{r}$. The generated code edits $c_{rp}$ are fed, along
with the comment, to the teacher model $\mathcal{M}_{t1}$ that outputs the
probability $d_{p}$ of the code changes to be accepted or rejected. That is,
this value indicates the extent to which the code edits satisfy the review
comment given as input. The loss gradients of $\mathcal{M}_{t1}$
backpropagates to $\mathcal{M}_{s1}$. We consider this as feedback from the
teacher model that tells the relevance and conformance of the code edits to
the review comment.

(a) DISCOREV applied to the code refinement and quality estimation tasks

(b) DISCOREV applied to comment generation and code refinement tasks

Figure 2. Detailed architecture of DISCOREV in the context of code review.

$\mathcal{M}_{t1}$ and $\mathcal{M}_{s1}$ are defined, respectively, as
follows:

(1) $\left\\{\begin{array}[]{cc}\mathcal{M}_{s1}:&c,r\rightarrow c_{rp}\\\
\mathcal{M}_{t1}:&c_{rp},r\rightarrow d\end{array}\right.$

The loss of the teacher model $\mathcal{L}_{t1}$ is defined by:

(2) $\mathcal{L}_{t1}=\mathcal{L}_{BCE}(d,d_{p})$

It consists of comparing the truth value $d$ with the predicted value of
$\mathcal{M}_{t1}$ using the binary cross-entropy loss function
$\mathcal{L}_{BCE}$ that is defined with the following equation:

(3) $\mathcal{L}_{BCE}(y,\hat{y})=y\log(\hat{y})+(1-y)\log(1-\hat{y})$

where $\mathcal{L}_{BCE}$ represents the binary cross-entropy loss,
$y\in\\{0,1\\}$ is the truth value, and $\hat{y}$ is the probability predicted
by the classifier.

The loss of the student model $\mathcal{L}_{s2}$ is defined as a combination
of the cross-entropy loss and the teacher model loss.

(4)
$\mathcal{L}_{s1}=\alpha\mathcal{L}_{CE}(\mathcal{P}_{c_{r}},\mathcal{P}_{c_{rp}})+\beta\mathcal{L}_{t1}$

where $\alpha$ and $\beta$ are weights that indicate the importance of each
loss.

The cross-entropy loss compares the probability distributions of the predicted
code edits $\mathcal{P}_{c_{r}}$ and the real ones $\mathcal{P}_{c_{rp}}$. The
formula that defines the cross-entropy loss is given by:

(5) $\mathcal{L}_{CE}(p,q)=-\sum_{i=1}^{N}p_{i}\log(q_{i})$

where $\mathcal{L}_{CE}$ represents the cross-entropy loss, $N$ is the number
of tokens in the sentence, $p_{i}$ is the one-hot encoded vector representing
the true $i^{th}$ token, $q_{i}$ is the predicted probability vector of the
$i^{th}$ token. The vector is of size $|vocabulary|$ and indicates for each
word in the vocabulary its probability to be the $i^{th}$ token.

The teacher model provides feedback to the student model to indicate whether
the generated code edits conform to the review comment (i.e., accept or reject
the pull request). That is, the student model will try not to only generate
code edits that are close to the ground truth but have also a higher chance of
being accepted by the second model.

###  3.3. Comment generation guided by code refinement using DISCOREV

In code review, _comment generation_ and _code refinement_ tasks are strongly
related. The reviewer’s comment guides the _code refinement_ task and
describes what the developers should do to improve/fix their code. On the
other hand, developers refine their code by executing the instructions of the
review trying to satisfy the reviewer’s comments. We apply DISCOREV on these
two tasks to address them simultaneously.

Figure 2(b) illustrates the details of our unified architecture for the
_comment generation_ and _code refinement_ tasks. The student model generates
a review comment for the source code given as input. The generated review is
given, along with the code, as input to the teacher model to generate the
necessary code edits that fix the input code with respect to the review. The
loss gradients of the teacher model backpropagate to the first model. That is,
the teacher model gives feedback to the student model to indicate the
relevancy of the generated review and to what extent it enables the teacher
model to generate the appropriate code edits.

Formally, we denote $\mathcal{D}_{2}$ our dataset that is composed of triplets
$(c,r,c_{r})$ where $c$ is the initial version of the source code submitted by
the developer for review, $r$ is the review comment, and $c_{r}$ is a refined
version of $c$ with respect to $r$. $\mathcal{M}_{t2}$ and $\mathcal{M}_{s2}$
refer to the teacher and student models, respectively, such that:

(6) $\left\\{\begin{array}[]{cc}\mathcal{M}_{s2}:&c\rightarrow r_{p}\\\
\mathcal{M}_{t2}:&c,r_{p}\rightarrow c_{rp}\end{array}\right.$

The student model takes as input $c$ and generates the review $r_{p}$. $r_{p}$
is fed along with $c$ to the teacher model that generates $c_{rp}$ trying to
refine the source code. The loss of the teacher model $\mathcal{L}_{t2}$ is
defined as follows:

(7)
$\mathcal{L}_{t2}=\mathcal{L}_{CE}(\mathcal{P}_{c_{r}},\mathcal{P}_{c_{rp}})$

It consists of comparing the predicted code edits to the real ones using the
cross-entropy loss function $\mathcal{L}_{CE}$ that compares the probability
distribution $\mathcal{P}$ of the predicted output to the ground truth.

The loss of the student model $\mathcal{L}_{s2}$ is defined as follows:

(8)
$\mathcal{L}_{s2}=\alpha\mathcal{L}_{CE}(\mathcal{P}_{r},\mathcal{P}_{r_{p}})+\beta\mathcal{L}_{t2}$

where $\alpha$ and $\beta$ indicate the importance of each loss.

It is a combination of the common cross-entropy loss and the loss of the
teacher model. In short, the loss of $\mathcal{M}_{s2}$ compares the predicted
review to the real one, but also considers the feedback of the teacher. That
is, the student model will try not only to generate reviews that are close to
the ground truth but also relevant reviews that enable the second model to
predict the right code edits.

#### Embeddings alignment objective

Until now, we have been considering the loss of the _code refinement_ model
within one of the _comment generation_ model. However, we can capitalize on
another property of our problem to enhance interconnection-based learning. In
our approach, the models responsible for generating reviews and code edits are
closely intertwined, given that reviews often describe the necessary code
edits, and conversely, code edits typically represent the required alterations
to the code to align with the review comment. Therefore, a review comment and
its corresponding code edits inherently share similar meanings. This close
relationship suggests the existence of shared underlying patterns or features,
and consequently, similar representations, within the embeddings of reviews
and code edits.

In natural language processing, hidden representations refer to intermediate
representations of textual data learned by deep learning models during
training. These representations are not directly interpretable by humans, as
they usually consist of high-dimensional vectors capturing intricate patterns
and relationships among words and phrases in the input text. Within the
context of language models, each hidden layer may encode different aspects of
the input text, such as syntactic structure, semantic meaning, or discourse
coherence (López et al., 2022). The encoding of each hidden layer can vary in
several ways, including the level of abstraction, granularity of the
representations, amount of contextual information captured, or specialization
for specific tasks or domains. For instance, lower-level layers in a language
model may encode fundamental linguistic features like word-level embeddings or
syntactic parse trees, while higher-level layers may capture more abstract
semantic or discourse-level connections between words and sentences.
Typically, the last hidden layer of the encoder serves as an embedding for the
input text, encapsulating its meaning. This embedding is utilized by the
decoder to generate the desired output. Developing models capable of
harnessing effective representations can have implications for downstream
tasks such as text classification or language generation.

To enhance the performance of our models, we introduce an _embeddings
alignment objective_ aimed at aligning the embeddings of the review comment
and the code edits. The objective is to achieve close representations, as
review comments and code edits share the same meaning. Figure 2(b) illustrates
this objective. We use codeT5 for models $M_{s2}$ and $M_{t2}$, both of which
are based on the (encoder-decoder) transformer architecture. The
representations generated by the encoder of $M_{s2}$ are employed as
embeddings for the predicted review, while the embeddings generated by the
encoder of $M_{t2}$ are utilized for the predicted revised version of the
code.

Let us assume a predicted review $r_{p}$ and its corresponding predicted code
edits $c_{rp}$ whose vector representations are $E_{r}$ and $E_{c}$ of size
$n$. To align these embeddings, we define an objective $\mathcal{L}_{embed}$
that minimizes the distance between the two vectors defined as their mean
square error (MSE):

(9)
$\mathcal{L}_{embed}=MSE(E_{r},E_{c})=\frac{1}{n}\sum_{i=1}^{n}\left({E_{c}}_{i}-{E_{r}}_{i}\right)^{2}$

Our objective is to minimize the embedding alignment loss
$\mathcal{L}_{embed}$ to achieve better and shared representations. Thus, we
incorporate it in the teacher and student losses as follows:

(10)
$\mathcal{L^{\prime}}_{t_{2}}=\alpha_{1}\mathcal{L}_{t2}+\beta_{1}\mathcal{L}_{embed}$
(11)
$\mathcal{L^{\prime}}_{s2}=\alpha_{2}\mathcal{L}_{s2}+\beta_{2}\mathcal{L}_{embed}$

where $\alpha_{i}$ and $\beta_{i}$ are weights that denote the importance of
each loss. For the new student loss $\mathcal{L^{\prime}}_{s2}$, the teacher
loss is incorporated in $\mathcal{L}_{s2}$.

##  4\. Evaluation

We conducted a series of experiments to assess the performance of our proposed
architecture111The replicate package is provided in the supplementary material
of this paper.. In this section, we first outline the research questions we
sought to address, followed by a description of the experimental setup.
Subsequently, we present the results and discuss various potential threats
associated with our experiments.

###  4.1. Research questions

To assess DISCOREV, we formulate six research questions. The first question
pertains to the training of the _code refinement_ model, which is guided by
the _quality estimation_ model, the _quality estimation_ model being a
straightforward binary classifier. The remaining five questions focus on
training the _comment generation_ model, which is guided by the _code
refinement_ model. In this case, the teacher model is more complex, and the
relationship between the student and the teacher necessitates a deeper
analysis.

  * •

RQ1: Accuracy on code refinement
_How does the _code refinement_ model, guided by the _quality estimation_
model, perform compared to baseline models?_

  * •

RQ2: Accuracy on comment generation
_How does the _comment generation_ model, guided by the _code refinement_
model (and the embedding alignment), perform compared to baseline models?_

  * •

RQ3: Effect of the embedding alignment objective
_Considering that the inclusion of the embedding has an effect on the training
cost, do we have similar results without considering the embedding?_

  * •

RQ4: Training time of DISCOREV
_What is the additional time of the joint training compared to baseline
models?_

Taking into account the training time, we considered two additional questions
for the simple version of DISCOREV.

  * •

RQ5: Accuracy on comment generation for each programming language
_How does the same model perform, compared to baseline models, on _comment
generation_ with respect to each programming language?_

  * •

RQ6: Impact of the teacher pre-finetuning
_To what extent does the pre-finetuning phase of the teacher contribute to
enhancing the accuracy of the student model?_

###  4.2. Experimental setup

#### Dataset

We use a dataset of $176~{}616$ code reviews obtained from (Li et al., 2022a).
The dataset has code review entries for nine distinct programming languages.
Table 1 provides an overview of the dataset. The data is split into $85\%$ for
_Train_ , $7.5\%$ for _Validation_ , and $7.5\%$ for _Test_. The data contains
more than $1.3M$ lines of code. To have a fair and consistent comparison with
state-of-the-art works, we use the same data split as in (Li et al., 2022a).
We cleaned the dataset and we kept the necessary features as explained below.
The first resulting dataset is a set of triplets $(c,r,c_{r})$ where $c$ is
the initial version of the code, $r$ is the review comment, and $c_{r}$ is the
revised source code after addressing the comment. The second dataset is a set
of triplets $(c_{c},r,d)$ where $d$ is a binary value indicating whether the
candidate code changes $c_{c}$ should be accepted or rejected according to
comment $r$.

Table 1. Distribution of the dataset over programming languages | PHP | Ruby | C# | C | Java | Python | C++ | Go | JS | Total
---|---|---|---|---|---|---|---|---|---|---
Train | 7 979 | 5 730 | 15 630 | 3 077 | 31 288 | 30 648 | 13 274 | 30 369 | 12 411 | 150 409
Val | 973 | 504 | 717 | 543 | 2 177 | 2 834 | 1 362 | 2 865 | 1 128 | 13 103
Test | 1 032 | 479 | 738 | 488 | 2 206 | 2 900 | 1 308 | 2 889 | 1 064 | 13 104

#### Preprocessing

To properly train DISCOREV, the various inputs and outputs should be
represented conveniently.

The _comment generation_ model is fed with the code $c$ as input and generates
a review $r_{p}$. We clean the input code $c$ and use RoBERTa tokenizer (Liu
et al., 2019) to split it into tokens.

$c\Rightarrow c_{1}c_{2}\dots c_{n}$

The inputs are either truncated or padded with a special token
$~{}<\\!pad\\!>~{}$ to assert a fixed input length. We add two other special
tokens $~{}<\\!s\\!>~{}$ and $~{}<\\!/s\\!>~{}$ to denote the beginning and
the end of the input respectively. The resulting input has this
representation:

$c_{1}c_{2}\dots c_{n}\Rightarrow~{}<\\!s\\!>~{}c_{1}c_{2}\dots
c_{n}~{}<\\!pad\\!>~{}\dots~{}<\\!pad\\!>~{}~{}~{}<\\!/s\\!>~{}$

where $c_{i}$ is the $i^{th}$ code token.

The output $r_{p}$ is also preprocessed in the same way:

$r\Rightarrow~{}<\\!s\\!>~{}r_{1}r_{2}\dots
r_{n}~{}<\\!pad\\!>~{}..~{}<\\!pad\\!>~{}~{}~{}<\\!/s\\!>~{}$

where $r_{i}$ is the $i^{th}$ review token.

The _code refinement_ model takes as input the predicted review along with the
code and generates the revised version of the code. The inputs are also
cleaned, truncated or padded, and augmented with the special tokens. The
resulting input is represented as follows:

$(c,r)\Rightarrow~{}<\\!s\\!>~{}c_{1}c_{2}\dots
c_{n}~{}<\\!pad\\!>~{}\dots~{}<\\!pad\\!>~{}~{}~{}<\\!msg\\!>~{}r_{1}r_{2}\dots
r_{m}~{}<\\!pad\\!>~{}\dots~{}<\\!pad\\!>~{}~{}~{}<\\!/s\\!>~{}$

where $~{}<\\!msg\\!>~{}$ is a special token that separates the input code and
review. The output is preprocessed similarly as follows:

$c_{r}\Rightarrow~{}<\\!s\\!>~{}c_{r_{1}}c_{r_{2}}\dots
c_{r_{n}}~{}<\\!pad\\!>~{}..~{}<\\!pad\\!>~{}~{}~{}<\\!/s\\!>~{}$

#### Experiments

In _RQ1_ , we assessed our approach’s performance on _code refinement_.
Initially, we pre-finetuned the _quality estimation_ model to adapt it to the
downstream task. Subsequently, we performed joint training of DISCOREV on the
_code refinement_ and _quality estimation_ tasks, as depicted in Figure 2(a).
During this phase, the student model (i.e.,  _code refinement_ model
$\mathcal{M}_{s1}$) underwent fine-tuning using its own loss function
$\mathcal{L}_{s1}$ while receiving feedback from $\mathcal{M}_{t1}$ through
the loss $\mathcal{L}_{t1}$. For $\mathcal{M}_{t1}$, we compared its
prediction with $1$ to enforce the student model to generate code edits that
would be accepted by the teacher model. To prevent the teacher model from
learning to consistently predict $1$ simply to minimize its loss, we kept its
weights frozen. Consequently, the student model is responsible for minimizing
both losses (i.e., generating code edits that align with the expected changes
and ensuring the pull request was accepted). We assessed the performance of
the resulting _code refinement_ model on the test set and compared it to the
baseline.

For _RQ2_ , we similarly fine-tuned DISCOREV on the _code refinement_ and
_comment generation_ tasks. In the first phase, we fine-tuned
$\mathcal{M}_{t2}$ on _code refinement_ to equip the teacher model with prior
knowledge for providing informative and accurate feedback to the student
model. In the subsequent phase, we jointly fine-tuned models
$\mathcal{M}_{s2}$ and $\mathcal{M}_{t2}$, as depicted in Figure 2(b). For
each model, we extracted the output of the last hidden layer of the encoder as
embeddings. We evaluated our generated models on the test set and compared
their performance to that of state-of-the-art works.

For _RQ3_ , to assess the impact of the embeddings alignment objective
($\mathcal{L}_{embed}$), we omitted this component and re-ran the previous
experiment.

To answer _RQ4_ , we ensured that all models were trained using the same
resources (i.e., GPUs and memory). We recorded the time required for each
model to complete one epoch.

For a deeper investigation into our proposed architecture, we selected the
_comment generation_ and _code refinement tasks_ to conduct additional
experiments. In _RQ5_ , we evaluated the performance of DISCOREV across
different programming languages by using the separate datasets shown in Table
1.

In _RQ6_ , we explored the impact of the pre-finetuning phase through two
experiments. First, we pre-finetuned the teacher model on the _code
refinement_ task before fine-tuning DISCOREV. Second, we fine-tuned the
DISCOREV model without pre-finetuning the teacher model. We evaluated the
resulting models on the test set and compared the two alternatives.

#### Baseline models

We compare DISCOREV with state-of-the-art works: T5 (Tufan et al., 2021),
CodeT5 (Wang et al., 2021), and CodeReviewer (Li et al., 2022a). In (Tufan et
al., 2021), the authors fine-tuned T5 on _comment generation_ and _code
refinement_. We denote CodeT5 as the fine-tuned version of this model for
downstream tasks. In (Li et al., 2022a), the authors introduced _CodeReviewer_
, a codeT5 model that was pre-trained on some tasks related to code review,
and then fine-tuned on _comment generation_ , _code refinement_ , and _quality
estimation_ tasks.

#### Performance metrics

To assess the effectiveness of the models, we compute the Bilingual Evaluation
Understudy (BLEU) score (Papineni et al., 2002) for the generated comments and
CodeBLEU score (Ren et al., 2020) for the generated code. The BLEU score is a
widely used metric for evaluating the quality of text generated by deep
learning models. It measures the degree of similarity between the generated
text and a set of reference translations, in our case, the generated review
and the actual review. A higher BLEU score indicates better text quality with
respect to the references. We use the BLEU-4 variant, that computes the n-gram
overlap ($1\leq n\leq 4$) using this formula:

$BLEU\\!-\\!4=min(1,\frac{output\\_length}{reference\\_length})(\prod_{i=1}^{4}precision_{i})^{\frac{1}{4}}$

It computes the precision for n-grams of size 1 to 4 and adds a brief penalty
for short sentences. CodeBLEU is an adaptation of BLEU tailored for code. It
combines the n-gram match (computed by the BLEU score), weighted n-gram match,
syntactic AST match, and semantic data flow match. These metrics are also used
in (Tufano et al., 2022; Li et al., 2022a) which sets a consistent and fair
basis for comparison.

#### Implementation and parameters

To implement the proposed models and run the different experiments, we used
the PyTorch framework. We run the training on a machine with four GPUs _RTX
3090_ and $24GB$ of RAM per GPU. The models were trained using the Adam
optimizer with a learning rate of $10^{-5}$, and we used cross-entropy as a
loss function. The training process was performed on the training set for $30$
epochs, with a batch size of $16$. The validation set was used to monitor the
model’s performance and adjust the different hyperparameters accordingly.
Lastly, the test set was used in the final step to evaluate the performance of
the models. We equally set the parameters $\alpha$ and $\beta$ to
$\frac{1}{2}$.

###  4.3. Results

### Results for RQ1 - Accuracy on code refinement

Table 2 shows the results of this research question. In a first phase, we
fine-tuned our model on the task of code change _quality estimation_ to
accurately predict if a pull request should be accepted or rejected given the
input code changes. Within this preliminary phase, a good accuracy of $71.46$
was attained, which sets a good knowledge base for the _quality estimation_
model to guide the _code refinement_ model.

Then, we fine-tuned DISCOREV on both tasks simultaneously. As shown in Table
2, DISCOREV outperforms the baseline models on _code refinement_. It reaches
$85.49$ as CodeBLEU score compared to $82.61$ for CodeReviewer, the highest
score of the baselines. This confirms our earlier conjecture, suggesting that
simultaneously addressing code review tasks could improve the accuracy of the
learning.

Table 2. Performance of DISCOREV on code refinement compared to baseline models Model | Code refinement | Quality estimation
---|---|---
Pre-finetuning
DISCOREV |  | 71.46
Fine-tuning
T5 | 77.03 |
CodeT5 | 80.82 |
CodeReviewer | 82.61 | 73.89
DISCOREV | 85.49 | 71.46

### Results for RQ2 - Accuracy on comment generation

Table 3 reports the results obtained for the _comment generation_ task. In the
pre-finetuning, the _code refinement_ model achieved a codeBLEU score of
$81.79$. The produced model was used in the fine-tuning phase of DISCOREV to
guide, through feedback, the student model on the _comment generation_ task.
As illustrated in Table 3, DISCOREV outperforms the baseline models in the
_comment generation_ task, registering remarkable improvements in BLEU scores
of $67\%$, $52\%$, and $38\%$ over the T5, CodeT5, and CodeReviewer baselines,
respectively.

The discernible difference in accuracy for _comment generation_ shows the
effectiveness of our proposed feedback-based learning strategy when the pre-
trained _code refinement_ model provides valuable guidance for the _comment
generation_ task.

In our approach, we do not anticipate an enhancement in the accuracy of the
teacher model through joint training. This is primarily because the _code
refinement_ model is fine-tuned with synthetic reviews generated by the
_comment generation_ model, as opposed to actual reviews. These generated
reviews may potentially contain grammatical errors and lack precision.
Furthermore, the feedback process is unidirectional, meaning that the teacher
model does not receive direct feedback from the student model. Nevertheless,
it is noteworthy that the _code refinement_ model demonstrated a marginal
improvement in accuracy. We conjecture that this improvement is attributed to
the embedding alignment objective.

Table 3. Performance of DISCOREV on comment generation compared to baseline models Model | Comment generation | Code refinement
---|---|---
Pre-finetuning
DISCOREV |  | 81.79
Fine-tuning
T5 | 4.39 | 77.03
CodeT5 | 4.83 | 80.82
CodeReviewer | 5.32 | 82.61
DISCOREV | 7.33 | 82.84

### Results for RQ3 - Effect of the embedding alignment objective

Table 4 illustrates the impact of the embeddings alignment objective on the
performance of our proposed model DISCOREV compared to baseline models. As
shown in this table, omitting this objective results in a drop in the model’s
performance (i.e., the BLEU score drops from $7.33$ to $6.68$) for _comment
generation_. However, DISCOREV still has better results compared to the
baselines even without the embedding alignment objective. That is, a simple
feedback signal from the teacher model to the student model may be sufficient
to improve the model’s accuracy. However, incorporating more informative
feedback signals may improve further the model’s accuracy. This illustrates
the importance of both components as each one contributes to the results’
improvement. Since the revised code and the review share the same meaning,
converging to close representations resulted in boosting the performance of
DISCOREV on code review tasks.

The CodeBLEU score dropped slightly on the _code refinement_ task. This
confirms our conjecture of RQ2 that the observed improvement in the _code
refinement_ came from the embedding alignment.

These findings suggest that aligning the embeddings for the different inputs
that share similar semantics might be beneficial to the improvement of the
model’s performance on the downstream task.

Table 4. Influence of the embeddings alignment objective on the performance of DISCOREV for the comment generation task Model | Comment generation | Code refinement
---|---|---
Pre-finetuning
DISCOREV |  | 81.79
Fine-tuning
T5 | 4.39 | 77.03
CodeT5 | 4.83 | 80.82
CodeReviewer | 5.32 | 82.61
DISCOREV | 6.68 | 80.96

### Results for RQ4 - Training time of DISCOREV

To evaluate the performance of DISCOREV in comparison to baseline models, we
recorded the training time required for each model to complete a single epoch.
We employed four NVIDIA RTX-A5000 GPUs, each equipped with 24 gigabytes of
dedicated memory. Careful consideration was given to configuring batch sizes
to fully utilize these resources. Our aim was to assess the temporal
efficiency and, by extension, the computational performance of all four models
when trained for a single epoch using identical resources. The empirical
results are detailed in Table 5.

DISCOREV exhibited a substantial training time, consuming $411$ minutes, and
extending to $524$ minutes when the embeddings alignment component was
included, indicating relatively prolonged training durations. In stark
contrast, the other models demonstrated remarkable computational efficiency,
as they share the same architecture (i.e., the T5 architecture). While our
proposed architecture outperforms the baseline models in terms of accuracy, it
is worth noting that it demands more computational resources. However, it is
important to emphasize that this resource-intensive phase occurs only once
during training, posing no significant challenges in subsequent real-world
deployment and usage.

Table 5. Performance of DISCOREV, in terms of training time per 1 epoch, compared to baseline models Model | Time (minutes)
---|---
T5 | 176
CodeT5 | 179
CodeReviewer | 178
DISCOREV | 411
DISCOREV+ embeddings | 524

In the next research questions, we examine further our proposed architecture.
Given the evident resource-intensive nature of DISCOREV, we omit the embedding
alignment objective. Through this deliberate simplification, we would be able
to run more experiments and shed light on the inherent potential of our
architectural design thereby enabling a more nuanced evaluation of its
performance and applicability.

### Results for RQ5 - Accuracy on comment generation for each programming
language

Table 6 shows the results of the performance of DISCOREV model, compared to
_Code reviewer_ , on the _comment generation_ task for each programming
language. We perform the comparison with the baseline with the best accuracy,
i.e., , _CodeReviewer_.

DISCOREV outperforms _CodeReviewer_ on _comment generation_ for all the
programming languages. However, this advantage is variable from one
programming language to another. This depends on the quality and the amount of
data for each programming language. Also, software issues and best practices
are different from one programming language to another.

We notice that DISCOREV has a much better BLEU score for _PHP_ , and _GO_.
However, the BLEU score is lower for _C_ and _Java_. This might depend on the
usage and best practices for each programming language. While some programming
languages (e.g., Java, C) have several best practices, coding standards,
conventional issues, etc., developers may be more tolerant and lenient with
other languages (e.g., PHP). For instance, the Java community has a strong
focus on code quality and best practices. There are many established coding
standards and guidelines, such as the Java Code Conventions, which encourage
developers to write high-quality, and maintainable code. This could lead to
more strict code review processes to ensure that these standards are upheld.

Moreover, the performance variability could be related to the nature and
characteristics of the programming language. Some programming languages are
compiled, which means that the program will not be functional if it has
errors. which could lead to more stringent code review processes. Other
languages are interpreted, which means they are more tolerant as the program
will still be functional and errors may occur during run-time.

Table 6. BLEU of DISCOREV, compared to baseline models, on comment generation per programming language Language | PHP | Ruby | C# | C | Java | Python | C++ | Go | JavaScript
---|---|---|---|---|---|---|---|---|---
CodeReviewer | 8.10 | 5.49 | 5.92 | 5.16 | 4.03 | 4.36 | 5.71 | 6.05 | 5.24
DISCOREV | 9.55 | 5.90 | 6.14 | 5.28 | 5.37 | 5.90 | 6.10 | 7.37 | 6.06

### Results for RQ6 - Impact of the teacher pre-finetuning

In this research question, we investigated the impact of the teacher pre-
finetuning phase on the performance of DISCOREV. Table 7 illustrates the
influence of the pre-finetuning of the _code refinement_ model (i.e., teacher)
on the performance of DISCOREV for the _comment generation_ task.

For _comment generation_ , there is an improvement of $0.46$ on the BLEU score
when performing the pre-finetuning phase. This demonstrates the importance of
this phase as it enables the teacher model to acquire prior knowledge and have
better performance on _code refinement_. Thus, the teacher would provide the
student with more relevant feedback during the fine-tuning phase.
Consequently, the student model would be able to generate more accurate
comments that align with the reviewers’ comments.

Still, both experiments produce better results for _comment generation_
compared to the literature. This demonstrates the effectiveness of jointly
addressing code review tasks using cross-task knowledge distillation.

These findings suggest that addressing several related tasks simultaneously
using cross-task knowledge distillation might be beneficial even without
having a very effective teacher model. However, the pre-finetuning phase is
important as it allows the teacher model to provide more informative feedback
yielding better results for the student model.

Table 7. Impact of pre-finetuning the teacher (i.e., code refinement model) on the performance of DISCOREV.  Comment generation | Code refinement
---|---
Without pre-finetuning
6.22 | 71.54
With pre-finetuning
6.68 | 82.84

###  4.4. Threats to validity

The evaluation results have demonstrated the effectiveness of our proposed
architecture in addressing interconnected code review tasks. Nevertheless,
there are certain threats that may limit the validity of the presented
evaluation results.

A primary threat pertains to the nature of the data, specifically the reviews,
which may contain noise and potentially non-English or misspelled words. We
have mitigated this by employing codeT5, a state-of-the-art language model
that utilizes Byte-Pair Encoding, a subword-based tokenization algorithm. This
algorithm breaks unseen words into several frequently seen sub-words that can
be effectively processed by the model.

A second concern revolves around the data imbalance, where some programming
languages have fewer examples than others. This disparity may lead to varying
performance across programming languages. However, the use of a large pre-
trained language model allows us to circumvent this issue, as CodeT5 is a
large language model that has already been trained on extensive code
repositories. Consequently, fine-tuning this model for downstream tasks does
not require a large volume of data. Moreover, prior research has shown that
the use of multilingual training datasets can result in enhanced model
performance compared to monolingual datasets, particularly for low-resource
languages, in tasks such as neural machine translation and code translation
(Chiang et al., 2021; Zhu et al., 2022).

A final concern pertains to the selection of hyperparameters, which play a
critical role in determining the model’s performance. To ensure a fair
comparison with (Li et al., 2022a), we conducted a grid search solely on the
learning rate and batch size parameters. For other hyperparameters, we adopted
the same settings as those used in (Wang et al., 2021). It is important to
note that further improvements may be achievable through additional
hyperparameter tuning.

##  5\. Related work

Various techniques have been explored, in the literature, to support different
tasks of code review, and these techniques can be classified into three
categories: static analysis tools, similarity-based methods, and generative
approaches.

In an effort to tackle the early stages of the code review process, a research
direction has emerged that seeks to identify potential issues through the use
of static analysis tools such as Checkstyle(che, 2001), PMD(pmd, 2000), and
FindBugs(Hovemeyer and Pugh, 2004; fin, 2005). These tools, commonly referred
to as linters, establish a set of rules that denote different types of issues
related to security, code style, code quality, etc., and highlight parts of
the code that are non-compliant with the defined rules. Albeit the usefulness
of such tools, the manual adaptation required to encompass most of the issues
renders these tools rigid and less effective. Moreover, these tools need to be
continuously calibrated with respect to issues and best practices that are
variable over time and are affected by several factors such as software
architecture, team composition, project characteristics, domain, and the like.
This inflexibility of static analysis tools diminishes their suitability for
general adoption within software projects.

Other works employed similarity-based approaches to assist the code review
process. These approaches assume that, in the context of code review,
analogous situations may be encountered implying that similar issues may
arise. Consequently, historical knowledge can be exploited to hasten the code
review process by resolving recurring issues. Gupta et al. (Gupta and
Sundaresan, 2018) introduced DeepCodeReviewer (DCR), an LSTM-based model that
is trained using positive and negative examples of (code, review) pairs. Given
a new code snippet, a subset of candidate reviews is selected, from a
predefined set of reviews, based on code similarity. Subsequently, DCR
predicts a relevance score for each review with respect to the input code
snippet and suggests reviews exhibiting high relevance scores. Another work,
proposed in (Siow et al., 2020), introduced a more sophisticated approach
based on multi-level embedding to learn the relevancy between code and
reviews. This approach uses word-level and character-level embeddings to
achieve a better representation of the semantics provided by code and reviews.
These approaches have poor performance for unique code snippets and recommend
irrelevant reviews in most cases.

The last category, i.e., generative approaches, attempted to use generative
deep learning techniques to recommend reviews or code edits. Tufano et al.
(Tufan et al., 2021) proposed an approach that partially automates code
corrections before and during code review. This approach is composed of two
main components: a 1-encoder transformer that recommends additional changes to
the developer before submitting her code and a 2-encoder transformer that
suggests the necessary code edits to satisfy the reviewer comments (Tufan et
al., 2021). The authors improved their approach, in (Tufano et al., 2022),
using T5, a pre-trained text-to-text transfer transformer (Raffel et al.,
2019). To provide the model with prior knowledge of the downstream tasks, the
authors pre-trained the T5 model on Java and technical English datasets. Li et
al. (Li et al., 2022a) pre-trained CodeT5 on four tasks, tailored specifically
for the code review scenario, using a large-scale multilingual dataset of code
reviews to better understand code differences and reviews. Then, the output
model was fine-tuned on three downstream tasks: quality estimation (i.e.,
accept/reject a pull request), review generation (i.e., generate review
comment), and code refinement (i.e., recommend code edits to satisfy the
reviewer comment).

Despite their tight relationship, these approaches addressed the different
tasks of code review separately. In contrast, our approach employs code edits
as feedback for the code review generation process. To achieve this, we rely
on knowledge distillation introduced in (Hinton et al., 2015), where a model
endowed with a greater number of parameters is employed to assist a smaller
model in learning to produce decisions of comparable quality to the larger
model, on the same task. More recently, cross-task distillation has been used
in computer vision (Yuan and Peng, 2020; Li et al., 2022c; Ye et al., 2020b),
allowing models to leverage knowledge gained from one task to improve
performance on another related task. Yang et al. (Yang et al., 2022) proposed
a cross-task knowledge distillation framework designed for multi-task
recommendation. The framework seeks to utilize the prediction results of one
task as supervised signals to instruct another task. The paper focuses on
multi-task learning for predicting different user feedback (e.g. click, like,
purchase, lookthrough).

##  6\. Conclusion

We introduce DISCOREV, a novel deep learning architecture that leverages
cross-task knowledge distillation to simultaneously tackle two interconnected
tasks. This architecture comprises two models that undergo joint fine-tuning,
guided by the aim of accomplishing both tasks. We have employed this framework
to enhance the performance of deep learning models in automating code review
tasks. The guidance within this architecture is implemented through two
strategies: a feedback-based learning objective and an embedding alignment
objective.

We evaluated DISCOREV , with a dataset of code reviews, and compared it to
state-of-the-art approaches. The results clearly demonstrate that DISCOREV
outperforms baseline models in both code review tasks: _comment generation_
and _code refinement_. Furthermore, we investigated the impact of the
embedding alignment objective and teacher pre-finetuning on DISCOREV’s
performance. Our findings illustrate that embedding alignment leads to further
performance enhancements, as the code edits and reviews share similar
meanings. Additionally, providing the teacher model with prior knowledge
results in a more efficient feedback mechanism. These outcomes suggest that
concurrently addressing multiple tasks can be advantageous, enabling the
sharing of knowledge and facilitating the development of collaborative models
that not only learn from ground truth but also incorporate feedback from other
activities.

In our future work, we intend to design an alternative architecture with
bidirectional feedback, wherein both models collaborate to achieve superior
performance. Additionally, we plan to apply this generic framework to address
other tasks with correlated activities. Finally, we aim to integrate our model
as a bot within IDEs, allowing us to conduct user studies to assess its
practical impact and usefulness from a developer’s perspective.

## References

  * (1)
  * pmd (2000) 2000\.  PMD.  https://pmd.github.io/.
  * che (2001) 2001\.  Checkstyle.  https://checkstyle.org/.
  * fin (2005) 2005\.  FindBugs.  https://findbugs.sourceforge.net/.
  * Ackerman et al. (1989) A. Frank Ackerman, Lynne S. Buchwald, and Frank H. Lewski. 1989.  Software inspections: an effective verification process.  _IEEE software_ 6, 3 (1989), 31–36.
  * Ackerman et al. (1984) A Frank Ackerman, Priscilla J Fowler, and Robert G Ebenau. 1984.  Software inspections and the industrial production of software. In _Proc. of a symposium on Software validation: inspection-testing-verification-alternatives_. 13–40.
  * Avgeriou et al. (2016) Paris Avgeriou, Philippe Kruchten, Ipek Ozkaya, and Carolyn Seaman. 2016.  Managing technical debt in software engineering (Dagstuhl seminar 16162). In _Dagstuhl Reports_ , Vol. 6. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik.
  * Bacchelli and Bird (2013) Alberto Bacchelli and Christian Bird. 2013.  Expectations, outcomes, and challenges of modern code review. In _2013 35th International Conference on Software Engineering (ICSE)_. 712–721.  https://doi.org/10.1109/ICSE.2013.6606617
  * Bavota and Russo (2015) Gabriele Bavota and Barbara Russo. 2015.  Four eyes are better than two: On the impact of code reviews on software quality. In _2015 IEEE International Conference on Software Maintenance and Evolution (ICSME)_. IEEE, 81–90.
  * Bielik et al. (2017) Pavol Bielik, Veselin Raychev, and Martin Vechev. 2017.  Learning a static analyzer from data. In _International Conference on Computer Aided Verification_. Springer, 233–253.
  * Chiang et al. (2021) Ting-Rui Chiang, Yi-Pei Chen, Yi-Ting Yeh, and Graham Neubig. 2021.  Breaking down multilingual machine translation.  _arXiv preprint arXiv:2110.08130_ (2021).
  * Devlin et al. (2018) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018.  BERT: Pre-training of deep bidirectional transformers for language understanding.  _arXiv preprint arXiv:1810.04805_ (2018).
  * Eick et al. (2001) Stephen G Eick, Todd L Graves, Alan F Karr, J Steve Marron, and Audris Mockus. 2001.  Does code decay? assessing the evidence from change management data.  _IEEE Transactions on Software Engineering_ 27, 1 (2001), 1–12.
  * Fagan (2002) Michael Fagan. 2002.  Design and code inspections to reduce errors in program development.  In _Software pioneers_. Springer, 575–607.
  * Feng et al. (2020) Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, et al. 2020\.  CodeBERT: A pre-trained model for programming and natural languages.  _arXiv preprint arXiv:2002.08155_ (2020).
  * Fowler and Foemmel (2006) Martin Fowler and Matthew Foemmel. 2006.  Continuous integration.  https://martinfowler.com/articles/continuousIntegration.html.
  * Goodfellow et al. (2020) Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2020.  Generative adversarial networks.  _Commun. ACM_ 63, 11 (2020), 139–144.
  * Gupta and Sundaresan (2018) Anshul Gupta and Neel Sundaresan. 2018.  Intelligent code reviews using deep learning. In _Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’18) Deep Learning Day_.
  * Hinton et al. (2015) Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015.  Distilling the knowledge in a neural network.  _arXiv preprint arXiv:1503.02531_ (2015).
  * Hovemeyer and Pugh (2004) David Hovemeyer and William Pugh. 2004.  Finding bugs is easy.  _Acm sigplan notices_ 39, 12 (2004), 92–106.
  * Joyce (2011) James M Joyce. 2011.  Kullback-leibler divergence.  In _International encyclopedia of statistical science_. Springer, 720–722.
  * Kim et al. (2021) Taehyeon Kim, Jaehoon Oh, NakYil Kim, Sangwook Cho, and Se-Young Yun. 2021.  Comparing kullback-leibler divergence and mean squared error loss in knowledge distillation.  _arXiv preprint arXiv:2105.08919_ (2021).
  * Li et al. (2022b) Deng Li, Aming Wu, Yahong Han, and Qi Tian. 2022b.  Prototype-guided Cross-task Knowledge Distillation for Large-scale Models.  _arXiv preprint arXiv:2212.13180_ (2022).
  * Li et al. (2022c) Deng Li, Aming Wu, Yahong Han, and Qi Tian. 2022c.  Prototype-guided Cross-task Knowledge Distillation for Large-scale Models.  arXiv:2212.13180 [cs.CV]
  * Li et al. (2022a) Zhiyu Li, Shuai Lu, Daya Guo, Nan Duan, Shailesh Jannu, Grant Jenks, Deep Majumder, Jared Green, Alexey Svyatkovskiy, Shengyu Fu, et al. 2022a.  Automating code review activities by large-scale pre-training. In _Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering_. 1035–1047.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  RoBERTa: A robustly optimized BERT pretraining approach.  _arXiv preprint arXiv:1907.11692_ (2019).
  * López et al. (2022) José Antonio Hernández López, Martin Weyssow, Jesús Sánchez Cuadrado, and Houari Sahraoui. 2022.  AST-Probe: Recovering abstract syntax trees from hidden representations of pre-trained language models.  _arXiv preprint arXiv:2206.11719_ (2022).
  * McIntosh et al. (2014) Shane McIntosh, Yasutaka Kamei, Bram Adams, and Ahmed E Hassan. 2014.  The impact of code review coverage and code review participation on software quality: A case study of the Qt, VTK, and ITK projects. In _Proceedings of the 11th working conference on mining software repositories_. 192–201.
  * McIntosh et al. (2016) Shane McIntosh, Yasutaka Kamei, Bram Adams, and Ahmed E Hassan. 2016.  An empirical study of the impact of modern code review practices on software quality.  _Empirical Software Engineering_ 21, 5 (2016), 2146–2189.
  * Morales et al. (2015) Rodrigo Morales, Shane McIntosh, and Foutse Khomh. 2015.  Do code review practices impact design quality? A case study of the Qt, VTK, and ITK projects. In _2015 IEEE 22nd international conference on software analysis, evolution, and reengineering (SANER)_. IEEE, 171–180.
  * Papineni et al. (2002) Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002.  Bleu: a method for automatic evaluation of machine translation. In _Proceedings of the 40th annual meeting of the Association for Computational Linguistics_. 311–318.
  * Radford et al. (2019) Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019\.  Language models are unsupervised multitask learners.  _OpenAI blog_ 1, 8 (2019), 9.
  * Raffel et al. (2019) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2019.  Exploring the limits of transfer learning with a unified text-to-text transformer.  _arXiv preprint arXiv:1910.10683_ (2019).
  * Ren et al. (2020) Shuo Ren, Daya Guo, Shuai Lu, Long Zhou, Shujie Liu, Duyu Tang, Neel Sundaresan, Ming Zhou, Ambrosio Blanco, and Shuai Ma. 2020.  Codebleu: a method for automatic evaluation of code synthesis.  _arXiv preprint arXiv:2009.10297_ (2020).
  * Sadowski et al. (2015) Caitlin Sadowski, Jeffrey Van Gogh, Ciera Jaspan, Emma Soderberg, and Collin Winter. 2015.  Tricorder: Building a program analysis ecosystem. In _2015 IEEE/ACM 37th IEEE International Conference on Software Engineering_ , Vol. 1. IEEE, 598–608.
  * Sghaier and Sahraoui (2023) Oussama Ben Sghaier and Houari Sahraoui. 2023.  A Multi-Step Learning Approach to Assist Code Review. In _2023 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER)_. IEEE.
  * Shahin et al. (2017) Mojtaba Shahin, Muhammad Ali Babar, and Liming Zhu. 2017.  Continuous integration, delivery and deployment: a systematic review on approaches, tools, challenges and practices.  _IEEE Access_ 5 (2017), 3909–3943.
  * Siow et al. (2020) Jing Kai Siow, Cuiyun Gao, Lingling Fan, Sen Chen, and Yang Liu. 2020.  Core: Automating review recommendation for code changes. In _2020 IEEE 27th International Conference on Software Analysis, Evolution and Reengineering (SANER)_. IEEE, 284–295.
  * Tufan et al. (2021) Rosalia Tufan, Luca Pascarella, Michele Tufanoy, Denys Poshyvanykz, and Gabriele Bavota. 2021.  Towards automating code review activities. In _2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE)_. 163–174.
  * Tufano et al. (2022) Rosalia Tufano, Simone Masiero, Antonio Mastropaolo, Luca Pascarella, Denys Poshyvanyk, and Gabriele Bavota. 2022.  Using pre-trained models to boost code review automation.  _arXiv preprint arXiv:2201.06850_ (2022).
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  _Advances in neural information processing systems_ 30 (2017).
  * Wang et al. (2021) Yue Wang, Weishi Wang, Shafiq Joty, and Steven CH Hoi. 2021.  Codet5: Identifier-aware unified pre-trained encoder-decoder models for code understanding and generation.  _arXiv preprint arXiv:2109.00859_ (2021).
  * Wichmann et al. (1995) Brian A Wichmann, AA Canning, DL Clutterbuck, LA Winsborrow, NJ Ward, and D William R Marsh. 1995.  Industrial perspective on static analysis.  _Software Engineering Journal_ 10, 2 (1995), 69–75.
  * Yang et al. (2022) Chenxiao Yang, Junwei Pan, Xiaofeng Gao, Tingyu Jiang, Dapeng Liu, and Guihai Chen. 2022.  Cross-task knowledge distillation in multi-task recommendation. In _Proceedings of the AAAI Conference on Artificial Intelligence_ , Vol. 36. 4318–4326.
  * Yang et al. (2019) Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019.  XLNet: Generalized autoregressive pretraining for language understanding.  _Advances in neural information processing systems_ 32 (2019).
  * Ye et al. (2020a) Han-Jia Ye, Su Lu, and De-Chuan Zhan. 2020a.  Distilling cross-task knowledge via relationship matching. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 12396–12405.
  * Ye et al. (2020b) Han-Jia Ye, Su Lu, and De-Chuan Zhan. 2020b.  Distilling Cross-Task Knowledge via Relationship Matching. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_.
  * Yuan and Peng (2019) Mingkuan Yuan and Yuxin Peng. 2019.  CKD: Cross-task knowledge distillation for text-to-image synthesis.  _IEEE Transactions on Multimedia_ 22, 8 (2019), 1955–1968.
  * Yuan and Peng (2020) Mingkuan Yuan and Yuxin Peng. 2020.  CKD: Cross-Task Knowledge Distillation for Text-to-Image Synthesis.  _IEEE Transactions on Multimedia_ 22, 8 (2020), 1955–1968.  https://doi.org/10.1109/TMM.2019.2951463
  * Zhu et al. (2022) Ming Zhu, Karthik Suresh, and Chandan K Reddy. 2022.  Multilingual code snippets training for program translation. In _Proceedings of the AAAI Conference on Artificial Intelligence_ , Vol. 36. 11783–11790.