# ELECTRAMed: a new pre-trained language representation model for biomedical
NLP

Giacomo Miolo , Giulio Mantoan∗, Carlotta Orsenigo
Department of Management, Economics and Industrial Engineering
Politecnico di Milano, Italy
<EMAIL_ADDRESS>
Corresponding authors.

###### Abstract

The overwhelming amount of biomedical scientific texts calls for the
development of effective language models able to tackle a wide range of
biomedical natural language processing (NLP) tasks. The most recent dominant
approaches are domain-specific models, initialized with general-domain textual
data and then trained on a variety of scientific corpora. However, it has been
observed that for specialized domains in which large corpora exist, training a
model from scratch with just in-domain knowledge may yield better results.
Moreover, the increasing focus on the compute costs for pre-training recently
led to the design of more efficient architectures, such as ELECTRA. In this
paper, we propose a pre-trained domain-specific language model, called
ELECTRAMed, suited for the biomedical field. The novel approach inherits the
learning framework of the general-domain ELECTRA architecture, as well as its
computational advantages. Experiments performed on benchmark datasets for
several biomedical NLP tasks support the usefulness of ELECTRAMed, which sets
the novel state-of-the-art result on the BC5CDR corpus for named entity
recognition, and provides the best outcome in 2 over the 5 runs of the 7th
BioASQ-factoid Challange for the question answering task.

_K_ eywords Pre-trained language models $\cdot$ ELECTRA $\cdot$ Biomedical NLP

##  1 Introduction

The immense body of biomedical scientific texts, which steadily grows at an
exponential rate, makes it imperative to develop effective machine learning
methods able to automatically extract the rich knowledge therein contained,
that can be used to address several biomedical natural language processing
(NLP) tasks.

Prominent NLP advancements in recent years have been mostly driven by the use
of deep neural models, which require large corpora of annotated training data.
Compared to the general domain, however, the collection of such data in the
biomedical field is difficult and expensive, since it necessarily involves
domain experts for accurate data labelling. For this reason, semi-supervised
pre-trained language models, such as ELMo [1] and BERT [2], were developed and
successfully applied in a wide range of NLP tasks.

ELMo and BERT leverage contextualized word embeddings for which the
representation of a word depends on the context where it is used and,
therefore, is a function of the entire input sequence. ELMo exploits a deep
bidirectional language model pre-trained on a large corpus of texts. It was
proposed to address both the syntactic and semantic complexities and
ambiguities of words and has been proven to achieve notable results in a
variety of NLP problems [1]. BERT [2] resorts to the transformer architecture
to pre-train bidirectional language representations, instead of relying on
recurrent neural networks. By embracing attention mechanisms, BERT showed to
distinguish the sense of words at a very fine level and to grasp many of their
syntactic and semantic properties. This ability made BERT the state-of-the-art
contextualized word representation model for the most challenging natural
language understanding problems.

While ELMo and BERT architectures pre-trained on general-domain corpora are
well-established top performers for general NLP tasks, they might yield poor
results in case of scientific or specific domains, since the corpora used for
pre-training, such as news articles and Wikipedia [3], might not include the
same terminology adopted in the in-domain tasks. For specialized contexts past
studies showed that general-domain language models can largely benefit from
the use of in-domain textual data [4]. As a consequence, recent models for
biomedical NLP relied on adapted versions of general-domain approaches. Among
these, two of the most noteworthy and successful examples are represented by
BioBERT [5] and BlueBERT [4], which are domain-specific language models
initialized with the general-domain BERT, and then pre-trained on a wide range
of biomedical and scientific corpora. In principle, these last methods rely on
the assumption that initializing the pre-training from general-domain models
might improve the overall performance for domain-specific purposes. However,
it has been recently observed that for domains in which large corpora exist,
like the biomedical field, pre-training language models from scratch yields
better results than feeding the pre-training phase with general-domain
knowledge [6].

To obtain contextualized word embeddings, BERT pre-training is based on masked
language modelling (MLM) which aims at predicting a small random subset of
masked input tokens, considering only the token context. This approach allows
the model to learn bidirectional representations. A different input corruption
procedure has been recently proposed, in which instead of masking, and
therefore losing, some of the input tokens, these are replaced with plausible
alternatives produced by a small generator network. By learning from all the
input tokens this novel approach, called ELECTRA (Efficiently Learning an
Encoder that Classifies Token Replacements Accurately), is computationally
much more efficient than BERT, and has been shown to outperform the latter in
several tasks [7].

Inspired by previous research achievements which showed the effectiveness of
language models built on domain-specific knowledge, and recognizing the
importance of resorting to efficiently pre-trained methods while preserving
downstream performances, in this study we describe a new ELECTRA-based model,
called ELECTRAMed, suited for the biomedical domain.

In particular, the main contributions of the present work are as follows:

  * •

We propose a novel ELECTRA-based language representation model (ELECTRAMed)
pre-trained on biomedical corpora. To the best of our knowledge, this is the
first ELECTRA-based model specifically developed for the biomedical domain,
and the present study is the first which applies, to this domain, a
transformer architecture different from BERT.

  * •

We tested ELECTRAMed on several biomedical benchmark NLP tasks. The results
achieved empirically show the effectiveness of the proposed approach, which
performed at par, and sometimes better, than state-of-the-art models while
leveraging the reduced computational effort required by ELECTRA-based
architectures.

  * •

We make publicly available the pre-processed datasets used in our study as
well as the pre-trained weights of ELECTRAMed and the source code for fine-
tuning the model111https://github.com/gmpoli/electramed.

##  2 Materials and methods

As it is generally the case with pre-trained methods for language
representation, the development of the new model encompassed two main phases
represented, respectively, by pre-training and fine-tuning, as illustrated in
the following sections. Since the proposed approach shares the same
architecture of ELECTRA, a brief description of the latter is also required.

###  2.1 ELECTRA

ELECTRA (Efficiently Learning an Encoder that Classifies Token Replacements
Accurately) is a pre-trained language model recently proposed to overcome the
computational drawback of approaches based on masked language modelling (MLM),
like BERT, where the input is first corrupted by masking some tokens in the
sentence and a network is then trained to recover the original identity of the
corrupted tokens. Since the network learns from a small percentage of masked-
out tokens (typically 15%), these techniques usually require a huge amount of
computations to be effective [7].

To address this inefficiency, ELECTRA implements an alternative pre-training
procedure called Replaced Token Detection (RTD), in which a network is trained
to distinguish real input tokens from synthetic but plausible replacements.
Specifically, the model consists of two neural networks which are pre-trained
jointly. The first is a generator that performs MLM by providing tokens
substitutes, and that then learns to predict the original token from the
masked form. The second is a discriminator which is trained to detect the
synthetic tokens, i.e. to distinguish real tokens from those replaced by the
generator. After pre-training the generator is dropped and the discriminator
is fine-tuned on labeled data for downstream tasks.

Compared to MLM, the innovative pre-training procedure embedded in ELECTRA is
more computationally efficient since the discriminator is required to predict
the class, real vs. replaced, of each of the input tokens, thereby learning
from the entire input sequence instead of a small portion of it. This feature
lets ELECTRA compete with state-of-the-arts models in several NLP problems
while reducing dramatically the computational effort needed for pre-training,
as described in the seminal paper [7]. Moreover, by resorting to plausible
tokens alternatives for masking, RTD helps in mitigating a disadvantage of the
traditional MLM approach, which introduces a potential mismatch in the
learning framework between pre-training and fine-tuning due to the use, for
input corruption, of the same conventional [MASK] token which does not appear
in the fine-tuning phase. Its computational efficiency, together with the
promising results achieved for different NLP tasks, motivated the adoption of
ELECTRA in the present study.

###  2.2 ELECTRAMed pre-training

Differently from other prominenent pre-trained models, such as BioBERT,
ELECTRAMed wasn’t initialized with the weights from ELECTRA. Indeed, pre-
training was performed entirely on a biomedical domain corpus. The corpus at
hand was published by [4] and consists of 28,714,373 PubMed abstracts
(uncompressed, approximately 26GB), representing the whole amount of abstracts
published on the free digital repository until September 2018. In more detail,
the corpus contains $\sim$181M sentences and $\sim$4B words and was subject to
the following common NLP pre-processing steps applied by the proponents of the
dataset:

  * •

Text lowercasing;

  * •

Special characters (\x00-\x7F) removal;

  * •

Text tokenization using NLTK Treebank tokenizer 222http://www.nltk.org/.

In any NLP task, vocabularies are needed to encode tokens with numbers, and
are generally built so to contain the most frequent words or subword units. In
the present study we made use of SciVocab, a WordPiece vocabulary proposed by
[8] and built on a scientific text corpus by means of the SentencePiece
library 333https://github.com/google/sentencepiece. Compared to the more
commonly used vocabulary released with BERT, SciVocab is characterized by
almost the same size ($\sim$30k) and 42% of tokens overlap, thereby showing a
substantial difference in the words used frequently in scientific texts with
respect to the general domain case. Notice that, the use of a more scientific-
oriented vocabulary should reduce the incidence of out-of-vocabulary (OOV)
tokens and, therefore, the loss of information in the text encoding phase. The
hyperparameters applied for pre-training ELECTRAMed on the corpus mentioned
above are reported in Table 1. These correspond to the same set of parameters
used for the ELECTRA-base model, as described in [7]. For tokenization we
instead resorted to the WordPiece scheme adopted by BERT.

It is well known that generating models which adhere to the pre-training and
fine-tuning paradigm is a highly resource-intensive process. In the case of
BERT, for example, the computational complexity of the self-attention layer
increases quadratically with the length of the sequences after tokenization.
As a result, limiting the maximum length of the input sequence to 128, at
least for the first part of the training, and then increasing it to 512 at a
later stage, is a commonly adopted strategy to gain efficiency. This is done
despite the risk of reducing the ability of the model to capture long-distance
dependencies and, therefore, to negatively affect its performance. Differently
from other approaches, by leveraging the computational advantages provided by
the ELECTRA framework, we were able to investigate the use of a maximum
sequence length of 512, instead of 128, for the whole pre-training phase,
while keeping the training time on par with other methods. Indeed, pre-
training ELECTRAMed took $\sim$10 days by using one TPU v3 with 8 cores.

Table 1: Hyperparameters used for ELECTRAMed pre-training Hyperparameter |  Value
---|---
Number of layers |  12
Hidden size |  768
FFN inner hidden size |  3072
Attention heads |  12
Attention head size |  64
Embedding Size |  768
Generator Size |  1/3
Mask percent |  15
Learning Rate Decay |  Linear
Warmup steps |  10000
Learning Rate |  2e-4
Adam $\epsilon$ |  1e-6
Adam $\beta_{1}$ |  0.9
Adam $\beta_{2}$ |  0.999
Attention Dropout |  0.1
Dropout |  0.1
Weight Decay |  0.01
Batch size |  256
Train epochs |  1M

###  2.3 ELECTRAMed fine-tuning

After pre-training, ELECTRAMed was fine-tuned and tested on three biomedical
NLP tasks, represented by named entity recognition (NER), relationship
extraction (RE) and question answering (QA).

Named entity recognition (NER) is aimed at automatically finding and tagging
in a text meaningful terms, called named entities. In a general domain these
typically refer to sequences of words corresponding to specific entities in
the real world, such as locations, persons, organizations. In the biomedical
field named entities can represent the name of genes, diseases, chemical
compounds, and drugs, to name a few. The goal of NER is tagging each token in
a sentence with one taken from a list of possible named entities, or with the
"no entity" label. NER is often applied as a preliminary step for many
applications, such as relationship extraction and knowledge base completion.
It is worthwhile to notice that, compared to the general domain, NER in the
biomedical field is considered to be more complex, since biomedical entities
constantly grow in number with the scientific progress, may contain special
characters and can be referred to using a wide variety of synonyms and
abbreviations.

Among the available annotation schemes used to label multi-token named
entities, in the present work we resorted to BIO tagging [9], for which three
binary classifiers are trained to label each token in the text as _B_ (the
token is the beginning of a named entity), _I_ (the token is inside a named
entity but is not the beginning), and _O_ (otherwise).

For fine-tuning and testing ELECTRAMed on biomedical NER, three publicly
available corpora were used. The first, denoted here as NCBI-disease, is the
disease corpus of the National Center for Biotechnology Information (NCBI)
[10], which is a collection of 793 PubMed abstracts annotated at the mention
and concept level. In detail, it contains 6892 disease mentions mapped into
790 unique disease concepts. The second dataset (BC5CDR) was used for the
BioCreative V Chemical Disease Relation task and is composed by 1500 PubMed
articles with 4409 annotated chemicals, 5818 diseases and 3116 chemical-
disease interactions [11]. From the entire collection we extracted the subset
with chemical and disease entities, along the lines of previous studies. The
third dataset (JNLPBA) was released for an open challenge and derives from the
GENIA corpus, which is a collection of abstracts retrieved by controlled
search on MEDLINE. For the shared task, the authors simplified the original 36
classes into five super-classes represented, respectively, by proteins, DNA,
RNA, cell types and cell lines [12]. These correspond to the biological named
entities of interest.

Table 2: Description of the corpora used as benchmarks for biomedical NER Dataset |  Entity Type |  N. entities |  N. train |  N. val |  N. test
---|---|---|---|---|---
NCBI-disease |  Disease |  6,892 dis. |  5,429 |  923 |  941
BC5CDR |  Disease and |  10,227 TOT |  3,951 |  3,957 |  4,145
|  Chemical |  4,409 chemicals |  |  |
|  |  5,818 diseases |  |  |
JNLPBA |  Gene and |  59,963 TOT |  16,845 |  1,743 |  3,869
|  Protein |  35,336 proteins |  |  |
|  |  10,589 DNA |  |  |
|  |  1,069 RNA |  |  |
|  |  8,639 cell types |  |  |
|  |  4,330 cell lines |  |  |

Relationship extraction (RE) generally follows NER and is aimed at finding
semantic relationships which may occur in a text between two or more entities.
In the biomedical domain relationships are extracted between biological
entities and, for example, can take the form of different kinds of relations
among genes and diseases or of protein-protein interactions. RE is cast as a
text classification problem in which, given a pair of entities, the goal is to
assign the correct type of relationship, whether this exists. To validate
ELECTRAMed on RE problems two datasets were analyzed. The first is CHEMPROT
[13], which is a corpus containing annotated relationship types between
chemicals and proteins. From the initial group of 10 different kinds of
relationships, five of these (CPR:3, CPR:4, CPR:5, CPR:6 and CPR:9) were used
for evaluation. The second corpus (DDI-2013) contains 792 texts extracted from
the DrugBank database and other 233 MEDLINE abstracts describing drug-drug
interactions (DDIs). The corpus was annotated by considering four different
types of interactions (i.e. effect, mechanism, advice, and int), where the
last was used when a DDI appeared in the text without providing any specific
additional information [14].

Table 3: Description of the corpora used as benchmarks for biomedical RE Dataset |  Relationship |  N. relations |  N. train |  N. val |  N. test
---|---|---|---|---|---
CHEMPROT |  Chemical- |  10,028 TOT |  19,460 |  11,820 |  16,943
|  Protein |  1,983 CPR:3 |  |  |
|  |  5,006 CPR:4 |  |  |
|  |  484 CPR:5 |  |  |
|  |  727 CPR:6 |  |  |
|  |  1,828 CPR:9 |  |  |
DDI-2013 |  Drug-Drug |  2,937 TOT |  18,779 |  7,244 |  5,761
|  |  1,212 effect |  |  |
|  |  946 mechanism |  |  |
|  |  633 advice |  |  |
|  |  146 int |  |  |

Question answering (QA) can be considered an extension of information
retrieval and has the purpose of delivering direct and precise responses to
questions asked in natural language. Focusing on the biomedical domain, and in
particular on inquiries about the treatment of a disease, a QA tool would
provide, as answers, specific drugs which are effective against that disease
or short text passages containing the response. Indeed, there are three
categories of questions which are typically asked to a QA system: confirmation
questions, that can be dealt with by yes-or-no statements, list questions,
which expect a list of entities or facts, and factoid questions, which require
a single short phrase or sentence as response. In our work, we focused on this
last type of questions, for which the goal is finding the correct answer
inside a passage. This is achieved by training a model capable of predicting
the starting and the ending tokens of the text segment containing the full
expected response. For biomedical semantic QA we selected as benchmark the
pre-processed version of the BioASQ Task 7b-factoid dataset [15], which
contains both biomedical questions and gold standard answers, in the form of
relevant concepts, articles, snippets, exact answers, summaries and the
corresponding full PubMed abstracts as passages.

As for NER, question answering in the biological domain poses a major
challenge in contrast to the open or other restricted domains, for the
presence of a highly specialized terminology and for a potential larger gap in
technicality between the questions made by non-expert users and the target
documents. Another critical issue is the scarcity of labelled datasets. To
tackle this problem, prior to addressing the biomedical QA task, we fine-tuned
ELECTRAMed on the Stanford Question Answering Dataset (SQuAD v1.1), a large-
scale general-domain reading comprehension dataset published by [16] and
including 87,599 and 10,570 examples for training and testing, respectively.
This additional fine-tuning was performed in line with previous studies, which
showed its effectiveness in improving the performance on domain-specific
activities [17], [15].

Table 4: Description of the corpus (BioASQ 7b-factoid) used as benchmark for biomedical QA Batch |  N. questions |  N. question-context pairs
---|---|---
Train |  556 |  5537
Test batch 1 |  39 |  98
Test batch 2 |  25 |  56
Test batch 3 |  29 |  84
Test batch 4 |  34 |  90
Test batch 5 |  35 |  79

The description of the corpora used for the three NLP tasks in terms of number
of training, testing and validation examples, and number of observations for
each class, are provided in Tables 2, 3 and 4. The hyperparameters applied for
fine-tuning ELECTRAMed on all the tasks are instead indicated in Table 5.
Notice that, the datasets at hand are gold standards for the evaluation of
biomedical NLP tools. To enable a fair comparison with previous studies,
therefore, we adopted the same partition into training, test and validation
sets as originally proposed by the authors of the corpora. Finally, by way of
example, Table 6 shows the detected named entities and the answers provided by
ELECTRAMed for samples extracted from the corpora used for NER and QA tasks.

Table 5: Hyperparameters used for ELECTRAMed fine-tuning Hyperparameter |  NER |  RE |  QA_SQuAD |  QA_BioASQ
---|---|---|---|---
Learning Rate |  5e-5 |  5e-5 |  3e-5 |  5e-6
Adam $\epsilon$ |  1e-6 |  1e-6 |  1e-6 |  1e-6
Adam $\beta_{1}$ |  0.9 |  0.9 |  0.9 |  0.9
Adam $\beta_{2}$ |  0.999 |  0.999 |  0.999 |  0.999
Layerwise LR Decay |  0.8 |  0.8 |  0.8 |  0.8
Learning Rate Decay |  Linear |  Linear |  Linear |  Linear
Attention Dropout |  0.1 |  0.1 |  0.1 |  0.1
Dropout |  0.1 |  0.1 |  0.1 |  0.1
Weight Decay |  0 |  0 |  0 |  0
Batch size |  32 |  32 |  16 |  16
Max Sequence Length |  128 |  128 |  384 |  384
Document Stride |  NA |  NA |  128 |  128
Table 6: Detected named entities and answers provided by ELECTRAMed (in bold) for samples taken from NER and QA corpora Task |  Dataset |  Sample
---|---|---
NER |  BC5CDR |  Thus, indomethacin by inhibition of prostaglandin synthesis may diminish the blood pressure maintaining effect of the stimulated effect of the stimulated renin-angiotensin system in sodium and volume depletion.
NER |  NCBI-disease |  Occasional missense mutations in ATM were also found in tumour DNA from patients with B-cell non-Hodgkins lymphomas (B-NHL) and a B-NHL cell line..
QA |  BioASQ-7b |  Q: What is the cause of a STAG3 truncating variant?
|  |  A: STAG3 truncating variant as the cause of primary ovarian insufficiency. Primary ovarian insufficiency (POI) is a distressing cause of infertility in young women …
QA |  BioASQ-7b |  Q: Which receptor is targeted by Erenumab?
|  |  A: … Erenumab, a human monoclonal antibody that inhibits the calcitonin gene-related peptide receptor, is being evaluated for migraine prevention …

##  3 Results

For named entity recognition the performance of ELECTRAMed was evaluated by
means of the F1-score (F), defined as the harmonic mean between precision (P)
and recall (R). Accordingly to [9], precision was computed as the percentage
of named entities found by the model that were correct (i.e. were an exact
match of the corresponding entity in the corpus), whereas recall was set as
the percentage of entities included in the corpus and detected by the model.
For the purpose of relationship extraction, the F1-score was applied as well.
In this case, the predicted triplets entity-relation-entity were deemed
correct if the relation and the two entities were the same as the ground
truth. Finally, for question answering we resorted to three quality measures,
commonly used for assessing tools which generate a list of possible responses
to a given inquiry. The first is the mean reciprocal rank (MRR), given by the
average of the reciprocal ranks of the results for a sample of queries. The
other two are the strict accuracy (SACC) and the lenient accuracy (LACC), for
which a question is correctly answered if the gold response is the first
element of the list returned by the model, or it is included in the list,
respectively.

For each of the NLP problems ELECTRAMed was compared with the current best
state-of-the-art (SOTA) models, including those developed by the participants
of the 7th BioASQ Challenge runs (QA activity). The models comprised within
each SOTA group are specified in the captions of Tables 7, 8 and 9, for every
task and corpus. These tables also contain missing values denoted as “?”, to
indicate the unavailability of the corresponding measure for the given pair
model-dataset. Missing outcomes are associated to precision and recall for
both NER and RE tasks. For the sake of completeness, we deemed relevant to
provide the results for ELECTRAMed also in terms of these two quality
measures. It is also worthwhile to observe that, all the values referred to
ELECTRAMed in the tables are obtained by averaging the respective metrics over
five runs with different seeds. This is in line with the experimental settings
adopted for SciBERT [8], for which the outcomes were computed as average over
multiple runs. For the other SOTA models, instead, we were unable to establish
whether the results reported by the authors were averaged or corresponded to
the best outcomes achieved.

The results obtained by ELECTRAMed for the task of named entity recognition
are shown in Table 7. The proposed model reached the highest F1-score (90.03)
on the BC5CDR dataset. By performing better than the current SOTA approaches,
ELECTRAMed sets a novel state-of-the-art performance on this corpus for NER
purposes in terms of F1-score. For NCBI-disease and JNLPBA datasets,
ELECTRAMed was not among the top three SOTA models, but still reached
comparable results on the first corpus.

Table 7: Precision (P), recall (R) and F1-score (F) for the NER task

Benchmark |  Metrics |  SOTA1 |  SOTA2 |  SOTA3 |  ELECTRAMed
---|---|---|---|---|---
NCBI-disease |  P |  88.22 |  ? |  ? |  85.87
|  R |  91.25 |  ? |  ? |  89.29
|  F |  89.71 |  89.13 |  88.85 |  87.54
BC5CDR |  P |  92.05 |  ? |  ? |  88.76
|  R |  87.91 |  ? |  ? |  91.34
|  F |  89.93 |  89.73 |  89.42 |  90.03
JNLPBA |  P |  ? |  72.24 |  ? |  69.33
|  R |  ? |  83.26 |  ? |  78.56
|  F |  81.29 |  77.59 |  77.03 |  73.65

State-of-the-art (SOTA) performance. For NCBI-disease, SOTA1 is BioBERT, [5],
SOTA2 is Spark NLP, [18], SOTA3 is BioFLAIR, [19]. For BC5CDR, SOTA1 is
RL+DS+PA, [20], SOTA2 is Spark NLP and SOTA3 is BioFLAIR. For JNLPBA, SOTA1 is
Spark NLP, SOTA2 is BioBERT and SOTA3 is BioFLAIR.

The outcomes for the task of relationship extraction are indicated in Table 8.
These results clearly demonstrate the superior effectiveness of SciBERT over
the other models, but also show the promising performance of ELECTRAMed.
Indeed, on the DDI-2013 dataset the proposed approach provided results which
are close to the second-best model (SOTA2) and ranked at the third position by
reaching a fairly higher F1-score compared to the current SOTA3 (79.13 vs.
72.90).

Finally, the results achieved for the question answering task are depicted in
Table 9. ELECTRAMed was able to outperform all the competitors in two (batches
1 and 4) out of the five runs, and to provide comparable results with the best
approach (KU-DMIS-5) for the second batch. For the remaining runs (batches 3
and 5), ELECTRAMed ranked at the seventh and sixth position, respectively,
among all the participants. To investigate the ability of ELECTRAMed of
providing high-quality responses across the whole challenge, besides the
BioASQ baseline method we selected the models that competed in all the runs
and, for each of them, we computed a score given by the ratio between their
MRR and the highest MRR reached in a given batch. For each model, the sum of
the scores over all the runs can be seen as a measure of its ability of
providing responses which are close, if not equal, to the best ones in terms
of MRR. The results of this analysis are reported in Table 10 and support the
effectiveness of ELECTRAMed for the QA task at hand. The proposed model,
indeed, achieved the highest total score being associated with ratios more
densely distributed around 1. In particular, ELECTRAMed performed better than
systems based on the BioBERT approach (KU-DMIS-5 model).

Table 8: Precision (P), recall (R) and F1-score (F) for the RE task

Benchmark |  Metrics |  SOTA1 |  SOTA2 |  SOTA3 |  ELECTRAMed
---|---|---|---|---|---
CHEMPROT |  P |  ? |  77.02 |  ? |  75.47
|  R |  ? |  75.90 |  ? |  70.67
|  F |  83.64 |  76.46 |  74.40 |  72.94
DDI-2013 |  P |  ? |  ? |  74.10 |  80.07
|  R |  ? |  ? |  71.80 |  78.24
|  F |  84.08 |  79.90 |  72.90 |  79.13

State-of-the-art (SOTA) performance. For CHEMPROT, SOTA1 is SciBERT, [8],
SOTA2 is BioBERT, SOTA3 is BlueBERT, [4]. For DDI-2013, SOTA1 is
DESC+MOL+SciBERT, [21], SOTA2 is BlueBERT, SOTA3 is Hierarchy Bi-LSTMs
+Att.+SDP, [22].

Table 9: Strict accuracy (SACC), lenient accuracy (LACC) and mean reciprocal
rank (MRR) for the QA task

Batch |  Competitor |  SACC |  LACC |  MRR
---|---|---|---|---
1 |  (1) ELECTRAMed |  44.62 |  51.28 |  47.95
|  (2) KU-DMIS-1 |  41.03 |  53.85 |  46.37
|  (3) BJUTNLPGroup |  30.77 |  41.03 |  34.83
|  (4) auth-qa-1 |  25.64 |  30.77 |  27.78
2 |  (1) KU-DMIS-5 |  52.00 |  64.00 |  56.67
|  (2) ELECTRAMed |  46.40 |  62.40 |  53.16
|  (3) QA1 |  36.00 |  48.00 |  40.33
|  (4) transfer-learning |  24.00 |  44.00 |  32.67
3 |  (1) QA1 |  44.83 |  58.62 |  51.15
|  (2) UNCC_QA_1 |  44.83 |  58.62 |  51.15
|  (3) google-gold-input |  41.38 |  65.52 |  50.23
|  (7) ELECTRAMed |  37.93 |  58.62 |  46.62
4 |  (1) ELECTRAMed |  61.18 |  82.35 |  69.55
|  (2) KU-DMIS-1 |  58.82 |  82.35 |  69.12
|  (3) FACTOIDS |  52.94 |  73.53 |  61.03
|  (4) UNCC_QA3 |  52.94 |  73.53 |  61.03
5 |  (1) KU-DMIS-5 |  28.57 |  51.43 |  36.38
|  (2) BJUTNLPGroup |  28.57 |  40.00 |  33.81
|  (3) UNCC_QA_1 |  28.57 |  42.86 |  33.05
|  (6) ELECTRAMed |  24.57 |  44.00 |  31.42

Note: The number in round brackets beside each model indicates the ranking in
the challenge run.

Table 10: Scores over the five runs of the 7h BioASQ-factoid Challange

Competitor |  Batch1 |  Batch2 |  Batch3 |  Batch4 |  Batch5 |  Total
---|---|---|---|---|---|---
BioASQ Baseline |  0.323 |  0.241 |  0.258 |  0.364 |  0.238 |  1.424
auth-qa-1 |  0.579 |  0.541 |  0.669 |  0.536 |  0.412 |  2.737
KU-DMIS-1 |  0.967 |  0.771 |  0.924 |  0.994 |  0.886 |  4.542
LabZhu,FDU |  0.120 |  0.441 |  0.775 |  0.699 |  0.615 |  2.650
ELECTRAMed |  1.000 |  0.938 |  0.911 |  1.000 |  0.864 |  4.713

Note: Three models by Lab Zhu at Fudan University were proposed. The table
includes the one that performed better across the runs.

##  4 Conclusions & Future Developments

The recent literature on biomedical NLP has been heavily influenced by BERT-
based architectures and by the usage of domain-specific corpora for pre-
training. Meanwhile, in the general-domain NLP literature, a plethora of
transformer-based architectures have flourished, bringing significant
improvements to the first wide-spread implementation. In this study we
presented ELECTRAMed, a new pre-trained language model for biomedical NLP.
ELECTRAMed was pre-trained from scratch on a biomedical corpus using a domain-
specific vocabulary, and was shown to obtain valuable results for some of the
most commonly addressed NLP tasks arising in the biomedical field. For
ELECTRAMed pre-training we leveraged the efficiency of the ELECTRA
architecture, and we were able to perform at par of the current state-of-the-
art models while keeping the computational effort low, in terms of both time
and cost.

The results achieved in the present work encourage future studies that can be
undertaken along different directions. From one side, it would be worthwhile
to investigate the performance of ELECTRAMed when the maximum sequence length
of the input is reduced from 512 to 128 for the entire, or at least, the most
part of the training phase, with the aim of further reducing the computational
requirements. From the other, it would be useful to explore the impact on the
model performance of using a vocabulary built upon the selected pre-training
corpus. With this study we hope to spark a new wave of transformer-based
architectures in the biomedical domain. Consequently, as a future research
line it would be also interesting to investigate potential improvements for
tasks related to biomedical information extraction by combining existing
biomedical domain knowledge resources (e.g. knowledge bases) into novel
transformer-based learning frameworks.

## Acknowledgements

We thank Dr. Sandra Coecke from the Joint Research Center at European
Commission and Dr. Anna Beronius from Karolinska Institute for their valuable
and fruitful discussions that fostered a positive and encouraging environment
which greatly contributed to the development of our work.

## References

  * [1] M.E. Peters, M. Neumann, M. Iyyer, M. Gardner, C. Clark, K. Lee, and L. Zettlemoyer.  Deep contextualized word representations.  In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 2227–2237. Association for Computational Linguistics, New Orleans, Louisiana, 2018.
  * [2] J. Devlin, M.W. Chang, K. Lee, and K. Toutanova.  BERT: pre-training of deep bidirectional transformers for language understanding.  In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186. Association for Computational Linguistics, Minneapolis, USA, 2019.
  * [3] Y. Belinkov and J. Glass.  Analysis methods in neural language processing: A survey.  Transactions of the Association for Computational Linguistics, 7:49–72, 2019.
  * [4] Y. Peng, S. Yan, and Z. Lu.  Transfer learning in biomedical natural language processing: An evaluation of BERT and ELMo on ten benchmarking datasets.  In Proceedings of the 18th BioNLP Workshop and Shared Task, pages 58–65. Association for Computational Linguistics, Florence, Italy, 2019\.
  * [5] J. Lee, W. Yoon, S. Kim, D. Kim, S. Kim, C.H. So, and J. Kang.  BioBERT: a pre-trained biomedical language representation model for biomedical text mining.  Bioinformatics, 36:1234–1240, 2020.
  * [6] Y. Gu, R. Tinn, H. Cheng, M. Lucas, N. Usuyama, X. Liu, T. Naumann, J. Gao, and H. Poon.  Domain-specific language model pretraining for biomedical natural language processing.  arXiv preprint: 2007.15779, 2020.
  * [7] K. Clark, M.H. Luong, Q.V. Le, and C.D. Manning.  ELECTRA: Pre-training text encoders as discriminators rather than generators.  arXiv preprint: 2003.10555, 2020.
  * [8] I. Beltagy, A. Cohan, and K. Lo.  SciBERT: Pretrained contextualized embeddings for scientific text.  arXiv preprint: 1903.10676, 2019.
  * [9] E.F.T.K. Sang and S. Buchholz.  Introduction to the CONLL-2000 shared task: Chunking.  In Proceedings of the fourth conference on computational natural language learning and of the second learning language in logic workshop (CONLL/LLL 2000), pages 127–132. Lissabon, Portugal, 2000.
  * [10] R.I. Doğan, R. Leaman, and Z. Lu.  NCBI disease corpus: a resource for disease name recognition and concept normalization.  Journal of Biomedical Informatics, 47:1–10, 2014.
  * [11] J. Li, Y. Sun, R.J. Johnson, D. Sciaky, C.-H. Wei, R. Leaman, A.P. Davis, C.J. Mattingly, T.C. Wiegers, and Z. Lu.  BioCreative V CDR task corpus: a resource for chemical disease relation extraction.  Database: The Journal of Biological Databases and Curation, 2016\.  2016 baw068.
  * [12] J.D. Kim, T. Ohta, Y. Tsuruoka, Y. Tateisi, and N. Collier.  Introduction to the bio-entity recognition task at JNLPBA.  In Proceedings of the International Joint Workshop on Natural Language Processing in Biomedicine and its Applications (NLPBA/BioNLP), pages 73–78. COLING, Geneva, Switzerland, 2004.
  * [13] M. Krallinger, O. Rabal, S.A. Akhondi, M. Pérez, J. Santamaría, G. Pérez Rodríguez, G. Tsatsaronis, A. Intxaurrondo, J.A. López, U. Nandal, E.V. Buel, A. Chandrasekhar, M. Rodenburg, A. Lægreid, M.A. Doornenbal, J. Oyarzábal, A. Lourenço, and A. Valencia.  Overview of the BioCreative VI chemical-protein interaction track.  In Proceedings of the sixth BioCreative challenge evaluation workshop, pages 141–146. 2017.
  * [14] M. Herrero-Zazo, I. Segura-Bedmar, P. Martínez, and T. Declerck.  The DDI corpus: An annotated corpus with pharmacological substances and drug–drug interactions.  Journal of Biomedical Informatics, 46(5):914–920, 2013.
  * [15] W. Yoon, J. Lee, D. Kim, M. Jeong, and J. Kang.  Pre-trained language model for biomedical question answering.  In Machine Learning and Knowledge Discovery in Databases, pages 727–740. Springer International Publishing, 2020.
  * [16] P. Rajpurkar, J. Zhang, K. Lopyrev, and P. Liang.  SQuAD: 100,000+ questions for machine comprehension of text.  arXiv preprint: 1606.05250, 2016.
  * [17] G. Wiese, D. Weissenborn, and M. Neves.  Neural question answering at BioASQ 5B.  arXiv preprint: 1706.08568, 2017.
  * [18] V. Kocaman and D. Talby.  Biomedical named entity recognition at scale.  arXiv preprint: 2011.06315, 2020.
  * [19] S. Sharma and R. Daniel.  BioFLAIR: Pretrained pooled contextualized embeddings for biomedical sequence labeling tasks.  arXiv preprint: 1908.05760, 2019.
  * [20] F. Nooralahzadeh, J.T. Lønning, and L. Øvrelid.  Reinforcement-based denoising of distantly supervised NER with partial annotation.  In Proceedings of the 2nd Workshop on Deep Learning Approaches for Low-Resource NLP (DeepLo 2019), pages 225–233. Association for Computational Linguistics, Hong Kong, China, 2019.
  * [21] M. Asada, M. Miwa, and Y. Sasaki.  Using drug descriptions and molecular structures for drug-drug interaction extraction from literature.  Bioinformatics, 2020.  btaa907.
  * [22] Y. Zhang, W. Zheng, H. Lin, J. Wang, Z. Yang, and M. Dumontier.  Drug–drug interaction extraction via hierarchical RNNs on sequence and shortest dependency paths.  Bioinformatics, 34:828–835, 2018.


11institutetext: Cédric Join 22institutetext: CRAN (CNRS, UMR 7039),
Université de Lorraine, BP 239, 54506 Vandœuvre-lès-Nancy, France.
22email<EMAIL_ADDRESS>33institutetext: Hassane Abouaïssa
44institutetext: Univ. Artois, UR 3926, Laboratoire de Génie Informatique et
d’Automatique de l’Artois (LGI2A) F-62400 Béthune, France.
44email<EMAIL_ADDRESS>55institutetext: Michel Fliess
66institutetext: LIX (CNRS, UMR 7161), École polytechnique, 91128 Palaiseau,
France.
66email<EMAIL_ADDRESS>77institutetext: Cédric Join and
Michel Fliess 88institutetext: AL.I.E.N., 7 rue Maurice Barrès, 54330
Vézelise, France.
88email:  {cedric.join<EMAIL_ADDRESS>
#  $3$rd DECOD Workshop
(DElays and COnstraints in Distributed parameter systems)
23-26 November 2021, Gif-sur-Yvette, France
Ramp metering:
modeling, simulations and control issues

Cédric Join  Hassane Abouaïssa and Michel Fliess

###### Abstract

The aim of ramp metering is to improve the highway traffic conditions by an
appropriate regulation of the inflow from the on-ramps to the highway
mainstream. Our presentation rests on several improvements: 1) Our simulation
techniques do not need contrarily to other approaches any heuristic
fundamental law. 2) There is no need of crucial time-varying quantities, like
the critical density, which is most difficult to estimate correctly online. 3)
Our feedback loop, which is stemming from model-free control, is easy to
implement and shows an excellent robustness with respect to model mismatch.
Several computer experiments are displayed and discussed.

Key Words Ramp metering, Lighthill-Whitham-Richards partial differential
equation, fundamental diagrams, fundamental laws, ALINEA, model-free control,
intelligent proportional controllers.

##  1 Introduction

_Ramp metering_ is about the use of traffic signals at highway on-ramps in
order to control the rate of vehicles entering the highway. The signals can be
set for different metering rates to optimize freeway flow and minimize
congestion. The social and ecological damages due to traffic jams justify the
huge academic literature which has been devoted to traffic flow dynamics and
its regulation. See, e.g., kachroo1 ; kachroo2 ; kerner1 ; kerner2 ; mammar ;
may ; papageorgiou0 ; treiber , and references therein. Lack of space prevents
us from a careful analysis of the various standpoints which are often
antagonist.

Our contributions may be summarized like follows:

  1. 1.

No fundamental diagram for simulation purposes. Before being implemented in
practice any ramp metering procedure ought to be tested via computers. The
similarities between highway traffic and hydrodynamics explain the importance
of the Lighthill-Whitham-Richards (LWR) first order partial differential
equation LWR1 ; LWR2 . Its numerical integration necessitates to know the
relationship between the traffic density and the traffic speed. Several
empirical laws, called _fundamental diagrams_ , have been proposed. The
calibration of those diagrams is far from being obvious. That is why
contrarily to other approaches (see, e.g., barcelo ) we ignore them.
Fundamental diagrams are replaced by traffic data which are collected during
various situations. Note moreover that our viewpoint, contrarily to other ones
like METANET kots , does not necessitate higher order partial differential
equations.

  2. 2.

No difficult estimation technique. Implementing any control law requires
crucial quantities like the _critical density_ , which are time-varying and
therefore almost impossible to estimate in real time. They are replaced by ad
hoc quantities which are deduced at once from easily measurable data.

  3. 3.

Model-free control. ALINEA111It is the acronym of Asservissement LINéaire
d’Entrée Autoroutière. hist1 ; hist2 ; papageorgiou1 ; papageorgiou2  is
perhaps the most popular feedback control algorithm for traffic regulation.
After first trials in Paris it has been quite often employed in many different
places. Following sofia , we are using here _model-free control_ in the sense
of mfc13 ; nicu  and, more precisely, _intelligent proportional controllers_.
This setting, which has been successfully tested in many concrete situations,
has already been illustrated via various questions about intelligent
transportation systems (see, e.g., andrea1 ; baciu ; haddar ; menhour ; milan
; polack ; villagra ; wang ; yang ). Concrete experiments show today that
model-free control not only yields better traffic regulation than ALINEA but
is also simpler to implement.222There are many variants of ALINEA in the
literature. We have selected for our computer comparisons the version alinea0
.

Some remarks might be useful:

  * •

Let us emphasize that the LWR partial differential equation is only related to
simulation purposes and not to traffic control. Such a control would
necessitate a real-time calibration, which seems today beyond all reasonable
hope.

  * •

Achieving an efficient traffic regulation is simpler than obtaining reliable
computer simulations. This ascertainment remains valid in most applications
where model-free control plays a key rôle.

Our paper is organized as follows. Section 2 reviews model-free control and
the corresponding _intelligent proportional controller_. Following alinea ,
Section 2.3 investigates the relationship with proportional controllers, which
should be considered as the backbone of ALINEA. Modeling issues are discussed
in Section 3: The partial differential equation stemming from elementary
hydrodynamic conservation laws, some fundamental empirical laws, the space
discretization and its use without any fundamental empirical law, the ALINEA
and model-free regulation without the critical density. Computer simulations
are displayed and commented in Section 4. Section 4.2 in particular exhibits
the superiority of model-free control in the presence of sudden changes.
Section 5 contains some concluding remarks.

##  2 Model-free control

###  2.1 A short review

#### Ultra-local model

Consider only, for simplicity’s sake, SISO (single-input single-output)
systems. Elementary functional analysis and differential algebra as used in
mfc13  show as well as practical experiments that most, or at least many,
concrete systems may be approximated by the _ultra-local_

$\boxed{\dot{y}=F+\alpha u}$ (1)

where

  * •

the control and output variables are respectively $u$ and $y$,

  * •

the constant $\alpha\in\mathbb{R}$ is chosen by the practitioner such that
$\alpha u$ and $\dot{y}$ are of the same magnitude. Therefor $\alpha$ does not
need to be precisely estimated.

The following comments might be useful:

  * •

$F$ is estimated via the knowledge of the control and output variables $u$ and
$y$,

  * •

$F$ subsumes not only the unknown structure of the system, but also any
external disturbance.

#### Intelligent controllers

Close the loop with the following _intelligent proportional controller_ , or
_iP_ ,

$\boxed{u=-\frac{F-\dot{y}^{\ast}+K_{P}e}{\alpha}}$ (2)

where:

  * •

$y^{\ast}$ is the reference trajectory,

  * •

$e=y-y^{\ast}$ is the tracking error,

  * •

$K_{P}$ is a tuning gain.

Combining Equations (1) and (2) yields:

$\dot{e}+K_{P}e=0$

where $F$ does not appear anymore. Thus $\lim_{t\to+\infty}e(t)=0$ iff
$K_{P}>0$. This local stability property proves that the tuning of $K_{P}$ is
straightforward. This is a major difference with the classic gain tuning for
PIs and PIDs (see, e.g., astrom  and the references therein).

#### Estimation of $F$

Under a weak integrability condition, $F$ in Equation (1) may be “well”
approximated by a piecewise constant function $F_{\text{est}}$ (see, e.g.,
bourbaki ). The estimation techniques below are borrowed from sira1 ; sira2 ;
sira .333They were often used in practice for parameter identification (see,
e.g., train ). Let us summarize two types of computations:

  1. 1.

Rewrite Equation (1) in the operational domain (see, _e.g._ , yosida ):

$sY=\frac{\Phi}{s}+\alpha U+y(0)$

where $\Phi$ is a constant. We get rid of the initial condition $y(0)$ by
deriving both sides with respect to $s$:

$Y+s\frac{dY}{ds}=-\frac{\Phi}{s^{2}}+\alpha\frac{dU}{ds}$

Noise attenuation is achieved by multiplying both sides on the left by
$s^{-2}$, since integration with respect to time is a lowpass filter (see
noise  for further details). It yields in the time domain the realtime
estimate, thanks to the equivalence between $\frac{d}{ds}$ and the
multiplication by $-t$,

${\small
F_{\text{est}}(t)=-\frac{6}{\tau^{3}}\int_{t-\tau}^{t}\left[(\tau-2\sigma)y(\sigma)+\alpha\sigma(\tau-\sigma)u(\sigma)\right]d\sigma}$
(3)

where $\tau>0$ might be quite small. This integral may of course be replaced
in practice by a classic digital filter.

  2. 2.

Close the loop with the iP (2). It yields:

$F_{\text{est}}(t)=\frac{1}{\tau}\left[\int_{t-\tau}^{t}\left(\dot{y}^{\ast}-\alpha
u-K_{P}e\right)d\sigma\right]$

###  2.2 PI and iP

Consider the classic proportional-integral controller, or PI,

$\boxed{u(t)=k_{p}e(t)+k_{i}\int e(\tau)d\tau}$ (4)

where $k_{p},k_{i}\in\mathbb{R}$ are constants. A crude sampling of the
integral $\int e(\tau)d\tau$ through a Riemann sum ${\cal{I}}(t)$ leads to

$\int e(\tau)d\tau\simeq{\cal{I}}(t)={\cal{I}}(t-h)+he(t)$

where $h$ is the sampling interval. The corresponding discrete form of
Equation (4) reads:

$u(t)=k_{p}e(t)+k_{i}{\cal{I}}(t)=k_{p}e(t)+k_{i}{\cal{I}}(t-h)+k_{i}he(t)$

Combining the above equation with

$u(t-h)=k_{p}e(t-h)+k_{i}{\cal{I}}(t-h)$

yields

$u(t)=u(t-h)+k_{p}\left(e(t)-e(t-h)\right)+k_{i}he(t)$ (5)

Replace in Equation (2) $F$ by ${\dot{y}}(t)-\alpha u(t-h)$ and therefore by

$\frac{y(t)-y(t-h)}{h}-\alpha u(t-h)$ (6)

It yields

$u(t)=u(t-h)-\frac{e(t)-e(t-h)}{h\alpha}-\dfrac{K_{P}}{\alpha}\,e(t)$ (7)

Equations (5) and (7) become identical if we set

$\displaystyle k_{p}$ $\displaystyle=-\dfrac{1}{\alpha h},\quad
k_{i}=-\dfrac{K_{P}}{\alpha h}$ (8)

Let us emphasize that this important result, which was first stated in andrea
; mfc13 , is only valid in discrete time: Formulae (8) become meaningless if
$h\downarrow 0$.

###  2.3 Application to integral controllers

Set in Equation (4) $k_{p}=0$. It yields the integral controller, or I,

$\boxed{u(t)=k_{i}\int e(\tau)d\tau}$ (9)

Derive both sides in Equation (4):

$\dot{u}=k_{p}\dot{e}+k_{i}e$

It shows that the PI and I controllers are “close” if $\dot{e}$ remains
“small.” It implies of course that the initial condition $y(0)$ is “close” to
the initial point $y^{\ast}(0)$ of the reference trajectory. Let us try to
explain why:

  * •

the feedback loop is supposed to render $y-y^{\ast}$ quickly as small as
possible,

  * •

the derivative $\dot{e}$ might therefore be large.

It follows from Equations (5) and (6) that the sampled versions of the iP (2)
and the I (9) are “close” (see alinea  for more details) if

  * •

the reference trajectory $y^{\ast}$ is “slowly” varying, and starts at the
initial condition $y(0)$ or, at least, at a point which is quite close to it,

  * •

the disturbances and the corrupting noises are rather mild.

##  3 Application to ramp metering

###  3.1 Traffic flow modeling

#### The LWR partial differential equation

The Lighthill-Whitham-Richards, or LWR, partial differential equation LWR1 ;
LWR2 , which was derived 65 years ago, is mimicking hydrodynamic conservation
laws. It is the simplest model for a macroscopic traffic flow model

$\frac{\partial}{\partial t}\rho(t,x)+\frac{\partial}{\partial x}f(t,x)=0$
(10)

  * •

$\rho$ is the traffic density,

  * •

the traffic flux $f=\rho\times v$ is the product of the traffic density and of
the (mean) traffic speed $v$.

#### Fundamental laws

There are many empirical laws relating $\rho$ and $v$. The Greenshield model
green

$v(\rho)=v_{f}\left(1-\frac{\rho}{\rho_{m}}\right)$ (11)

and the May formula may :

$v(\rho)=v_{f}\exp\left(-\frac{1}{a}\left(\frac{\rho}{\rho_{c}}\right)^{a}\right)$
(12)

for instance are quite popular. We will not try here to define the
corresponding parameters. Note however that they depend on $t$ and $x$. Their
estimation is therefore rather cumbersome.

#### A space discretization for numerical simulations

Figure 1 displays a portion of a motorway in France, with $7$ segments
$S\iota$, $\iota=1,\dots,7$. There are $3$ lanes on $S\iota$,
$\iota=1,\dots,6$, and $4$ on $S_{7}$. The lengths of the various segments are
respectively $4.7,0.6,1.4,1.7,3.7,0.6,0.9$ ([km]). Write $Q_{0}$ (resp.
$Q_{7}$) the inlet (resp. output) flow. The traffic flow on $S\iota$ is
defined by $(Q_{\iota},To_{\iota},V_{\iota})$, where $Q_{\iota}$,
$T{o_{\iota}}$ and $V_{\iota}$ are respectively the flow rate, the occupation
rate and the speed. Our aim is to regulate the flow rates $Q_{11}$, $Q_{12}$,
$Q_{13}$ on the $3$ access ramps in order to ease the traffic flow on the
motorway. The control variables are the green light durations $GD_{\kappa}$,
$\kappa=1,2,3$, on the $3$ on-ramps. The duration of a cycle of the ramp
signals is $40$s. Set $15{\rm s}\leq GD_{\kappa}\leq 29{\rm s}$.

Figure 1: Segments and ramps of motorway portion

The vehicle conservation principle for each segment $\iota$ should be
understood as a space discretization of Equation (10). It reads

$\dot{\rho}_{\iota}=\frac{1}{L_{\iota}}\left(Q_{\iota-1}-Q_{\iota}\right)$

  * •

$\rho_{\iota}$ is the density ([Veh/km]),

  * •

$L_{\iota}$ is the length ([km]),

  * •

$Q_{\iota-1}$ is the inlet flow ([Veh/min]): it is the sum of the upstream
flow and of the ramp flow if any,

  * •

$Q_{\iota}$ is the output flow ([Veh/min]).

Define the _occupancy_ rate ([%]) by

$To_{\iota}=\frac{\rho_{\iota}}{\rho_{\iota,{\rm max}}}\times 100$

where $\rho_{\iota,{\rm max}}=\frac{\lambda_{\iota}}{{\tt size}}$,
$\lambda_{\iota}$ is the number of lanes, ${\tt size}$ is the mean length
([km]) of a vehicle (here $5.5.10^{-3}$).

The fundamental diagrams in Figure 2 relate the (mean) speed ([km/h]) and the
occupancy, i.e., $v_{i}=\mathcal{D}_{\iota}(To_{\iota})$. They are derived
from real data which are, therefore, confidential..

###  3.2 Feedback loops

#### ALINEA without calibration

Equation (9) yields ALINEA in ramp metering. For $\kappa=2$ it reads444The
cases $\kappa=1$ and $\kappa=3$ are similar.

$GD_{2}=-K_{I,2}\int_{0}^{t}(To_{4}-To^{\ast}_{4})d\tau$

where the gain $K_{I,2}\in\mathbb{R}$ is set equal to $1$. A classic anti-
windup setting (see, e.g., astrom ) is mandatory. The _critical_ occupancy
rate $To^{\ast}_{4}$ corresponds to the _critical_ density $\rho_{c}$ in
Formula (12). A “good” real-time calibration of $To^{\ast}_{4}$ seems today
out of reach. We therefore set as in alinea0

$\begin{cases}To^{\ast}_{4}(t+1)=To^{\ast}_{4}(t)+\delta^{+}\text{ if
}V_{4}>V_{4,\text{threshold}}\\\
To^{\ast}_{4}(t+1)=To^{\ast}_{4}(t)-\delta^{-}\text{ if not}\end{cases}$

where

  * •

$V_{4}$ is the mean speed on the 4th segment,

  * •

$V_{4,\text{threshold}}=V_{4,f}-10$,

  * •

$V_{4,f}$ is the _free_ speed, i.e., the maximum speed when the traffic is
light,

  * •

$\delta^{+}=0.15$, $\delta^{-}=0.3$.

#### iP

Equation (1) reads here

$\displaystyle\dot{T}o_{1}$ $\displaystyle=F_{1}+\alpha_{1}GD_{1}$
$\displaystyle\dot{T}o_{4}$ $\displaystyle=F_{2}+\alpha_{2}GD_{2}$
$\displaystyle\dot{T}o_{7}$ $\displaystyle=F_{3}+\alpha_{3}GD_{3}$

The analogous feedback loops of Equation (2) become

$\displaystyle GD_{1}$
$\displaystyle=\frac{\dot{To}_{1}^{\ast}-F_{1}-\mathcal{K}_{1}(To_{1}-To^{\ast}_{1})}{\alpha_{1}}$
$\displaystyle GD_{2}$
$\displaystyle=\frac{\dot{To}_{4}^{\ast}-F_{2}-\mathcal{K}_{2}(To_{4}-To^{\ast}_{4})}{\alpha_{2}}$
$\displaystyle GD_{3}$
$\displaystyle=\frac{\dot{To}_{7}^{\ast}-F_{3}-\mathcal{K}_{3}(To_{7}-To^{\ast}_{7})}{\alpha_{3}}$

Set $\alpha_{j}=30$, $K_{p,j}=0.5$. $To^{\ast}_{\kappa}$, $\kappa=1,2,3$ is
replaced as above.

##  4 Simulations

###  4.1 Generalities

Figures 3 \- 13 display convincing results via the setting of Section 3.2:

  * •

The reference trajectory is a decreasing time function when the traffic is
dense (see Figure 10). This property does not hold anymore if the traffic
becomes fluid.

  * •

Figure 11 shows that the green light duration is set to the whole cycle of 40s
when the queue on the ramp is large (see Figure 12).

  * •

Figure 13 indicates that the output flow on the ramps are reduced when there
is a congestion.

###### Remark 1

Those results are only slightly better than those obtained via ALINEA with the
same calibration of the setpoint. Let us emphasize however that ALINEA when
implemented with the critical density $\rho_{c}$ gives results which are
disapointly similar to those obtained without any control, i.e., without ramp
metering.

###  4.2 What is happening with a sudden change?

Assume only $2$ lanes, instead of $3$, on Segment $S_{2}$: it might be due to
an accident or to some work on the highway. Figures 14 shows that the iP,
without any new tuning, behave much better than ALINEA, especially outside the
congestion hours when ramp metering is more or less useless. Those results
agree with Section 2.3.

##  5 Conclusion

Our approach has already been successfully employed on several French
highways. Its adaptative features explains its excellent robustness with
respect to unexpected events. Its implementation moreover is quite elementary.
There are of course other ramp metering settings which ought to be compared
with our viewpoint, especially now those stemming from artificial
intelligence. Le us notice that _deep reinforcement learning_ methods do not
seem today to perform better than ALINEA (see, e.g., bel ).

The _coordination_ of several ramp-metering actions on highway networks is
being investigated. Coordinated ALINEA techniques have been used in Australia
some time ago coordin . It should be clear that any such coordination ought to
be connected to a _variable speed limit_ (see, e.g., khon ) in order to be
more efficient.

## References

  * (1) Abouaïssa, H., Fliess, M., Iordanova, V., Join, C.: Freeway ramp metering control made easy and efficient. 13th IFAC Symp. Contr. Transport. Syst.., Sofia (2012).
https://hal.archives-ouvertes.fr/hal-00711847/en/

  * (2) Abouaïssa, H., Fliess, M., Join, C.: A fresh look at ramp metering control: ALINEA without any tedious calibration. Res. Report.
https://hal.archives-ouvertes.fr/hal-01117068/en/

  * (3) Abouaïssa, H., Fliess, M., Join, C.: On ramp metering: towards a better understanding of ALINEA via model-free control. Int. J. Contr. 90, 1018–1026 (2017).
  * (4) d’Andréa-Novel, B.: Model-free control of longitudinal and lateral dynamics for automated vehicles. JTEKT Engin. J., 1015E (2018).
  * (5) d’Andréa-Novel, B., Fliess, M., Join, C., Mounier, H., Steux, B.: A mathematical explanation via “intelligent” PID controllers of the strange ubiquity of PIDs. In: 18th Medit. Conf. Contr. Automat., Marrakech (2010). https://hal.inria.fr/inria-00480293/en/
  * (6) Åström, K.J., Murray, R.M.: Feedback Systems: An Introduction for Scientists and Engineers. Princeton University Press (2008).
  * (7) Baciu, A., Lazar, C.: Model-free iPD control design for a complex nonlinear automotive system. 24th Int. Conf. Syst. Theor. Contr. Comput., Sinaia (2020).
  * (8) Barceló, J. (Ed.): Fundamental of Traffic Simulation. Springer (2010).
  * (9) Belletti, F., Haziza, D., Gomes, G., Bayen, A.M.: Expert level control of ramp metering based on multi-task deep reinforcement learning. IEEE Trans. Intel. Transport. Syst. 19, 1198-1207 (2018).
  * (10) Bourbaki, N.: Fonctions d’une variable réelle. Hermann (1976). English translation: Functions of a Real Variable. Springer (1994).
  * (11) Fliess, M.: Analyse non standard du bruit. C.R. Acad. Sci. Paris Ser. I 342, 797–802 (2006).
  * (12) Fliess, M., Join, C.: Model-free control. Int. J. Contr. 86, 2228–2252 (2013).
  * (13) Fliess, M., Join, C.: An alternative to proportional-integral and proportional-integral-derivative regulators: Intelligent proportional-derivative regulators. Int. J. Robust Nonlinear Contr. (2021). https://doi.org/10.1002/rnc.5657
  * (14) Fliess, M., Sira-Ramírez, H.: An algebraic framework for linear identification. ESAIM Contr. Optimiz. Calc. Variat. 9, 151–168 (2003).
  * (15) Fliess, M.,. Sira-Ramírez, H.: Closed-loop parametric identification for continuous-time linear systems via new algebraic techniques. In: Garnier, H., L. Wang (eds) Identification of Continuous-time Models from Sampled Data. Springer, pp. 362–391 (2008).
  * (16) Greenshields, B.D.: A study of traffic capacity. Highway Research Board 14, 448–477 (1935).
  * (17) Haddar, M., Chaari, R., Baslamisli, S.C., Chaari, F., Haddar, M.: Intelligent PD controller design for active suspension system based on robust model-free control strategy. Proc. Inst. Mech. Engin. Part C: J. Mech. Engin. Sci., 233, 4863-4880 (2019).
  * (18) Haj-Salem, H., Blosseville, J.-M., Davée, M.M., Papageorgiou, M.: ALINEA: Un outil de régulation d’accès isolé sur autoroute – Étude comparative sur site réel. Rapport INRETS n∘ 80, Arcueil (1988).
  * (19) Haj-Salem, H., Blosseville, J.-M., Papageorgiou, M.: ALINEA - a local feedback control law for on-ramp metering: a real life study. 3rd IEE Intern. Conf. Road Traffic Control, London (1990).
  * (20) Kachroo, P., Özbay, K.: Feedback Ramp Metering in Intelligent Transportation Systems. Springer (2003).
  * (21) Kachroo, P., Özbay, K.: Feedback Control Theory for Dynamic Traffic Assignment (2nd ed.). Springer (2018).
  * (22) Kerner, B.S.: Introduction to Modern Traffic Flow Theory and Control. Springer (2009).
  * (23) Kerner, B.S. (Ed.): Dynamics of Traffic Management. Springer (2019).
  * (24) Khondaker, B., Kattan, L.: Variable speed limit: an overview, Transport. Lett., 7, 264–278 (2015).
  * (25) Kotsialos, A., Papageorgiou, M., Diakaki, C., Pavlis, Y., Middelham, F.: Traffic flow modeling of large-scale motorway networks using the macroscopic modeling tool METANET. IEEE Trans. Intel. Transport. Syst., 3, 282–292 (2002).
  * (26) Lighthill M.J., Whitham G.,B.: On kinematic waves II. A theory of traffic flow on long crowded roads, Proc. Royal Soc. 229-1178, 317–345 (1955).
  * (27) Liu, X., Xun, J., Ning, B., Wang, C.: Braking process identification of high-speed trains for automatic train stop control. ISA Trans. 111, 171–179 (2021).
  * (28) Mammar, S (Ed.): Systèmes de transport intelligents : modélisation, information et contrôle. Hermes - Lavoisier (2007).
  * (29) May, A.D.: Traffic Flow Fundamentals. Prentice-Hall (1990).
  * (30) Menhour, L., d’Andréa-Novel, B., Fliess, M., Gruyer, D., Mounier, H.: An efficient model-free setting for longitudinal and lateral vehicle control: Validation through the interconnected Pro-SiVIC/RTMaps prototyping platform. IEEE Trans. Intel. Transport. Syst., 19, 461–475 (2018).
  * (31) Milanés, V., Villagrá, J., Godoy, J., González, C.: Comparing fuzzy and intelligent PI controllers in stop-and-go manoeuvres, IEEE Trans. Contr. Syst. Techno., 20, 770-778 (2011).
  * (32) Papageorgiou, M.: Applications of Automatic Control Concepts to Traffic Flow Modeling and Control. Springer (1983).
  * (33) Papageorgiou, M., Blosseville, J.-M., Hadj-Salem, H.: Modelling and real-time control of traffic flow on the southern part of boulevard périphérique in Paris: Part II: Coordinated on-ramp metering. Transport. Res. A 24A, 361–370 (1990).
  * (34) Papageorgiou, M., Blosseville, J.-M., Hadj-Salem, H.: ALINEA: A local feedback control law for on-ramp metering. Transport. Res. Record, 1320, 58–64 (1991).
  * (35) Papamichail, I, Papageorgiou, M, Vong, V, Gaffney, J.: Heuristic ramp-metering coordination strategy implemented at Monash freeway, Australia. Transport. Research Record., 2178, 10–20 (2010).
  * (36) Polack, P., Delprat, S., d’Andréa-Novel, B.: Brake and velocity model-free control on an actual vehicle. Contr. Engin. Pract., 92, 104072 (2019).
  * (37) Richards P.I.: Shock waves on the highways, Operations Research 4, 42–51 (1956).
  * (38) Sira-Ramírez, H., García-Rodríguez, C., Cortès-Romero, J., Luviano-Juárez, A.: Algebraic Identification and Estimation Methods in Feedback Control Systems. Wiley (2013).
  * (39) Treiber, M., Kesting, A.: Verkehrsdynamik und -simulation – Daten, Modelle und Anwendungen der Verkehrsflussdynamik. Springer (2010). English translation: Traffic Flow Dynamics – Data, Models and Simulation. Springer (2013).
  * (40) Villagra, J., Herrero-Pérez, D.: A comparison of control techniques for robust docking maneuvers of an AVG. IEEE Trans. Contr. Syst. Techno., 20, 1116-1123 (2012).
  * (41) Wang, Z., Wang, J.: Ultra-local model predictive control: A model-free approach and its application on automated vehicle trajectory tracking. Contr. Engin. Pract., 101, 104482 (2020).
  * (42) Yang, H., Liu, C., Shi, J., Zheng, G.: Development and control of four-wheel independent driving and modular steering electric vehicles for improved maneuverability limits, SAE Techn. Paper, 2019-01-0459 (2019).
  * (43) Yosida, K.: Operational Calculus (translated from the Japanese). Springer (1984).

(a) $v_{1}=\mathcal{D}_{1}(To_{1})$

(b) $v_{2}=\mathcal{D}_{2}(To_{2})$

(c) $v_{3}=\mathcal{D}_{3}(To_{3})$

(d) $v_{4}=\mathcal{D}_{4}(To_{4})$

(e) $v_{5}=\mathcal{D}_{5}(To_{5})$

(f) $v_{6}=\mathcal{D}_{6}(To_{6})$

(g) $v_{7}=\mathcal{D}_{7}(To_{7})$

Figure 2: Diagrams

(a) Flow in [Veh/min]

(b) Occupancy rate in %

(c) Speed in [km/h]

Figure 3: Section 1

(a) Flow in [Veh/min]

(b) Occupancy rate in %

(c) Speed in [km/h]

Figure 4: Section 2

(a) Flow in [Veh/min]

(b) Occupancy rate in %

(c) Speed in [km/h]

Figure 5: Section 3

(a) Flow in [Veh/min]

(b) Occupancy rate in %

(c) Speed in [km/h]

Figure 6: Section 4

(a) Flow in [Veh/min]

(b) Occupancy rate in %

(c) Speed in [km/h]

Figure 7: Section 5

(a) Flow in [Veh/min]

(b) Occupancy rate in %

(c) Speed in [km/h]

Figure 8: Section 6

(a) Flow in [Veh/min]

(b) Occupancy rate in %

(c) Speed in [km/h]

Figure 9: Section 7

(a) Ramp 1

(b) Ramp 2

(c) Ramp 3

Figure 10: Setpoint time evolution

(a) Ramp 1

(b) Ramp 2

(c) Ramp 3

Figure 11: Time evolution of green lights according to the control inputs
$GD_{1}$, $GD_{2}$, $GD_{3}$

(a) Ramp 1

(b) Ramp 2

(c) Ramp 3

Figure 12: Queue length in [m]

(a) Ramp 1

(b) Ramp 2

(c) Ramp 3

Figure 13: Demand $D_{1},j$ (- -) and flow $Q_{1},j$ (–) in [Veh/min]

(a) TTS

(b) TTP

(c) MS

Figure 14: ALINEA and MFC
(TTS: Travel Time Spent, TTD: Total Travel Distance, MS: Mean Speed)