# Toward Diverse Precondition Generation

Heeyoung Kwon1, Nathanael Chambers2, and Niranjan Balasubramanian1
1Stony Brook University, Stony Brook, New York
2 US Naval Academy, Annapolis, MD
{heekwon<EMAIL_ADDRESS>
<EMAIL_ADDRESS>

###### Abstract

Language understanding must identify the logical connections between events in
a discourse, but core events are often unstated due to their commonsense
nature. This paper fills in these missing events by generating _precondition
events_. Precondition generation can be framed as a sequence-to-sequence
problem: given a target event, generate a possible precondition. However, in
most real-world scenarios, an event can have _several_ preconditions,
requiring diverse generation – a challenge for standard seq2seq approaches. We
propose DiP, a Diverse Precondition generation system that can generate unique
and diverse preconditions. DiP uses a generative process with three components
– an event sampler, a candidate generator, and a post-processor. The event
sampler provides control codes (precondition triggers) which the candidate
generator uses to focus its generation. Unlike other conditional generation
systems, DiP automatically generates control codes without training on diverse
examples. Analysis against baselines reveals that DiP improves the diversity
of preconditions significantly while also generating more preconditions.

##  1 Introduction

Preconditions are an important part of language understanding with numerous
applications, ranging from event understanding to story generation. They
provide the semantic glue to understand (or generate) the chains of events
common in narrative text. How can we build intelligent systems to fill in
these chains, or to identify semantically related events in context? Kwon et
al. (2020) took a first step by introducing a precondition generation task,
where given a target event mention the goal is to generate text that describes
a precondition for the target. They released the ‘PeKo’ dataset for training,
and showed that a GPT-2 model can be fine-tuned on input/output sequence
pairs.

While PeKo is useful, it is constrained by annotating a single relation for
each target event. This is contrast to the real-world where most events have
many preconditions. For example, “opening a door” has several preconditions
like approaching the door, turning a key in the door, and pushing the door.
PeKo’s annotation limits the ability of models to learn to generate multiple
and _diverse_ preconditions111In order to observe diverse preconditions for
the same (or similar) target event we would need a much larger training set..
In this work, we address the challenge of generating more preconditions for
each target event while still maintaining quality.

Generating non-repetitive diverse outputs is a challenge for any conditional
language generation system. Our analysis of the GPT-2 based model shows that
this is also the case for preconditions. Table 1 shows such top preconditions
for an example event. Standard sampling techniques produce high-levels of
lexical and semantic redundancy. In the absence of any explicit mechanisms to
force diversity, the model just produces minor variations of the same event as
preconditions. To obtain diverse candidate preconditions, we have to start
looking lower in the model’s ranked lists of probable preconditions, thereby
sacrificing quality.

TARGET: [BLANK] to fill Mr. Lavelle ’s seat, for a term
---
that expires on Dec. 31, 2008.
The Senate voted overwhelmingly on Thursday
The Senate voted on Wednesday
The Senate voted overwhelmingly on Wednesday
The Senate voted overwhelmingly on Tuesday
Mr. Lavelle was appointed by Gov. Eliot Spitzer
Table 1: Top 5 preconditions generated from GPT-2 with beam search decoding.
Key problem: the top 4 preconditions are almost identical.

How can we induce a model to generate diverse outputs without losing quality?
Context sensitivity might help with quality, but it also hinders diversity. To
address this we introduce a three-stage generative process, which we call DiP.
In the first stage, DiP uses an event sampler whose only goal is to generate
event trigger words as precondition candidates. In the second stage, DiP
forces the generative model to use the candidate triggers from the first stage
to produce the full description of the precondition event. In the third stage,
DiP re-ranks and filters the generated descriptions using a precondition
classifier (also trained from the same training data).222We will release the
source code upon acceptance. A brief example is shown here:

Target Event: I apologized for the debacle of the day before, and [BLANK] to
help me make it right
---
Stage 1 |  Stage 2 |  Stage 3
trying |  I am trying now |  _delete_
use |  I use my time |  _delete_
used |  I used my time |  #2
asked |  asked me |  #3
hired |  hired a new staff |  #1

Experiments on the PeKo dataset show that DiP produces more diverse and better
quality preconditions compared to standard beam decoding, as well as an
iterative filtering extension that applies a standard repetition penalty in a
sampling strategy. Analyses show that DiP is able to better balance the need
for diversity against quality. While the iterative repetition penalty method
generates lexically diverse outputs, it often introduces irrelevant
information rather than producing distinct types of preconditions. Our human
evaluation shows that DiP on the other hand is able to produce text that is
more likely to be preconditions.

All code and data are available at https://stonybrooknlp.github.io/DiP/.

##  2 Related Work

Most work on logical preconditions has focused on identification/extraction
from text. For example, Sil et al. (2010) identified preconditions using a
SVM-based score function with hand-crafted PMI and WordNet based features.
Branavan et al. (2012) extracted domain-specific precondition relations from
instructions for the game of Minecraft. This paper is instead focused on
generating novel preconditions. To the best of our knowledge, only the prior
PeKo work Kwon et al. (2020) has attempted this. We are building on those
initial ideas.

There has been research for diverse generation using control codes or latent
variables. Some works use explicit cues to control text generation. Huang et
al. (2018) used emotion embeddings to generate dialogue responses in a
specific mood. Keskar et al. (2019) trained a LM with human readable control
codes, which describe domain, style, or topics. Then the model learns to
generate text conditioned on a given code. The model requires manually
predefined control codes and a corresponding training corpus for each code.

Other diverse generation works learn latent representations or codes from
input text, and then generate text conditioned on those codes. Shu et al.
(2019) applied a sentence embedding to generate syntactically diverse
translations. They find that syntax-based encoding with TreeLSTM Socher et al.
(2011) yields better diversity than a contextual encoding using BERT Devlin et
al. (2019) or FastText Bojanowski et al. (2017). Bao et al. (2020) used $K$
categorical latent variables to control the generation context of dialogue
responses and pick the highest probability response from the responses
generated using the latent variables. COD3S Weir et al. (2020) is designed to
generate diverse causal relations. It uses locality-sensitive hashing (LSH)
Indyk and Motwani (1998) on representations from Sentence-BERT Reimers and
Gurevych (2019). Conditioning on these 16-bit LSH signatures, it generates
cause/effect sentences using a Transformer architecture Vaswani et al. (2017)
but with a limited vocabulary size of 10K.

These previous approaches have some drawbacks – they either require explicit
control codes and training examples, or they have low interpretability of
their codes. Our approach addresses these two limitations: control codes are
learned from non-diverse input text and the codes are human-readable events.
And these approaches are not directly comparable to our method without proper
modification, which would not be fair comparisons. Thus, we present our own
baselines for evaluation, and these baselines serve as a proxy of ablation
studies as well.

##  3 Diverse Precondition Generation

This section describes our diverse precondition generation task and our
methodology for solving it. Our proposed approach does not require additional
diverse training examples.

###  3.1 Generation Task

This paper follows the precondition definitions from Kwon et al. (2020):

Precondition Definition – "Given a target event mention t and a candidate
event mention p, we assert p is a precondition event for t if p is necessary
for t to happen i.e., t likely would not have occurred without p, in the
current text context."

Precondition Generation – "Given a target event t, generate an event p that is
a precondition for t."

The precondition generation task is defined over sentences that contain both a
target and a precondition event. The precondition part is masked and a model
is asked to reconstruct the sentence including its precondition. For masking,
the syntactic subtree of a precondition is replaced with [BLANK]. In order to
indicate the events of interest – target and precondition – we use special
tokens <event> .. </event> and <pre> .. </pre>.

For our new task, instead of generating the entire sentence, we only generate
a precondition clause that would fit into the input’s [BLANK]. Since a
precondition could be stated in either preceding or succeeding position of its
target event, we modeled this as a text infilling task. This approach is
inspired by Donahue et al. (2020) and this modification allows the model to
focus solely on generating preconditions because the model doesn’t need to
copy over its input text. Thus, the model can learn faster and more
efficiently.

Figure 1: The DiP pipeline. Candidates are generated conditioned on the Event
Sampler. The Re-ranker and Similarity Filter improve quality/diversity.

###  3.2 Diverse Precondition Generator

Generating preconditions is a difficult task even for a single output setting
Kwon et al. (2020). With the training data derived from existing news
articles, generative models only get to see one possible precondition for each
target event. Not surprisingly the top candidates in beam search tend to be
focused towards a specific type of precondition event with minor variations.
This suggests that we need to provide explicit guidance to the model to
explore diverse candidates.

How can we get such diverse guidance? A main strength of large generative
language models is that they learn to generate text that fits with the input
context. If we can get the input context to be less specific then we can aim
to get more general outputs. We can exploit this behavior by training a
separate event sampler that is fed a reduced version of the target event
description. For example, we can denote the target event by just the event
trigger and its arguments. The event sampler learns to predict possible
precondition event triggers based on this reduced context. This task forces
the sampler to learn a more general mapping between target and precondition
events that can produce a diverse set of starting points for generating the
precondition events. We can then train another generative model to condition
on the precondition trigger in addition to the input sentence. This gives us a
model whose outputs we can control by providing different possible
precondition triggers. Not all precondition triggers may yield high quality
preconditions. To further assist the model, we also devise a precondition re-
ranker.

Our overall system, shown in Figure 1, consists of three components – an event
sampler, a candidate generator, and a post processor (Precondition re-ranker
and Similarity filter). The first two stages are used for generation – they
use two separate generation models, and the last is employed to improve the
quality of generated preconditions. We refer to this system as DiP short for
Diverse Preconditions.

####  3.2.1 Event Sampler

The event sampler provides possible precondition event triggers given a target
event. This can be formulated as a sequence to sequence problem where the
input sequence is a target event and the output sequence is a precondition
event trigger. Since our goal here is to get diverse precondition events, we
can experiment with input contexts of different levels of detail. To get more
general precondition events, we use just the target event triggers as the
input. To get more specific preconditions, we can use larger contexts
surrounding the target event trigger as the input. During inference, we sample
top $n$ event triggers based on their probability.

Formally, let $x^{\prime}$ be a subset of the full description $x$ of the
target event. The sampler can be seen as a generative model that outputs event
triggers $e$ for the preconditions of the target event.

$\hat{e}=\arg\max_{e}\log p(e|x^{\prime})$

The generative model is trained to maximize the probability for the correct
precondition trigger $e$ and during inference can be used to sample a
candidate set of top $n$ precondition event triggers.

####  3.2.2 Candidate Generator

The candidate generator, as the name suggests, is a language model that we
fine-tune for generating precondition candidates. We want this model to
generate preconditions corresponding to the triggers from the event sampler.
To this end, in addition to the full target event description $x$, we also
provide a precondition trigger marked by a special token – <E>
precondition_event – at the end of the input. This can be seen as a form of a
control code similar to those used in Keskar et al. (2019); Weir et al.
(2020). The crucial difference, however, is that the codes in our case are
dynamically generated conditioned on the input and not restricted to a
predefined set.

Formally, the candidate generator is a language model that generates a
description of the precondition event $c_{i}$ conditioning on the full
description of the target event $x$ and a given precondition trigger $e_{i}$
from the event sampler.

$\hat{c_{i}}=\arg\max_{c_{i}}\log p(c_{i}|x,e_{i})$

The model is trained to maximize the probability of the observed precondition
text for the target event when provided with the correct precondition trigger.
Note that during training, the precondition trigger provided as input always
appears in the correct precondition description output ($\hat{c_{i}}$). This
encourages the model to learn to incorporate the trigger provided at the end
of the input as part of its output. During inference, the model generates a
set of preconditions one for each of the top $n$ triggers obtained from the
event sampler.

####  3.2.3 Post Processor

Precondition Re-ranker We use a precondition re-ranker to reorder the
generated candidates based on how likely they are to be preconditions of the
target event. Note that the generative model is implicitly trained for a
similar objective. However, the model is also forced to include the input
precondition trigger which could make it harder to focus on ensuring the
result is indeed a precondition. Therefore, we introduce a separate
precondition classifier that scores the generated candidates. Note that the
original PeKo dataset is already setup for training such a classifier Kwon et
al. (2020). Each instance in this dataset consists of an input text that
includes a pair of marked event triggers (target, candidate) and a label that
indicates whether the candidate is a precondition of the event denoted by the
target trigger. The output from the precondition generator is essentially
equivalent to an instance from this dataset. We build a classifier that scores
a pair of events in text, and we use this score as an indicator of the
precondition quality of the generated candidates and re-rank them based on
this score.

Iterative Redundancy Filtering The resulting candidates are a mix of candidate
precondition events from different triggers. To further avoid redundancy we
also include an explicit filtering step, where we post-process the generated
text based on their pairwise similarity. Specifically, we start with the
highest ranked instance in the output set, and iteratively walk down the
ranked list and add instances to the output if the highest similarity score
they have with any of the current output set is lower than a certain
threshold.

##  4 Evaluation

Target Event |  The Metropolitan Transportation Authority recently canceled some large projects
---|---
Trigger Only |  took, canceled, succeeded, began, scheduled, died, rejected, decision, filed, pushed
$\pm 3$ tokens |  planned, took, began, expected, needed, approved, designed, intended, completed, brought
$\pm 5$ tokens |  planned, intended, took, began, aimed, devised, completed, expected, designed, needed
Target Event |  They tried to rebuild their shattered nation
Trigger Only |  took, rebuilt, losing, sustained, lost, opened, came, bought, died, used
$\pm 3$ tokens |  took, lost, war, losing, reached, brought, moved, began, fled, came
$\pm 5$ tokens |  took, lost, war, collapsed, left, failed, abandoned, came, laid, began
Table 2: Top 10 generated event triggers from the event sampler. As more context is provided, the model generate more specific events related to the provided context. Input Text |  Generation Target
---|---
[BLANK] that donations be made to the Crohn’s and Colitis Foundation or NYBOT Futures and Options for Kids in memory of Harry. <E> requests |  In lieu of flowers, the family requests
[BLANK] to start trading an important Nymex product, West Texas intermediate crude oil. <E> inspired |  Nymex’s foray also inspired ICE
Mr. Robbins played hard and fluidly, [BLANK] to give his solos funk and shape. <E> landing |  landing heavily on unexpected notes
Table 3: Examples of training instance pairs for the candidate generator.
Unlike Kwon et al. (2020), we add the precondition event at the end of the
input to help the model utilize the event trigger when generating a
precondition.

Our goal is to investigate the impact of our DiP approach for generating
diverse and high-quality preconditions. We closely follow Kwon et al. (2020)
for the experimental setup and the GPT-2 based generation system for our
evaluation.

###  4.1 Datasets

For the fine-tuning task, we use the precondition generation instances in the
PeKo dataset. In addition, we also create a large additional pre-training
dataset that includes temporal generation instances. With this additional
dataset we can perform a form of domain adaptive pre-training (DAPT)
introduced by Gururangan et al. (2020). The main idea here is to create
generation instances where the model gets to see a target event but now is
required to produce an event that temporally precedes the target event. Since
preconditions are supposed to be temporally preceding this temporal generation
task can be seen as a more permissive yet related generation task, which is
then subsequently restricted to only preconditions in the fine-tuning stage.
We use the CAEVO Chambers et al. (2014) system to obtain temporally related
event pairs from the NYT corpus Sandhaus (2008). This yields 1.1 million
instances and each instance contains one temporal relation (BEFORE/AFTER).
Note that all systems are trained using the same pre-training and fine-tuning
strategy using both datasets.

###  4.2 Baselines

Beam Search As a baseline, we use text infilling GPT-2 system (inspired by
Donahue et al. (2020) with a standard beam search decoding strategy. This beam
search decoder can provide multiple responses up to its beam size. We expect
this simple baseline to contain high-levels of redundancy in its outputs.

Repetition Penalized Sampling (RPS) For a stronger baseline, we use a decoding
strategy that can generate diverse preconditions by penalizing previously
generated precondition event triggers. This is done by an iterative decoding
process applied to the same GPT-2 generation model. Given a target event, the
model generates $k$ preconditions in an iterative manner. When the model
generates a precondition trigger – after <pre> token – a repetition penalty is
applied to deter the model from selecting previously generated precondition
events. We adopt the penalized sampling from Keskar et al. (2019). Instead of
using a list of all generated tokens, we use a list of precondition event
triggers that are generated in the previous iterations. Given a list of
generated precondition events $t$, the probability distribution $p_{i}$ for
the next trigger token $x_{i}$ is defined as:

$\begin{split}p_{i}=\frac{\exp{(x_{i}/I(i\in t))}}{\sum_{j}{\exp(x_{j}/I(j\in
t))}}\\\ I(c)=\lambda\quad\textrm{if $c$ is true else 1}\end{split}$

We set $\lambda=1.2$ as in Keskar et al. (2019). For decoding, we use Nucleus
Sampling Holtzman et al. (2020) which has been claimed to generate a higher
quality of text. Finally, we test the RPS model with the post-processor from
DiP, to confirm that the major gain of DiP is from the Event Sampler.

###  4.3 DiP Model

DiP has three modules – Event sampler, Candidate generator, and Precondition
re-ranker. We train each module separately.

Event sampler We use the GPT-2 model for the event sampler. The model is
trained on the same data instances described in Section 4.1, but instead of
using the entire target-precondition pairs, we use target-precondition event
trigger pairs. We train three event samplers with different levels of context
– trigger only, 3 neighboring tokens, and 5 neighboring tokens – to understand
how different context affect candidate precondition sampling. As Table 2
shows, adding more context help the model to generate more specific events
related to describe situations while the model provides more general events if
only a trigger is given.

Candidate generator The GPT-2 model is also used for the candidate generator.
For training, as described in 3.2, we add <E> precondition_event at the end of
input so that the model can learn how to utilize the provided event trigger as
a control code. Table 3 shows the training examples for the candidate
generator.

Post-processor We train a precondition re-ranker using BERT Devlin et al.
(2019). The F1 score of the classifier is 71.91 with 64.65 of the precision.
To remove possibly redundant preconditions using iterative redundancy
filtering, we need to compute cosine similarity between the generated
preconditions. We take the precondition classifier’s [CLS] token
representation as the embedding for preconditions. Since the similarity score
distributions are different from instance to instance, instead of using a
fixed value as the threshold, we set the threshold as $\mu+\sigma$ of each
instance (the mean and the standard deviation of pairwise similarity scores).
This filters out $\sim$16% of the most similar generated preconditions. For
comparison with the baselines, we take top 10 preconditions from remaining
outputs.

###  4.4 Automatic Evaluation Metrics

We use Self-BLEU Zhu et al. (2018) and Self-BLEURT score to measure the
diversity of generated preconditions. Self-BLEU measures how similar a set of
sentences is to each other using BLEU score – the average of BLEU scores for
the all pairs of sentences in the set. In addition to direct lexical overlap,
we also measure semantic overlap using BLEURT Sellam et al. (2020), which is a
BERT-based learned evaluation metric that is trained on human ratings of
sentence pairs. We refer to this metric as Self-BLEURT. For both metrics, a
lower score implies more diverse preconditions.

###  4.5 Results

We compare the models on both diversity and quality. For diversity, we use an
automatic evaluation, and for quality we used human annotators.

Model Diversity Evaluation
Model Self-BLEU Self-BLEURT Beam Search 0.234 -0.450 RPS 0.016 -1.273
RPS+Post-proc. 0.013 -1.280 DiP 0.038 -1.111

Table 4: Diversity evaluation for different models. We evaluate top 10
preconditions for each model. RPS+Post-proc. produces the most diverse outputs
followed by RPS and DiP with a small margin.

Automatic Diversity Evaluation:

Table 4 shows the diversity metrics for all methods. We evaluated 5,000
preconditions generated for 500 target events. Comparing RPS+Post-proc to RPS,
Post-proc shows little effect, we compare just RPS to DiP in the rest of the
evaluations (See Appendix for more details between RPS and RPS+Post-proc).

In both metrics, DiP and RPS generate more diverse output than the beam search
decoder. DiP is compatible to RPS in shorter preconditions, and RPS produces
more diverse outputs when the generated text gets longer, as shown in Figure
2.

Figure 2: The Self-BLEURT scores across different lengths of generated text.
The numbers indicate the number of instances in each bucket.

Manual Quality Evaluation:

Quality of Preconditions
Model Average Score #Wins RPS 0.954 30 DiP 1.101 56

Table 5: The Top 10 generated preconditions for each target event were scored
on a 0-2 scale. A model "wins" a target if its average is highest. Using
Bootstrapping with $n=1000$ the 95% conf-interval for the RPS mean is (0.89,
1.01) and DiP is (1.05, 1.15).  Figure 3: The number of wins across generation
lengths. DiP wins more as the generations lengthen.

The automatic evaluation only measures diversity. To see if the models
generate legitimate preconditions, we conducted a manual evaluation for
quality. We evaluated 960 generated outputs covering 96 distinct target events
for both DiP and the RPS baseline. For each instance the annotators were
presented with the top ten generated outputs from two systems. For each output
the annotators provided a rating on a scale from 0 to 2, where 0 means not a
precondition, 1 is a maybe, and 2 is definitely a precondition. We split the
96 instances across 8 different annotators333These were computer science
graduate students.

Table 5 shows the results in terms of two metrics: one is the average score
across all 96 instances, and another is the number of "Wins" where a model
gets +1 point if the sum of its 10 precondition scores is higher than the
other. In both metrics, DiP outperforms RPS. Moreover, as shown in Figure 3,
DiP produces better preconditions across most output lengths and is best on
longer outputs.

###  4.6 Analysis

Examples: Table 8 shows the top 5 generations from our main three systems. The
beam search’s failure on the diversity metric is easy to see with its
repetitive output. Most verbs are the same. Both RPS and DiP are notably
better in terms of diversity, but RPS introduces lots of irrelevant
information that may have artificially increased its diversity score. Long
irrelevant phrases are clear to see, and verb synonyms are common. In
contrast, DiP generates more succinct and general preconditions, as well as
fewer direct synonyms.

Context Specificity:  Table 7 shows the diversity scores when different levels
of context are provided to the event sampler. Diversity gets slightly worse
with _more_ context. This aligns with our observation from Table 2 that the
event sampler with more context generates more context-specific precondition
events, which now appear to be closer to each other semantically. This makes
intuitive sense if you view context as closing a model’s view of broader
options.

Errors:  We categorize 4 types of frequently observed errors from DiP. 1) Rare
target events: some target event triggers are polysemious, and some meanings
are rarely used in the newswire. Thus, the event sampler is biased toward
generating precondition events based on more frequent usages of triggers. 2)
Conflict with sentence structure: the candidate generator has no ability to
modify the trigger for its syntactic context because it is trained only as
copying over the event trigger. 3) Re-ranking issue: the re-ranker can induce
errors due to its performance – 64.65 precision. 4) Limited available context:
when the provided context of a target event is too limited, the model often
fails to generate preconditions. Table 6 shows examples for each type of
error.

Rare target events
---
[BLANK] my homework,” and exercising like a demon at a nearby gym. She’d lost
weight and viewed herself as training for a fight.
Event |  Generated Precondition
war |  I war with the other women in my school, who had been working together on
died |  She died of cancer in 1998, after her husband was killed by an electrician who had been working with him”
The word lost is usually used in the context of war, election, or obituary in
the newswire. That may lead the event sampler to fail to generate proper
precondition event triggers.
Conflict with sentence structure
The proposal from Mr. Levy that the Justice Department struck down, she added,
would have violated nondiscrimination provisions of the consent decree and the
Civil Rights Act of 1964 by [BLANK].
Event |  Generated Precondition
created |  created a new state law requiring that a judge be present on the court
called |  called for a ”state level ban on political activity” in New York City
The candidate generator is not able to modify the provided triggers according
to the context.
Re-ranking issue
China’s markets nearly disintegrated in 2005, and one 2003 poll found [BLANK].
Event |  Generated Precondition
became (#4) |  that the market became more popular than in 2000 (#2)
named (>#10) |  that the country was named the world’s largest economy (#4)
Generated sentences are not preconditions but ranked high after re-ranking (#4
$\rightarrow$ #2 and >#10 $\rightarrow$ #4).
Limited available context
[BLANK] to hit a short forehand and guided it wide.
Event |  Generated Precondition
batted |  In the third round, he batted the ball with his left hand
pitched |  In the first inning, the Mets pitched three consecutive hits
The target context is related to tennis, but preconditions are generated in
the context of baseball because the provided context is too limited.
Table 6: Examples from each type of errors. There are 4 types of frequently observed errors and the first 3 types are caused by each stage in DiP. The last one is due to data instances. Context | Self-BLEU | Self-BLEURT
---|---|---
Trigger only | 0.038 | -1.111
$\pm 3$ tokens | 0.039 | -1.103
$\pm 5$ tokens | 0.040 | -1.098
Table 7: Diversity evaluation for different samplers. Precondition candidates
are generated from the event samplers given the input with a trigger only, a
trigger with neighboring 3 tokens, or a trigger with 5 tokens. Target Event:
[BLANK] to maintain below-market rents.
---
Beam Search |  RPS |  DiP
In the last few years, real estate prices have risen and real estate prices have dropped |  Wilhelmina’s stock dropped 8 percent in early 2005 after a lull, but more actively revived the strong dollar, lending to an expected influx of funds |  The city has reached a deal with the tenants
In the last few years, real estate prices have risen and real estate investment trusts have grown |  The City Council’s 10-member City Planning Committee voted 11 to 6 yesterday |  The city will use the money
In the last few years, real estate prices have risen and real estate prices have dropped, but rents have continued |  property values are rising, rising as the vacancy rate is expected |  In the 1980s, the city moved its building to a new site in the East River
In the last few years, real estate prices have risen and real estate investment trusts have grown in size |  In less than a year, such improvements have increased through acquisitions and capitalization at New York City police stations, legal firms and cruise ships, suggesting that housing can be bought |  The City Council passed a bill on Wednesday that would give the city the authority to build a new building at the site of the old Erez subway station, and
In the last few years, real estate prices have risen and real estate prices have dropped, and rents have risen in the last few years |  Matthew Hallico, president of the General Electric Company in Manhattan, and Robert Chrisin, a sales vice partner at Ira G. Albrecht His comments about the incentive package raised many concerns about how it works, as well as what shareholders might do |  In the 1980s, the city began a program
Target Event: By about 10 p.m., the proposals appeared dead for now [BLANK].
Beam Search |  RPS |  DiP
Mr. Spitzer took office |  after the judge, Col Richard Kultura of Thailand, signaled the end of his sentence |  after the commission filed a proposal to provide $ 2 million in new money for the project
Mr. Spitzer took over |  after city Hall learned it would begin public comment on ways it could add 27,000 new jobs to the island |  after the Senate ’s Democratic majority has taken over control of the House
the City Council voted on them |  as they were rejected by legislative leaders |  after the State Legislature put them on a vote
Mr. Spitzer took office in January |  when Mayor Mark Meehan heeded all of his smaller complaints |  when the State Legislature used the budget to cut a $ 2 billion tax break
the City Council passed them to the City Council |  after the State Senate voted yes on key issues |  after a suicide bomber killed a man in an Internet chat room
Target Event: [BLANK] to scout potential recruits.
Beam Search |  RPS |  DiP
The N.F.L. and the N.B.A. have taken steps |  The pending replacement of Carl Crawford has enticed some intelligence officials and top Qaeda leaders |  The police took over the department ’s operations , and they began
The N.F.L. and the N.F.L. have taken steps |  Employees are giving them the opportunity |  The department is sending a new system
The N.C.A.A. has taken steps |  Most Somalis want a law that would enable them |  The New York State Department of Education began a program last year
The N.F.L. and the N.B.A. have taken similar steps |  Shortly after Katrina , Post servicemen were chasing selectors after the storm ’s onset |  The department has sent a handful of officers to the police
The N.F.L. and the N.F.L. have taken similar steps |  Ever since college opened in 1983 , he shopped for school assignments |  The N.C.A.A. set up a task force
Table 8: Top 5 generations from 3 systems. Red cells are invalid
preconditions. Greyed out cells are repetitions from previous cells. DiP
produces both valid _and_ diverse preconditions.

##  5 Conclusion

Real-world events often have multiple preconditions, but today’s datasets do
not, including the latest PeKo, presenting a challenge for text-driven models.
Vanilla generative models have high-levels of redundancy in their outputs and
are thus not well suited for diverse generation. This work introduced an event
sampler that overcomes the issue of target context specificity to provide
diverse guidance to the generator. Coupled with a precondition ranker and
similarity filter, this multi-stage generation setup yields more diverse and
higher quality preconditions. Further, a new training corpus was not required.
More generally, this approach can be seen as an instance of controllable
diverse output generation for conditional language models.

## Acknowledgements

This material is based on research that is supported in part by the Air Force
Research Laboratory (AFRL), DARPA, for the KAIROS program under agreement
number FA8750-19-2-1003 and in part by the National Science Foundation under
the award IIS #2007290.

## References

  * Bao et al. (2020) Siqi Bao, Huang He, Fan Wang, Hua Wu, and Haifeng Wang. 2020.  PLATO: Pre-trained dialogue generation model with discrete latent variable.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 85–96, Online. Association for Computational Linguistics.
  * Bojanowski et al. (2017) Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017.  Enriching word vectors with subword information.  _Transactions of the Association for Computational Linguistics_ , 5:135–146.
  * Branavan et al. (2012) S.R.K. Branavan, Nate Kushman, Tao Lei, and Regina Barzilay. 2012.  Learning high-level planning from text.  In _Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 126–135, Jeju Island, Korea. Association for Computational Linguistics.
  * Chambers et al. (2014) Nathanael Chambers, Taylor Cassidy, Bill McDowell, and Steven Bethard. 2014.  Dense event ordering with a multi-pass architecture.  _Transactions of the Association for Computational Linguistics_ , 2:273–284.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Donahue et al. (2020) Chris Donahue, Mina Lee, and Percy Liang. 2020.  Enabling language models to fill in the blanks.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 2492–2501, Online. Association for Computational Linguistics.
  * Gururangan et al. (2020) Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A. Smith. 2020.  Don’t stop pretraining: Adapt language models to domains and tasks.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 8342–8360, Online. Association for Computational Linguistics.
  * Holtzman et al. (2020) Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020.  The curious case of neural text degeneration.  In _International Conference on Learning Representations_.
  * Huang et al. (2018) Chenyang Huang, Osmar Zaïane, Amine Trabelsi, and Nouha Dziri. 2018.  Automatic dialogue generation with expressed emotions.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers)_ , pages 49–54, New Orleans, Louisiana. Association for Computational Linguistics.
  * Indyk and Motwani (1998) Piotr Indyk and Rajeev Motwani. 1998.  Approximate nearest neighbors: towards removing the curse of dimensionality.  In _Proceedings of the thirtieth annual ACM symposium on Theory of computing_ , pages 604–613.
  * Keskar et al. (2019) Nitish Shirish Keskar, Bryan McCann, Lav R Varshney, Caiming Xiong, and Richard Socher. 2019.  Ctrl: A conditional transformer language model for controllable generation.  _arXiv preprint arXiv:1909.05858_.
  * Kwon et al. (2020) Heeyoung Kwon, Mahnaz Koupaee, Pratyush Singh, Gargi Sawhney, Anmol Shukla, Keerthi Kumar Kallur, Nathanael Chambers, and Niranjan Balasubramanian. 2020.  Modeling preconditions in text with a crowd-sourced dataset.  In _Findings of the Association for Computational Linguistics: EMNLP 2020_ , pages 3818–3828, Online. Association for Computational Linguistics.
  * Loshchilov and Hutter (2019) Ilya Loshchilov and Frank Hutter. 2019.  Decoupled weight decay regularization.  In _International Conference on Learning Representations_.
  * Reimers and Gurevych (2019) Nils Reimers and Iryna Gurevych. 2019.  Sentence-BERT: Sentence embeddings using Siamese BERT-networks.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 3982–3992, Hong Kong, China. Association for Computational Linguistics.
  * Sandhaus (2008) Evan Sandhaus. 2008.  The New York Times Annotated Corpus.
  * Sellam et al. (2020) Thibault Sellam, Dipanjan Das, and Ankur Parikh. 2020.  BLEURT: Learning robust metrics for text generation.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7881–7892, Online. Association for Computational Linguistics.
  * Shu et al. (2019) Raphael Shu, Hideki Nakayama, and Kyunghyun Cho. 2019.  Generating diverse translations with sentence codes.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 1823–1827, Florence, Italy. Association for Computational Linguistics.
  * Sil et al. (2010) Avirup Sil, Fei Huang, and Alexander Yates. 2010.  Extracting action and event semantics from web text.  In _2010 AAAI Fall Symposium Series_.
  * Socher et al. (2011) Richard Socher, Eric Huang, Jeffrey Pennin, Christopher D Manning, and Andrew Ng. 2011.  Dynamic pooling and unfolding recursive autoencoders for paraphrase detection.  _Advances in neural information processing systems_ , 24:801–809.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  In _Advances in neural information processing systems_ , pages 5998–6008.
  * Weir et al. (2020) Nathaniel Weir, João Sedoc, and Benjamin Van Durme. 2020.  COD3S: Diverse generation with discrete semantic signatures.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 5199–5211, Online. Association for Computational Linguistics.
  * Wolf et al. (2020) Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. 2020.  Transformers: State-of-the-art natural language processing.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations_ , pages 38–45, Online. Association for Computational Linguistics.
  * Zhu et al. (2018) Yaoming Zhu, Sidi Lu, Lei Zheng, Jiaxian Guo, Weinan Zhang, Jun Wang, and Yong Yu. 2018.  Texygen: A benchmarking platform for text generation models.  In _The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval_, pages 1097–1100.

##  Appendix A Appendix

###  A.1 Experimental Details

####  A.1.1 Data Split

For the dataset for pre-training, we split into train/dev/test with the ratio
of 8:1:1. For PeKo dataset, we follow the setting from Kwon et al. (2020).

####  A.1.2 Infrastructure

All models are trained using NVIDIA Titan RTX (24GB of GDDR6 VRAM).

####  A.1.3 Parameters

We use Wolf et al. (2020) library for all transformer models. For the beam
search baseline and RPS model we use the GPT-2 architecture, which has
124,445,184 trainable parameters. DiP model consists of two GPT-2 models for
the event sampler and the candidate generator – 2 $\times$ 124,445,184 – and
one BERT model for the re-ranker – 108,313,346 parameters. In total, DiP has
357,203,714 trainable parameters.

Optimizer: We use AdamW Loshchilov and Hutter (2019) for the optimizer across
all models. For pre-trianing, we fix the learning rate as 1e-3. For fine-
tuning, we experiment with [1e-4, 1e-5, 1e-6].

Event sampler: For pre-training, the epochs are set to 100 with the batch size
of 128 for the trigger only, 64 for the $\pm 3$ tokens, and 32 for the $\pm 5$
tokens model. For fine-tuning, the epochs are set to 10 with the batch size of
32.

Candidate generator: For pre-training, the epochs are set to 100 with the
batch size of 16. For fine-tuning, the epochs are set to 10 with the batch
size of 16.

Precondition re-ranker: We use the classifier provided by the authors of Kwon
et al. (2020) – https://stonybrooknlp.github.io/PeKo/.

All Models are picked based on the losses from the dev set.

###  A.2 Comparison between RPS and RPS+Post-processing

Table 9 shows the comparison between RPS and RPS+Post-processing. The effect
of Post-processor on RPS system is considered neutral. There are some cases
where the generation qualities are improved but also other cases where the
qualities are compromised.

Target Event: [BLANK] to maintain below-market rents.
---
RPS |  RPS+Post-processor
Wilhelmina’s stock dropped 8 percent in early 2005 after a lull, but more actively revived the strong dollar, lending to an expected influx of funds |  Mr. Berg last year signed a two-year contract with the firm
The City Council’s 10-member City Planning Committee voted 11 to 6 yesterday |  Another landlord is assembling a plan
property values are rising, rising as the vacancy rate is expected |  Ms. Cianci, 32, is building an intercom tower
In less than a year, such improvements have increased through acquisitions and capitalization at New York City police stations, legal firms and cruise ships, suggesting that housing can be bought |  Her workers hit a brick floor a few years ago
Matthew Hallico, president of the General Electric Company in Manhattan, and Robert Chrisin, a sales vice partner at Ira G. Albrecht His comments about the incentive package raised many concerns about how it works, as well as what shareholders might do |  Last week, Lloyd Pound, the influential Wall Street analyst, gave 75 percent shares of his money
Target Event: By about 10 p.m. , the proposals appeared dead for now [BLANK].
RPS |  RPS+Post-processor
after the judge, Col Richard Kultura of Thailand, signaled the end of his sentence |  after negotiators from both parties reconvened in Davis Park to talk things out
after city Hall learned it would begin public comment on ways it could add 27,000 new jobs to the island |  after the researchers analyzed DEMIC data on children and early adults whose ages began at 8 or 15
as they were rejected by legislative leaders |  after Google released its pie-in-pie template during an extensive public presentation
when Mayor Mark Meehan heeded all of his smaller complaints |  after the Council passed it on Monday
after the State Senate voted yes on key issues |  after the developer, Trivata Films of New Orleans, agreed to pay up to $14 million over seven years
Target Event: [BLANK] to scout potential recruits.
RPS |  RPS+Post-processor
The pending replacement of Carl Crawford has enticed some intelligence officials and top Qaeda leaders |  About the same time, Mr. Booker elicited state financing for another program that provided some of the funds through the Police Department’s National Guard to help workers find mental illness or
Employees are giving them the opportunity |  In 2005 , Mr. SCAD sent students from Iowa and Ohio to visit Johns Hopkins
Most Somalis want a law that would enable them |  In championing the elite classes last week, public school teachers mounted an extensive publicity campaign to persuade parents
Shortly after Katrina , Post servicemen were chasing selectors after the storm ’s onset |  Joel Packer, a Detroit Pistons and assistant coach with Brigham captured a larger campus and invited the scouts
Ever since college opened in 1983 , he shopped for school assignments |  As the trend forward moves into next season , larger colleges are beginning with faculty members from 75 sites on an extensive bioharker scholarship site
Table 9: Top 5 generation examples from RPS and RPS+Post-processor. Green
colored events are considered legitimate preconditions and red colored ones
are not. A red colored cell indicates invalid precondition text. As the
examples show, the effect of Post-processor on RPS system is neutral – in some
cases, the generation qualities are improved but compromised in other cases.

###  A.3 Manual Evaluation

Evaluation Instruction Figure 4 shows the evaluation instruction that we
provided to annotators.

Evaluation rating distribution Table 10 shows the distribution of voted
ratings by annotators. On average, DiP got higher ratings than RPS and RPS got
highest votes in “Not a Precondition.”

Evaluation rating distribution
Model Not a Precond. Maybe Def. a Precond. RPS 38.6% 27.3% 34.1% DiP 29.8%
30.3% 39.9%

Table 10: Evaluation rating distribution. On average, DiP got higher ratings
than RPS. Figure 4: Manual evaluation instruction


# Towards portable muography with small-area, gas-tight glass Resistive Plate
Chambers

S. Basnet, 11footnotetext: Corresponding author E. Cortina Gil  P. Demin
R.M.I.D. Gamage  A. Giammanco  M. Moussawi  M. Tytgat  S. Wuyckens

###### Abstract

Imaging techniques that use atmospheric muons, collectively named under the
neologism “muography”, have seen a tremendous growth in recent times, mainly
due to their diverse range of applications. The most well-known ones include
but are not limited to: volcanology, archaeology, civil engineering, nuclear
reactor monitoring, nuclear waste characterization, underground mapping, etc.
These methods are based on the attenuation or deviation of muons to image
large and/or dense objects where conventional techniques cannot work or their
use becomes challenging.

In this context, we have constructed a muography telescope based on “mini
glass-RPC” planes following a design similar to the glass-RPC detectors
developed by the CALICE Collaboration and used by the TOMUVOL experiment in
the context of volcano radiography, but with smaller active area (16 $\times$
16 cm2). The compact size makes it an attractive choice with respect to other
detectors previously employed for imaging on similar scales. An important
innovation in this design is that the detectors are sealed. This makes the
detector more portable and solves the usual safety and logistic issues for gas
detectors operated underground and/or inside small rooms. This paper provides
an overview on our guiding principles, the detector development and our
operational experiences. Drawing on the lessons learnt from the first
prototype, we also discuss our future direction for an improved second
prototype, focusing primarily on a recently adopted serigraphy technique for
the resistive coating of the glass plates.

##  1 Introduction

First used as a probe to explore the inner part of solid structures in 1955
[2], cosmic muons, which are produced abundantly when primary cosmic rays
interacts with the Earth’s atmosphere, have been employed as an imaging tool
in the subsequent decades. A variety of muon tomography (i.e., muography)
methods have also been developed, exploiting new detection techniques and
numerical algorithms for reconstruction and imaging. These methods are based
on the attenuation or deviation of cosmic muons to image large and/or dense
objects where conventional imaging techniques cannot work or their use becomes
challenging. The steep rise of muography in recent times can be attributed,
for the most part, to its range of applications across many different fields.
These disciplines include but are not limited to: volcanology, archaeology,
civil engineering, nuclear waste monitoring, underground mapping etc [3].

In the last decade, several large muon telescopes with cross-sectional areas
of $\mathcal{O}({1~{}m^{2}})$ have been developed mainly for applications in
volcanology [4, 5], where the size of the object of interest requires the
detectors to be at large distance. However, muography also finds applications
in the imaging of smaller targets, such as nuclear waste monitoring and
archaeology [6, 7]. For the latter case, the point of observation closest to
the target is typically in narrow and confined environments (e.g., tunnels,
underground chambers and crevasses etc.) with no power supply. This imposes
various limitations on the system such as size of the detector, total weight
including electronics, portability, and robustness. Moreover, it also provides
some logistical challenges which include complete autonomy and low power
consumption. Other teams have already been active in developing portable,
robust, and autonomous detectors for the same use cases based on scintillating
bars or fibers, nuclear emulsion, as well as gaseous detectors [3]. Each
detector type actually comes with its own set of benefits and drawbacks.
Plastic scintillators are the most popular ones where spatial, and
consequently angular resolution are not crucial. With the development of
Silicon Photo Multiplier (SiPM) in recent years, robust, scintillator based
muon trackers, suitable for harsh and logistically challenging environments,
can now be designed at a much lower cost and power budget. However, a major
drawback of SiPMs is a notable temperature dependence of the breakdown voltage
(20-30 mV/∘C) that eventually affects gain, dark rate counts, and reverse
current. With no power consumption, a limited cost, and excellent spatial
$\mathcal{O}({1~{}\mu m})$ and angular $\mathcal{O}(1~{}mrad)$ resolutions,
nuclear emulsion films, one of the earliest particle detectors, are perfect
devices for certain muography applications. However, emulsion films cannot be
triggered and start recording particle tracks right away from inception, and
are also affected by variation in temperature and humidity. Another major
drawback is the very specific equipment needed for the offline development and
analysis of the emulsion plates [3].

Various gaseous detectors such as drift chambers [8], micromegas [9],
resistive plate chambers (RPCs) [10], etc. have already been used in muography
with some of these technical solutions being more popular for certain specific
use cases. Here, a lower cosmic-ray flux is particularly advantageous since
high resolution can easily be achieved using fewer, relatively simple
electronic channels. However, there are some issues regarding the operation of
such detectors in applications that are not under supervised laboratory
condition. Since these detectors are normally operated with a continuous gas
flow, gas bottles need to be replenished at the measurement site. Moreover, in
some cases gas mixtures are used including quenchers and flammable components,
posing significant safety issues in all underground applications. In
particular large-area RPCs, while promising in many ways, have some specific
drawbacks: they require a continuous flow of flammable gases, extremely high
voltages ($\sim$ 10 kV) and a stable environmental operating conditions.With
these issues in mind, we started developing a prototype of a portable muon
telescope based on small and gas-tight glass RPCs (gRPCs), focusing mainly on
small targets in contexts of challenging logistics.

##  2 First Prototype and its Operational Experiences

A first muon telescope prototype was developed in 2017-18 based on the design
of the glass-RPC based Semi-Digital Hadronic Calorimeter developed within the
CALICE collaboration [11] and the TOMUVOL project for muography [10]. In
contrast to the large-area gRPCs required by high-energy physics and
volcanology cases, our first prototype is constructed with a much smaller
active area of 16 $\times$ 16 cm2. In order to make the detector more portable
and also to solve the safety and logistic challenge affecting most gaseous
detectors operated underground and/or inside small rooms, a specific guiding
principle for our design is gas tightness. In addition, we also aim at ease of
transportation of the entire set-up including the electronics. More detailed
explanations on the experimental set-up, data collection, analysis and our
experience while operating the prototype are provided in the following
sections and in Ref. [12].

###  2.1 Experimental Setup

Our first prototype consists of four layers of mini-gRPC detectors, as shown
in the left panel of Figure 1. Four removable spacer bars were used to create
a distance of 14.8 cm between second and third plane for better track
reconstruction. Each detector layer weighs 6.5 kg, contains 16 sensitive
strips, with 1 cm pitch and 0.9 cm width, and has gas-gap of 1.1 mm, as can be
seen in the right panel of Figure 1. To achieve bidimensional information, the
first and third detectors are aligned orthogonally to the second and the
fourth. The gas mixture for operating the detector consists of R134a Freon
(95.2%), SF6 (0.3%) and isobutane (4.5%) with a pressure slightly higher than
the atmospheric one.

Figure 1:  (a) Our muon detector consisting of 4 mini-gRPC detector layers.
(b) One of the mini-gRPC detectors inside its casing.

Glass sheets with an area of 20 $\times$ 20 cm2 and thickness of 1.1 mm were
used as electrodes. The outer side of each glass electrode was coated using a
paint roller, with a colloidal dispersion of antimony-doped tin oxide in water
(20% powder and 80% water) mixed with an equal quantity of methanol. To
maintain a uniform gas gap, nine round edge spacers made up of polyether ether
ketone (PEEK) were used between the glass sheets. The front-end electronic
boards (FEB) used for this prototype are spare boards of the current Muon RPC
system of the CMS experiment [13, 14]. Each FEB has four front-end modules
with eight channels each (i.e., one FEB for two detector layers). Each channel
consists of an amplifier with a charge sensitivity of 2 mV/fC, a
discriminator, a monostable and a LVDS driver. The LVDS outputs of all the
FEBs are connected to a System-on-Chip (SoC) module, which is installed on a
carrier board with a wireless connection.

To verify the gas-tightness of the chambers, tests were performed with a
vacuum pump, measuring the rate of leakage after creating vacuum inside the
chamber. Simultaneously, helium injection was used to localize potential
leaking points and the necessary adjustments were made (e.g. the PCB connector
had to be soldered as it was identified as a leaking point). In the end, a
leak rate as low as 10-9 mbar l s-1 was achieved, compared to 10-4 mbar l s-1
before.

###  2.2 Data Collection, Analysis and Results

The working point of the detector was determined by performing two different
scans in high voltage and threshold. From those scans, the optimal working
point was determined to be at a high voltage of 6.6 – 6.7 kV and a threshold
of 105 – 110 (arbitrary units). More details on the procedure are provided in
Ref. [12]. Since the first and last strip in each detector were found to be
very noisy, they were excluded from further analysis. Furthermore, in order to
gather the coincidence data, the muon telescope was operated in a self-
triggering mode, by imposing a logical AND of the four layers.

In the left panel of Figure 2, an illustrative event is shown with the
reconstructed trajectory of a muon giving signal in all four planes. The
distribution of the zenith angle reconstructed in events with exactly one hit
per layer with hits present in each of the four layers after 6 days of data
collection is shown in the right panel of Figure 2.

(a)  (b)

Figure 2:  (a) Typical event showing the fired strips in each detector layer.
X, Y, and Z represent spatial directions in mm. The black segment represents
the reconstructed trajectory of a muon giving signal in the fired strips. (b)
Distribution of reconstructed zenith angle collecting data for 6 days.

In an earlier stage of our project, when only two out of four detectors were
ready, we had an opportunity to test several logistical aspects of our project
in challenging operational outdoor conditions by participating in a two-week
long life-on-Mars simulation at the Mars Desert Research Station (MDRS) in
Utah, USA in March, 2018 [15]. The data collected from this campaign were
suffering from issues (documented in [12]) that were solved only after return
of the detector to our laboratory. However, we were able to verify the gas-
tightness of the system as the pressure in the detector casing did not
decrease significantly ($<0.05$ atm) over a month. Also, the two detectors and
the electronics (with total weight of 37.5 kg) did a successful round trip
from Louvain-la-Neuve (Belgium) to a remote location in the Utah desert,
validating its portability as well as robustness.

##  3 Current Work and Future Direction

###  3.1 Performance Studies

Even though the experience with this first prototype has been encouraging,
there are still various performance tests that need to be carried out with the
current detector before the final design for an improved second prototype can
be agreed upon. Previously, the data was taken without an external trigger,
which made the efficiency and performance studies with the detector
implausible and without having a good handle on the efficiency, pinning down
the spatial resolution of our detector also proved difficult. In order to
perform a reliable study of both the general performance and efficiency of our
mini-gRPC detector, we have now added an external trigger system made up of
two scintillating slabs with dimensions matching the active area of our gRPCs,
placed above and below our telescope. Further work is also being done to
include data from this external trigger in our current trigger and data
acquisition framework. Once all these above-mentioned issues are sufficiently
addressed, we plan to use our current detector to acquire muography data of a
known structure, such as the CP3 building in Louvain-la-neuve.

###  3.2 Resistive Layer Coating Procedure

While the work on the first prototype is still ongoing, preparations for the
second prototype are in their initial phase. For our first prototype, the
resistive coating was applied on the glass plates by hand using paint rollers.
This manual procedure can introduce some undesirable non-uniformity in the
coating quality (layer thickness, resistivity value) across the glass surface,
which is known to adversely affect the performance of gRPCs. For our next
prototype, a technique based on serigraphy is considered for the resistive
layer coating of the glass plates.

(a)  (b)

Figure 3:  (a) Surface resistivity (in M$\Omega$) as a function of conductive
component of the paste (in %). The black data points are used for fitting,
resulting in the solid red line with equation $R=13.94\cdot e^{-1.22}$. (b)
Surface resistivity (in M$\Omega$) measurement for one of the glass plates at
9 different locations, normalized to the highest reading.

Serigraphy, also called screen-printing, is a printing technique where a mesh
is used to transfer paint into a surface, in our case glass plates, except in
the areas that are made impermeable to the paint by blocking the stencil.
Using the CEA facility in Saclay, France, we recently procured glass plates
with uniform resistive layers, to be tested for our next prototype. To apply
coating with serigraphy, the first step is to make the conductive compound,
i.e. the paste, for which a mixture of conductive EDAG PM 404 and the EDAG
6017SS neutral component was used similar to the CALICE SDHCAL gRPCs [16]. The
exact amount of each component in the mixture is determined by varying the
amount of conductive component in the mixture and measuring the subsequent
surface resistivity, which is shown along with an exponential fit in the left
panel of Figure 3. For a surface resistivity of 2 M$\Omega$, $\sim$10.5%
conductive component was used in the mixture for our glass plates. Once the
mixture is made, serigraphy is performed first on a kapton sheet and then on
the glass surface, after which the glass is cured in the oven for 4-5 hours at
180∘C. The measurement of surface resistivity was done at nine different spots
in the glass; a sample measurement is shown in the right panel of Figure 3.

Next to serigraphy, a more common paint spraying technique will also be
studied. Before taking a final decision on the method to be used for the
construction of the second prototype, resistivity measurements of the coated
glass plates will be performed over a longer time period to ensure sufficient
stability.

###  3.3 Detector Readout

Furthermore, in order to perform a high resolution muography, our next
prototype aims at reaching a spatial resolution of $\mathcal{O}({1~{}mm})$ and
timing resolution of $\mathcal{O}({1~{}ns})$, both of which are considered
easily reachable for RPCs. This will require an increase in the number of
readout channels, which in our setup is limited by the capability of our
currently used FEBs. To this end, the use of the MAROC ASIC [17] is currently
being studied. At the same time, the number of readout channels and the
corresponding associated power consumption can be kept down via multiplexing,
which allows to read a large number of strips with as few channels as possible
by exploiting redundancy to compress the information in an unambiguous way. An
example of such method that may be appropriate for our setup is known as
genetic multiplexing [18]. Furthermore, alternative detector layouts are being
studied to decrease the gas volume, and thus also the gas consumption, by
using sealed chambers that include two resistive plates as outer layers.

##  4 Conclusion

A first prototype of a telescope for muography applications to be operated in
confined space was developed based on mini glass-RPC detectors. The system is
intended to be low cost and portable, not only in terms of size and weight but
also with respect to gas tightness and ease of transportation of the full
setup, including electronics. The experience gained has been encouraging, as
we were able to go from design to operation within a short amount of time.
While the performance studies with the current detector are on-going, a new
prototype will also be constructed soon – drawing on the lessons learnt in the
process – with improvements in both the detector layout and fabrication as
well as in the readout electronics.

## Acknowledgments

This work was partially supported by the EU Horizon 2020 Research and
Innovation Programme under the Marie Sklodowska-Curie Grant Agreement No.
822185, and by the Fonds de la Recherche Scientifique - FNRS under Grant No.
T.0099.19. The corresponding author also acknowledges additional research
grants from the FNRS - FRIA. We thank Dr. Stephan Aune and his serigraphy team
at CEA, Saclay for helping us with resitive coating. We also thank the
electronics group at the Center for Cosmology, Particle Physics, and
Phenomenology (CP3), Université catholique de Louvain, for their help in
setting up the external scintillator trigger.

## References

  * [1]
  * [2] E. P. George, _Cosmic rays measure overburden of tunnel_ , _Commonw. Eng._ , (1955) 455-457.
  * [3] L. Bonechi, R. D’Alessandro, A. Giammanco, _Atmospheric muons as an imaging tool_ , _Reviews in Physics 5_ , (2020) 100038 . https://doi.org/10.1016/j.revip.2020.100038
  * [4] G. Saracino, C. Carloganu, _Looking at volcanoes with cosmic-ray muons_ , _Phys. Today_ 65, 12, (2012) 60. http://dx.doi.org/doi:10.1063/PT.3.1829
  * [5] H. Tanaka, _Japanese volcanoes visualized with muography_ , _Phil. Trans. R. Soc. A_ , 377, 2137, (2018) 20180142. https://doi.org/10.1098/rsta.2018.0142
  * [6] G. Saracino et al., _Imaging of underground cavities with cosmic-ray muons from observations at Mt. Echia (Naples)_ , _Sci. Rep._ , 7, (2017) 1181. https://doi.org/10.1038/s41598-017-01277-3
  * [7] L. Bonechi et al., _A possible point of contact between cosmic ray physics and archaeology: muon absorption radiography at the Tharros Phoenician-Roman site_ , _Ann. Geophys._ , 60, 1, (2017) S0107. https://doi.org/10.4401/ag-7382
  * [8] C. L. Morris, J. Bacon et al. _Horizontal cosmic ray muon radiography for imaging nuclear threats_ , _Nucl. Inst. Methods B_ , 330, (2014) 42-46. https://doi.org/10.1016/j.nimb.2014.03.017
  * [9] S. Procureur et al. _A Micromegas-based telescope for muon tomography: The WatTo experiment_ , _Nucl. Inst. Methods A_ , 834, (2016) 223-228. https://doi.org/10.1016/j.nima.2016.08.002
  * [10] C. Cârloganu (TOMUVOL), _Density imaging of volcanoes with atmospheric muons using GRPCs_ , _International Europhysics Conference on High Energy Physics - HEP_ , (2011) in2p3-00642747. http://hal.in2p3.fr/in2p3-00642747
  * [11] V. Buridon et al., _First results of the CALICE SDHCAL technological prototype._ , _JINST_ , 11, (2016) P04001. https://doi.org/10.1088/1748-0221/11/04/P04001
  * [12] S. Wuyckens, A. Giammanco, E. Cortina Gil, P. Demin, _A portable muon telescope based on small and gas-tight resistive plate chambers_ , _Phil. Trans. R. Soc. A_ , 377, (2018) 20180139. http://dx.doi.org/10.1098/rsta.2018.0139
  * [13] M. Abbrescia et al., _New developments on front-end electronics for the CMS resistive plate chambers_ , _Nucl. Inst. Methods A_ , 456, 1-2 (2000) 143-149. https://doi.org/10.1016/S0168-9002(00)00980-3
  * [14] C. Binetti, R. Liuzzi, F. Loddo, B. Marangelli, A. Ranieri, _A new Front-End Board for RPC detector of CMS_ , _CERN-CMS-NOTE-1999-047_ (1999). http://cds.cern.ch/record/687142
  * [15] UCL to Mars, https://ucltomars.org/
  * [16] M. Bedjidian et al. _Performance of Glass Resistive Plate Chambers for a high-granularity semi-digital calorimeter_ , _JINST_ , 6, (2011) P02001. https://doi.org/10.1088/1748-0221/6/02/P02001
  * [17] https://www.weeroc.com/products/photomultipliers-read-out/maroc-3a
  * [18] S. Procureur, R. Dupré, S. Aune, _Genetic multiplexing and first results with a $50\times 50~{}cm^{2}$ Micromegas_, _Nucl. Inst. Methods A_ , 729, (2013) 888-894. https://doi.org/10.1016/j.nima.2013.08.071