# Polly Want a Cracker: Analyzing Performance of Parroting on Paraphrase
Generation Datasets

Hongren Mao, Hung-yi Lee
Department of Electrical Engineering
National Taiwan University
<EMAIL_ADDRESS><EMAIL_ADDRESS>

###### Abstract

Paraphrase generation is an interesting and challenging NLP task which has
numerous practical applications. In this paper, we analyze datasets commonly
used for paraphrase generation research, and show that simply parroting input
sentences surpasses state-of-the-art models in the literature when evaluated
on standard metrics. Our findings illustrate that a model could be seemingly
adept at generating paraphrases, despite only making trivial changes to the
input sentence or even none at all.

##  1 Introduction

The task of paraphrase generation has many important applications in NLP. It
can be used to generate adversarial examples of input text, which can then be
used to train neural networks so that they become less susceptible to
adversarial attack Iyyer et al. (2018). For knowledge-based QA systems, a
paraphrasing step can produce multiple variations of a user query and match
them with knowledge base assertions, enhancing recall Yin et al. (2015); Fader
et al. (2014). Relation extraction can also benefit from incorporating
paraphrase generation into its processing pipeline Romano et al. (2006).
Manually annotating translation references is expensive, and automatically
generating references through paraphrasing has been shown to be effective for
evaluation of machine translation Zhou et al. (2006); Kauchak and Barzilay
(2006).

Datasets used for paraphrase generation include
Quora111https://data.quora.com/First-Quora-Dataset-Release-Question-Pairs,
Twitter Lan et al. (2017) and MSCOCO Lin et al. (2014). Previous work on
paraphrase generation that used these datasets Wang et al. (2019); Gupta et
al. (2018); Li et al. (2018); Prakash et al. (2016) chose BLEU Papineni et al.
(2002), METEOR Lavie and Agarwal (2007) and TER Snover et al. (2006) as
evaluation metrics.

In this paper, we find that simply using the input sentence as output in an
unsupervised manner (i.e. fully parroting the input) significantly outperforms
the state-of-the-art on two metrics for Twitter, and on one metric for Quora.
Even after changing part of the input sentence (i.e. partially parroting the
input), state-of-the-art metric scores can still be surpassed.

Consequently, for future paraphrase generation research which achieve good
evaluation scores, we suggest investigating whether their methods or models
act differently from simple parroting behavior.

##  2 Method Description

Given an input sentence $i$, the goal of paraphrase generation is to generate
an output sentence $o$ which is semantically identical to $i$, but contain
variations in lexicon or syntax. Full parroting simply uses the input as
output ($o=i$).

Paraphrase generation models may not parrot the input sentence word for word,
but it is possible that they only modify a few words of the input, thus we
also experiment with simple methods of modifying $i$, such as replacing or
cutting words from the head, from the tail or from random positions.

Both full parroting and the forms of partial parroting we use are fully
unsupervised.

##  3 Datasets

Quora. The Quora dataset contains 149,263 paraphrase sentence pairs (positive
examples) and 255,027 non-paraphrase sentence pairs (negative examples).
Having both positive and negative examples makes it appealing for research on
paraphrase generation Gupta et al. (2018); Li et al. (2018) and identification
Lan and Xu (2018). After processing the dataset, there are 149,650 unique
sentences that have reference paraphrases.

Gupta et al. Gupta et al. (2018) sampled 4K sentences as their test set, but
did not specify which sentences they used. Li et al.Li et al. (2018) sampled
30K sentences as their test set, also not specifying which sentences they
used. To avoid selecting a subset of data that is biased in favor of our
method, we perform evaluation on the entire Quora dataset. Although we
evaluate on the entire dataset, the size of our training set is zero due to
the fully unsupervised nature of full and partial parroting.

We group sentences by the number of reference paraphrases they have, and plot
the relative counts in Appendix A. It can be seen that over 64% of entries
have only a single reference paraphrase, which is problematic because even if
a paraphrase of good quality is generated for any one of these entries, BLEU,
METEOR and TER scores could still be inferior if the generated paraphrase
differs too much from the single reference paraphrase. Previous paraphrase
generation work on Quora Gupta et al. (2018); Li et al. (2018) did not mention
removing these entries, thus we include them in our experiments for fair
comparison. However, we strongly recommend future work which wishes to use
BLEU, METEOR and TER as evaluation metrics to only consider entries that have
multiple reference paraphrases.

Twitter. There are 114,025 paraphrase sentence pairs in Twitter, which were
acquired by collecting tweets which contain identical URLs Lan et al. (2017).
As with Quora, prior paraphrase generation work on this dataset Li et al.
(2018) did not provide their sampled test set sentences, so we evaluate
parroting on the entire dataset to avoid bias. We follow the same data
processing steps as Quora, and plot the number of reference paraphrases in
Appendix A.

MSCOCO. This is an image captioning dataset, with multiple captions provided
for a single image Lin et al. (2014). There have been multiple works which use
it as a paraphrase generation dataset by treating captions of the same image
as paraphrases Wang et al. (2019); Gupta et al. (2018); Prakash et al. (2016).
The training and testing sets are available, containing 331,163 and 162,016
input sentences respectively.

However, relevance scores for captions of the same image score only 3.38 out
of 5 under human evaluation (in contrast, the score is 4.82 for Quora) Gupta
et al. (2018), due to the fact that different captions for the same image
often vary in the semantic information conveyed. This makes the use of MSCOCO
as a paraphrase generation dataset questionable.

We plot the number of reference paraphrases in Appendix A.

##  4 Experiments

| STATE-OF-THE-ART | PARROT
---|---|---
Metric | paper | score | num_train | score | num_train |  $\Delta$SOTA
BLEU $\uparrow$ | Li et al. (2018) | 43.54 | 100K | 41.59 | 0 | -4.47%
METEOR $\uparrow$ | Gupta et al. (2018) | 33.6 | 150K | 38.60 | 0 | +14.88%
TER $\downarrow$ | Gupta et al. (2018) | 39.5 | 150K | 45.22 | 0 | +14.47%
Table 1: Performance of full parroting v.s. state-of-the-art on Quora. Higher BLEU and METEOR scores are better, while higher TER scores are worse. Bold text represents best results. | STATE-OF-THE-ART | PARROT
---|---|---
Metric | paper | score | num_train | score | num_train |  $\Delta$SOTA
BLEU $\uparrow$ | Li et al. (2018) | 45.74 | 110K | 65.26 | 0 | +42.67%
METEOR $\uparrow$ | Li et al. (2018) | 20.18 | 110K | 41.73 | 0 | +106.77%
TER $\downarrow$ | - | - | - | 41.87 | 0 | -
Table 2: Performance of full parroting v.s. state-of-the-art on Twitter. | STATE-OF-THE-ART | PARROT
---|---|---
Metric | paper | score | num_train | score | num_train |  $\Delta$SOTA
BLEU $\uparrow$ | Wang et al. (2019) | 44.0 | 331K | 19.0 | 0 | -56.8%
METEOR $\uparrow$ | Wang et al. (2019) | 34.7 | 331K | 23.9 | 0 | -31.0%
TER $\downarrow$ | Wang et al. (2019) | 37.1 | 331K | 70.4 | 0 | +89.7%
Table 3: Performance of full parroting v.s. state-of-the-art on MSCOCO.

We evaluate the performance of full parroting on all three datasets and
compare with state-of-the-art models.

We also study the performance of partial parroting. Whereas full parroting
does not modify the input sentence, partial parroting replaces or cuts some of
the input words. We try three different modes of choosing words to be cut or
replaced: from the sentence head, from the tail or sampled randomly.

###  4.1 Evaluation

Following prior paraphrase generation research which used Quora, Twitter and
MSCOCO, we use BLEU, METEOR and TER as evaluation metrics. When calculating
metric scores, all available reference paraphrases for a given input sentence
are considered.

###  4.2 Results

Full parroting. Our results are organized in Tables 1, 2 and 3. We see for
Twitter, parroting outperforms the state-of-the-art by significant margins on
both BLEU and METEOR scores; for Quora, parroting outperforms the state-of-
the-art appreciably on METEOR while having comparable performance on BLEU.

The poor performance of full parroting on MSCOCO is due to higher edit
distances between input sentences and their reference paraphrases. TER
measures the edit distance of a sentence to a reference sentence, normalized
by the average length of all references Snover et al. (2006):

$\text{TER}=\frac{\text{\\# of edits}}{\text{average \\# of reference words}}$

We see that the TER score of full parroting is particularly high on MSCOCO
compared to the other two datasets. Correspondingly, the BLEU and METEOR
scores are lower by a wide margin.

Figure 1: Metric scores v.s. number of reference paraphrases (Quora). For
lower numbers of references, metric scores improve as the amount of references
increases. For higher numbers of references, there does not appear to be a
clear correlation. Figure 2: Metric scores v.s. number of reference
paraphrases (Twitter). For lower numbers of references, metric scores improve
as the amount of references increases. For higher numbers of references, there
does not appear to be a clear correlation.

For further investigation of parroting on Quora and Twitter, we plot parroting
performance versus the number of reference paraphrases available for a given
input sentence (Figures 1 and 2). If the number of references is not too high,
metric scores generally improve when the number of references rises. Once the
number of references exceeds a certain threshold, we do not observe a clear
correlation, showing that the probability of finding a reference sentence
which bears higher resemblance to the input does not increase proportionally
with the number of references.

|  Quora (5K test set $\times$ 1200) |  Twitter (4K test set $\times$ 250)
---|---|---
Statistic | BLEU $\uparrow$ | METEOR $\uparrow$ | TER $\downarrow$ | BLEU $\uparrow$ | METEOR $\uparrow$ | TER $\downarrow$
Average | 41.57 | 38.59 | 45.21 | 74.97 | 46.22 | 33.43
Std. Dev. | 0.50 | 0.20 | 0.50 | 0.36 | 0.14 | 0.35
Max. | 43.12 | 39.29 | 46.95 | 76.01 | 46.56 | 34.31
Min. | 39.98 | 37.85 | 43.53 | 74.01 | 45.89 | 32.41
Table 4: Performance of full parroting on randomly sampled test sets. The test
set size and sampling method is the same as that described in prior state-of-
the-art work. Here, scores for sampled Quora test sets are similar to those of
the full dataset, and the scores for Twitter test sets are better than scores
achieved on the full dataset.

The choice of testing on the entire dataset for Quora and Twitter experiments
was to avoid bias in favor of parroting. Nevertheless, we also randomly
sampled test sets of size 4K for Quora in the same manner as Gupta et al.
(2018) (which holds the most state-of-the-art records on Quora) and test sets
of size 5K for Twitter in the same manner as Li et al. (2018) (which holds all
state-of-the-art records on Twitter). In total, 1200 test sets of size 4K were
sampled for Quora and 250 test sets of size 5K were sampled for Twitter.
Parroting performance on these sampled test sets can be found in Table 4. It
can be observed that the average metric scores for Quora are similar to the
scores in Table 1, whereas the average scores for Twitter are noticeably
better than those in Table 2. Furthermore, the score deviation between
different samples is small. Consequently, although the exact test sets used by
Gupta et al. (2018) and Li et al. (2018) are not available, it is logical to
assume that parroting performance would still exceed or be on par with the
state-of-the-art on those test sets.

Figure 3: Metric scores v.s. ratio of text that is cut from the start of the
input sentence (Quora). Figure 4: Metric scores v.s. ratio of text that is cut
from the start of the input sentence (Twitter).

Partial parroting. We also introduce lexical variation into our parroting
method by replacing or cutting words of the input sentence. For replacement,
we substitute input words with an out-of-vocabulary word not found in any of
the input sentence’s reference paraphrases. Paraphrase generation models are
usually allowed to generate words which exist in reference paraphrases; we
purposely use out-of-vocabulary words to give harsher scores to our method.

Figures 3 and 4 show performance of cutting words from the start of input
sentences. For Quora, when over 10% of the input sentence has been modified by
being cut off, partial parroting underperforms the state-of-the-art by only
3.8% on METEOR. For Twitter, the same form of partial parroting (cutting off
words) still outperforms the state-of-the-art on BLEU when input sentences are
modified by 42% , and does the same on METEOR when the input is modified by
56%. Additionally, we experiment with cutting words in other positions, and
also replace words rather than cut them away. The results can be found in
Appendix B.

Earlier work using Quora and Twitter Gupta et al. (2018); Li et al. (2018)
only provided a few examples of output paraphrases, and did not study in
detail the paraphrasing behavior of their models, making it unclear whether
the models achieve qualitatively better results than our simple rule-based
parroting techniques, given that evaluation scores of the two are similar. We
recommend future research to perform such an analysis if their metric scores
are close to that of parroting.

##  5 Related Work

For the task of paraphrase generation, Wang et al. Wang et al. (2019) trained
a Transformer network on MSCOCO; Gupta et al. Gupta et al. (2018) trained a
seq2seq variational autoencoder (VAE) on Quora and MSCOCO; Li et al. Li et al.
(2018) trained a seq2seq pointer network on Quora and Twitter, then fine-tuned
it with an evaluator which was trained via inverse reinforcement learning;
Prakash et al. Prakash et al. (2016) trained a seq2seq model with residual
connections on MSCOCO.

Work on paraphrase generation using other datasets can also be found. Methods
include lexical substitution Hassan et al. (2007); Bolshakov and Gelbukh
(2004), back-translation Wieting and Gimpel (2018) and sequence-to-sequence
neural networks Iyyer et al. (2018).

It is worth noting that paraphrase generation serves practical purposes, such
as augmenting training data for NLP models to decrease their susceptibility to
adversarial attack Iyyer et al. (2018), or enhancing recall for QA systems Yin
et al. (2015); Fader et al. (2014). Improvement of downstream model
performance is a valid evaluation metric for paraphrase generation, and future
work wishing to use Quora entries which only have a single reference
paraphrase could choose such an evaluation metric instead of BLEU, METEOR or
TER.

As a sidenote, we also ran experiments in which BLEU scores were calculated
using non-reference dataset sentences. The results are in Appendix C.

##  6 Conclusion

In this work, we discover that various forms of simple parroting outperforms
state-of-the-art results on QUORA and TWITTER when evaluated using BLEU and
METEOR. An interpretation is that current models could simply be parroting
input sentences, and researchers should perform qualitative analysis of such
behavior. Another interpretation is that BLEU and METEOR are inappropriate for
evaluating paraphrase generation models, in which case other metrics such as
effectiveness of data augmentation Iyyer et al. (2018), may be used instead.

## References

  * Bolshakov and Gelbukh (2004) Igor A. Bolshakov and Alexander Gelbukh. 2004.  Synonymous paraphrasing using wordnet and internet.  In _Natural Language Processing and Information Systems_ , pages 312–323, Berlin, Heidelberg. Springer Berlin Heidelberg.
  * Fader et al. (2014) Anthony Fader, Luke Zettlemoyer, and Oren Etzioni. 2014.  Open question answering over curated and extracted knowledge bases.  In _Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining_ , KDD ’14, pages 1156–1165, New York, NY, USA. ACM.
  * Gupta et al. (2018) Ankush Gupta, Arvind Agarwal, Prawaan Singh, and Piyush Rai. 2018.  A deep generative framework for paraphrase generation.  In _AAAI_.
  * Hassan et al. (2007) Samer Hassan, Andras Csomai, Carmen Banea, Ravi Sinha, and Rada Mihalcea. 2007.  Unt: Subfinder: Combining knowledge sources for automatic lexical substitution.  In _Proceedings of the Fourth International Workshop on Semantic Evaluations (SemEval-2007)_ , pages 410–413. Association for Computational Linguistics.
  * Iyyer et al. (2018) Mohit Iyyer, John Wieting, Kevin Gimpel, and Luke Zettlemoyer. 2018.  Adversarial example generation with syntactically controlled paraphrase networks.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pages 1875–1885. Association for Computational Linguistics.
  * Kauchak and Barzilay (2006) David Kauchak and Regina Barzilay. 2006.  Paraphrasing for automatic evaluation.  In _Proceedings of the Main Conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics_ , HLT-NAACL ’06, pages 455–462, Stroudsburg, PA, USA. Association for Computational Linguistics.
  * Lan et al. (2017) Wuwei Lan, Siyu Qiu, Hua He, and Wei Xu. 2017.  A continuously growing dataset of sentential paraphrases.  In _EMNLP_.
  * Lan and Xu (2018) Wuwei Lan and Wei Xu. 2018.  Neural network models for paraphrase identification, semantic textual similarity, natural language inference, and question answering.  In _Proceedings of the 27th International Conference on Computational Linguistics (COLING)_.
  * Lavie and Agarwal (2007) Alon Lavie and Abhaya Agarwal. 2007.  Meteor: An automatic metric for mt evaluation with high levels of correlation with human judgments.  In _Proceedings of the Second Workshop on Statistical Machine Translation_ , StatMT ’07, pages 228–231, Stroudsburg, PA, USA. Association for Computational Linguistics.
  * Li et al. (2018) Zichao Li, Xin Jiang, Lifeng Shang, and Hang Li. 2018.  Paraphrase generation with deep reinforcement learning.  In _EMNLP_.
  * Lin et al. (2014) Tsung-Yi Lin, Michael Maire, Serge J. Belongie, Lubomir D. Bourdev, Ross B. Girshick, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. 2014.  Microsoft coco: Common objects in context.  In _ECCV_.
  * Papineni et al. (2002) Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002.  Bleu: A method for automatic evaluation of machine translation.  In _Proceedings of the 40th Annual Meeting on Association for Computational Linguistics_ , ACL ’02, pages 311–318, Stroudsburg, PA, USA. Association for Computational Linguistics.
  * Prakash et al. (2016) Aaditya Prakash, Sadid A. Hasan, Kathy Lee, Vivek Datla, Ashequl Qadir, Joey Liu, and Oladimeji Farri. 2016.  Neural paraphrase generation with stacked residual lstm networks.  In _Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers_ , pages 2923–2934. The COLING 2016 Organizing Committee.
  * Romano et al. (2006) Lorenza Romano, Milen Kouylekov, Idan Szpektor, Ido Dagan, and Alberto Lavelli. 2006\.  Investigating a generic paraphrase-based approach for relation extraction.  In _11th Conference of the European Chapter of the Association for Computational Linguistics_.
  * Snover et al. (2006) Matthew Snover, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul. 2006.  A study of translation edit rate with targeted human annotation.  In _In Proceedings of Association for Machine Translation in the Americas_ , pages 223–231.
  * Wang et al. (2019) Su Wang, Rahul Gupta, Nancy Chang, and Jason Baldridge. 2019.  A task in a suit and a tie: paraphrase generation with semantic augmentation.  In _AAAI_.
  * Wieting and Gimpel (2018) John Wieting and Kevin Gimpel. 2018.  Paranmt-50m: Pushing the limits of paraphrastic sentence embeddings with millions of machine translations.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 451–462. Association for Computational Linguistics.
  * Yin et al. (2015) Pengcheng Yin, Nan Duan, Ben Kao, Junwei Bao, and Ming Zhou. 2015.  Answering questions with complex semantic constraints on open knowledge bases.  In _Proceedings of the 24th ACM International on Conference on Information and Knowledge Management_ , CIKM ’15, pages 1301–1310, New York, NY, USA. ACM.
  * Zhou et al. (2006) Liang Zhou, Chin-Yew Lin, and Eduard H. Hovy. 2006.  Re-evaluating machine translation results with paraphrase support.  In _EMNLP_.

##  Appendix A Number of References Paraphrases for Datasets

Figure 5: Number of reference paraphrases v.s. percentage of Quora dataset

Figure 6: Number of reference paraphrases v.s. percentage of Twitter dataset

Figure 7: Number of reference paraphrases v.s. percentage of MSCOCO dataset

##  Appendix B Performance of Partial Parroting

Figure 8: Metric scores v.s. ratio of text that is cut from the end of the
input sentence (Quora)

Figure 9: Metric scores v.s. ratio of text that is cut from the end of the
input sentence (Twitter)

Figure 10: Metric scores v.s. ratio of text that is cut randomly from the
input sentence (Quora)

Figure 11: Metric scores v.s. ratio of text that is cut randomly from the
input sentence (Twitter)

Figure 12: Metric scores v.s. ratio of text that is replaced from the start of
the input sentence (Quora)

Figure 13: Metric scores v.s. ratio of text that is replaced from the start of
the input sentence (Twitter)

Figure 14: Metric scores v.s. ratio of text that is replaced from the end of
the input sentence (Quora)

Figure 15: Metric scores v.s. ratio of text that is replaced from the end of
the input sentence (Twitter)

Figure 16: Metric scores v.s. ratio of text that is replaced randomly from the
input sentence (Quora)

Figure 17: Metric scores v.s. ratio of text that is replaced randomly from the
input sentence (Twitter)

##  Appendix C Calculating BLEU score using non-reference sentences

For an input sentence, BLEU scores are usually calculated by comparing the
input sentence with a number of reference sentences. We ran experiments in
which 5 reference and 100 randomly sampled non-reference sentences were used,
and show part of our results below. It can be seen that sentences with higher
BLEU scores are more similar to the input sentence, which is to be expected.

| Input sentence: what would happen if you hired two private detectives to spy on each other ? |  |  |
---|---|---|---|---
BLEU |  |  |  |
0.3 - 0.35 |  | what will happen if i hire two private detectives to follow each other ?
---
what would happen if i got two private investigators to follow each other ?
what would happen if i sent two private investigators to find each other ?
|  |
0.25 - 0.3 | (None) |  |  |
0.2 - 0.25 |  | what would happen if earth collided with a black hole ?
---
what if i hired two private eyes and ordered them to follow each other ?
what would happen if donald trump lost and refused to concede the election ?
|  |
0.15 - 0.2 |  | would i be able to hire two private investigators and then get them to follow each other ?
---
what would happen if donald trump turned out to be a plant for hillary to win
the white house ?
|  |
0 - 0.15 |  | do i need to pay again on coursera if i switch sessions ?
---
is there anyway to tell if someone blocked you on facebook ?
what song do you listen to when you are angry ?
……
what are bugs you noticed on quora ?
if i eat a pot cookie , how long until i ’m able to pass a urine test ?
|  |
| Input sentence: who do you think portrayed batman better : christian bale or ben affleck ? |  |  |
---|---|---|---|---
BLEU |  |  |  |
0.4 - 0.45 | according to you , whose batman performance was best : christian bale or ben affleck ? |  |  |
0.35 - 0.4 | (None) |  |  |
0.3 - 0.35 | no fanboys please , but who was the true batman , christian bale or ben affleck ? |  |  |
0.25 - 0.3 | (None) |  |  |
0.2 - 0.25 | (None) |  |  |
0.15 - 0.2 |  | what do you think about ” chinese dream ” ?
---
what do you think about dota2 ?
who was better as batman : bale or affleck ?
did ben affleck shine more than christian bale as batman ?
do you think that the demonetization policy will backfire for bjp in 2019
elections ?
|  |
0 - 0.15 |  | biswapati sarkar : how do you overcome a writer ’s block ?
---
who is the better batman ? affleck or bale ?
how do you stop your cat from spraying ?
……
do we always get what we deserve ?
can a moon have a moon ?
|  |


# Test beam characterization of sensor prototypes for the CMS Barrel MIP
Timing Detector

R. Abbott  A. Abreu  F. Addesa  M. Alhusseini  T. Anderson  Y. Andreev  A.
Apresyan  R. Arcidiacono  M. Arenton  E. Auffray  D. Bastos  L.A.T. Bauerdick
R. Bellan  M. Bellato  A. Benaglia  M. Benettoni  R. Bertoni  M. Besancon  S.
Bharthuar  A. Bornheim  E. Brücken  J.N. Butler  C. Campagnari  M. Campana  R.
Carlin  P. Carniti  N. Cartiglia  M. Casarsa  O. Cerri  P. Checchia  H. Chen
S. Chidzik  F. Chlebana  F. Cossutti  M. Costa  B. Cox  I. Dafinei  F. De Guio
P. Debbins  D. del Re  A. Dermenev  E. Di Marco  K. Dilsiz  K.F. Di Petrillo
G. Dissertori  S. Dogra  U. Dosselli  I. Dutta  F. Caleb  C. Fernandez Madrazo
M. Fernandez  M. Ferrero  Z. Flowers  W. Funk  M. Gallinaro  S. Ganjour  M.
Gardner  F. Geurts  A. Ghezzi  S. Gninenko  F. Golf  J. Gonzalez  C. Gotti  L.
Gray  F. Guilloux  S. Gundacker  E. Hazen  S. Hedia  A. Heering  R. Heller  T.
Isidori  R. Isocrate  R. Jaramillo  M. Joyce  K. Kaadze  A.
Karneyeu,aafootnotetext: Also at Institute for Nuclear Research, Moscow,
Russia H. Kim  J. King  G. Kopp  M. Korjik  O. K. Koseyan  A. Kozyrev  N.
Kratochwil  M. Lazarovits  A. Ledovskoy  H. Lee  J. Lee  A. Li  S. Li  W. Li
T. Liu  N. Lu  M. Lucchini  W. Lustermann  C. Madrid  M. Malberti  I.
Mandjavize  J. Mao  Y. Maravin  D. Marlow  B. Marsh  P. Martinez del Arbol  B.
Marzocchi  R. Mazza  C. McMahon  V. Mechinsky  P. Meridiani  A. Mestvirishvili
N. Minafra  A. Mohammadi,bbfootnotetext: Now at University of Wisconsin
Madison, Madison, United States F. Monti,ccfootnotetext: Now at Institute of
High Energy Physics, Beijing, China C.S. Moon  R. Mulargia  M. Murray  Y.
Musienko  J. Nachtman  S. Nargelas  L. Narvaez  O. Neogi  C. Neu  T. Niknejad
M. Obertino  H. Ogul  G. Oh  I. Ojalvo  Y. Onel  G. Organtini  T. Orimoto  J.
Ott  I. Ovtin  M. Paganoni  F. Pandolfi  R. Paramatti  A. Peck  C. Perez  G.
Pessina  C. Pena,ddfootnotetext: Also at California Institute of Technology,
Pasadena, USA S. Pigazzini  O. Radchenko  N. Redaelli  D. Rigoni  E. Robutti
C. Rogan  R. Rossin  C. Rovelli  C. Royon  M.Ö. Sahin  W. Sands  F.
Santanastasio  U. Sarica  I. Schmidt  R. Schmitz  J. Sheplock  J. C. Silva  F.
Siviero  L. Soffi  V. Sola  G. Sorrentino  M. Spiropulu  D. Spitzbart  A. G.
Stahl Leiton  A. Staiano  D. Stuart  I. Suarez  T. Tabarelli de Fatis  G.
Tamulaitis  Y. Tang  B. Tannenwald  R. Taylor  E. Tiras  M. Titov  S. Tkaczyk
D. Tlisov, ††footnotetext: Deceased I. Tlisova  M. Tornago  M. Tosi  R.
Tramontano  J. Trevor  C.G. Tully  B. Ujvari  J. Varela  S. Ventura  I. Vila
T. Wamorkar  C. Wang  X. Wang  M. Wayne  J. Wetzel  S. White  D. Winn  S. Wu
S. Xie  Z. Ye  G.B. Yu  G. Zhang  L. Zhang  Y. Zhang  Z. Zhang  R. Zhu

###### Abstract

The MIP Timing Detector will provide additional timing capabilities for
detection of minimum ionizing particles (MIPs) at CMS during the High
Luminosity LHC era, improving event reconstruction and pileup rejection. The
central portion of the detector, the Barrel Timing Layer (BTL), will be
instrumented with LYSO:Ce crystals and Silicon Photomultipliers (SiPMs)
providing a time resolution of about 30 ps at the beginning of operation, and
degrading to 50-60 ps at the end of the detector lifetime as a result of
radiation damage. In this work, we present the results obtained using a 120
GeV proton beam at the Fermilab Test Beam Facility to measure the time
resolution of unirradiated sensors. A proof-of-concept of the sensor layout
proposed for the barrel region of the MTD, consisting of elongated crystal
bars with dimensions of about $3\times 3\times 57$ mm3 and with double-ended
SiPM readout, is demonstrated. This design provides a robust time measurement
independent of the impact point of the MIP along the crystal bar. We tested
LYSO:Ce bars of different thickness (2, 3, 4 mm) with a geometry close to the
reference design and coupled to SiPMs manufactured by Hamamatsu and Fondazione
Bruno Kessler. The various aspects influencing the timing performance such as
the crystal thickness, properties of the SiPMs (e.g. photon detection
efficiency), and impact angle of the MIP are studied. A time resolution of
about 28 ps is measured for MIPs crossing a 3 mm thick crystal bar,
corresponding to an MPV energy deposition of 2.6 MeV, and of 22 ps for the 4.2
MeV MPV energy deposition expected in the BTL, matching the detector
performance target for unirradiated devices.

##  1 Introduction

The MIP Timing Detector (MTD) is designed to measure the arrival time of
minimum ionizing particles (MIPs) with a resolution of about 30 ps and
hermetic coverage up to a pseudorapidity $|\eta|$ = 3. With this level of
precision, the MTD will help to disentangle different interactions that occur
in the same LHC bunch crossing and are distributed over time with an RMS of
180-200 ps, improving the event reconstruction and pileup rejection of the CMS
experiment during the High Luminosity LHC phase (HL-LHC). The benefits in
terms of physics performance and an overview of the detector design are
documented in the Technical Design Report [1].

The sensor technology used to instrument the central part (up to $|\eta|$ =
1.48) of the MTD, the Barrel Timing Layer (BTL), consists of Lutetium Yttrium
Orthosilicate crystal bars doped with Cerium ((Lu1-xY${}_{\rm
x})_{2}$SiO5:Ce), abbreviated as LYSO:Ce, with dimensions of about $3\times
3\times 57$ mm3. The scintillation light produced by a MIP track is measured
with a pair of Silicon Photomultipliers (SiPMs), one at each end of the
crystal bar, matching the size of the crystal end face ($\sim$3$\times$3 mm2)
for optimal light collection. This layout provides the advantage of minimizing
the SiPM active area per crystal surface and is thus suitable for
instrumenting the large area (38 m2) of the MTD barrel detector with a limited
number of channels and constrained power budget. Since the time resolution
strongly depends on the thermal noise in the SiPM (dark counts), which is
proportional to the active area, small area SiPMs are preferred for optimal
performance. In addition, the use of two independent SiPMs for the readout of
the light and the combination of their time measurements offer the dual
advantage of providing a uniform spatial response of the sensor and an
improvement of the time resolution by a factor $\sqrt{2}$ with respect to a
single readout per crystal.

In this work, we present the proof-of-concept of this sensor layout close to
the reference design and study its time resolution while varying key
parameters such as the crystal thickness, the SiPM bias voltage and the impact
angle of the MIP with respect to the crystal axis. The paper is organized as
follows. Section 2 describes the experimental setup. Section 3 provides
details on the analysis methods. Results are presented in Section 4 and
discussed in Section 5, demonstrating that the detector performance target of
about 30 ps time resolution for unirradiated devices can be achieved with this
sensor layout.

##  2 Experimental setup

###  2.1 The beam line

The tests were performed at the Fermilab Test Beam Facility (FTBF), which uses
120 GeV protons extracted from the Fermilab Main Injector accelerator. Protons
are delivered in batches of about 20k-50k particles for a duration of about 4
seconds, hereafter referred to as spills. The accelerator operation usually
provides about one spill every minute. The trigger was based on a 10 cm2
scintillation counter located a few meters upstream with respect to the
experimental setup, as shown in Figure 1. A silicon tracker telescope [2]
consisting of twelve strip modules with 60 $\mu$m pitch, in alternating
orientation along orthogonal directions, and located in front of the crystals
and SiPMs under test, determined the impact point of the beam particles with a
precision of about 0.2 mm. The crystals and SiPMs were located inside a dark
box with its temperature maintained at about $25\pm 1^{\circ}$C using a
thermoelectric cooler and monitored by a thermistor located close to the
SiPMs. The box was mounted on a support structure capable of rotating the
sensors with respect to the beam direction. A few pictures of the setup are
shown in Figure 2. A Photek 240 Micro Channel Plate-PMT (MCP-PMT), with a time
resolution of about 12 ps (see Section 3.2), was positioned along the beam
line just downstream of the crystals and SiPMs and was used to measure a
reference time.

Figure 1: Schematic view of the beam line. From left to right, the
scintillator is used for the trigger, the silicon tracker defines the MIP
impinging position in the $xy$ plane, the MCP-PMT is used to define the
reference time. The two crystal+SiPMs test setups, one for the 1-bar and the
other for the 3-bar array, are positioned along the beamline.

Figure 2: Pictures of the experimental setup on the beam line (left) and of
the mechanical support for the rotation of the box housing three crystal bars
(right).

###  2.2 Crystals and SiPMs

The crystal bars of LYSO:Ce used in this test were manufactured by Crystal
Photonics Inc. (CPI) in three different geometries, close to the reference
design, of $3\times t\times 57$ mm3, where the thickness, $t$, is varied
between 2, 3 and 4 mm. All the surfaces were polished to a degree of optical
surface quality with R${}_{a}<$ 15 nm.

In the setup, several layers of Teflon (5 or more, corresponding to a
thickness of about 100 $\mu$m) were used to wrap the long sides of the bars to
improve light collection and reduce the risk of damaging the bar during
handling. Two SiPMs, one at each bar end, were coupled to the crystal using
optical grease (DOW CORNING® Q2-3067 optical couplant, index of refraction n =
1.4658).

Two different SiPM types, both consistent with the BTL SiPMs specifications
[1], were tested, as listed in Table 1. The first type belongs to a set of
S12572-015 SiPMs from Hamamatsu (HPK) with an active area of $3\times 3$ mm2.
The second set was provided by Fondazione Bruno Kessler (FBK) and consisted of
devices with an active area of $5\times 5$ mm2 based on the NUV-HD-ThinEpi
technology. Both SiPM types have a cell size of 15 $\mu$m, which provides an
optimal signal-over-noise ratio after the integrated radiation levels (up to
about 1.9$\times$1014 1 MeV neutron equivalent fluence, neq/cm2) expected in
the CMS detector during HL-LHC operation.

Table 1: List of sensors tested. Sensor ID | Crystal dimensions | SiPM type | SiPM active area
---|---|---|---
| [mm3] |  | [mm2]
HPK1 | $3\times 3\times 57$ | HPK S12572-015 | $3\times 3$
HPK2 | $3\times 3\times 57$ | HPK S12572-015 | $3\times 3$
HPK3 | $3\times 3\times 57$ | HPK S12572-015 | $3\times 3$
FBK1 | $3\times 2\times 57$ | FBK NUV-HD-TE | $5\times 5$
FBK2 | $3\times 3\times 57$ | FBK NUV-HD-TE | $5\times 5$
FBK3 | $3\times 4\times 57$ | FBK NUV-HD-TE | $5\times 5$

The photon detection efficiency (PDE) of the SiPMs tested is shown in Figure 3
as a function of the over-voltage (OV), defined as the excess bias voltage
above the breakdown voltage V${}_{\text{br}}$. The PDE weighted by the
emission spectra of LYSO:Ce is similar for the two SiPMs and reaches about 36%
for 6 V OV. The HPK and FBK SiPMs provide gains of $1.8\times 10^{5}$ and
$2.5\times 10^{5}$ (at 2 V OV), respectively. The HPK SiPMs feature a larger
increase of the excess noise factor (ENF) at high over-voltage due to after-
pulses and cross-talk effects. The PDE, gain and ENF were measured using the
method presented in [3].

(a)

(b)

(c)

Figure 3: (a) Photon detection efficiency (PDE), (b) gain and (c) excess noise
factor (ENF) as a function of over-voltage for the two SiPM types tested.

The breakdown voltages of the SiPMs tested are estimated to be 66.35 V for HPK
SiPMs and 37 V for FBK SiPMs at 25∘C. These V${}_{\text{br}}$ values were
obtained by extrapolating the values of 66 V and 36.7 V measured in the lab at
about 19∘C, using temperature coefficients of 59 mV/∘C and 41 mV/∘C for HPK
and FBK SiPMs, respectively.

Crystal bars coupled to HPK SiPMs were assembled in a system with a three-bar
holder, where the bars are placed parallel to each other and can be rotated
simultaneously with respect to the beam direction. Bars read out by FBK SiPMs
were set up with a holder that allows hosting one bar at a time. Pictures of
crystal bars, SiPMs and holders are shown in Figure 4.

Figure 4: Top: naked and wrapped individual crystal bars with the two FBK
SiPMs (left) and single wrapped bar glued to SiPMs (right). Bottom, from left
to right: three-bar assembly in a crystal holder with the screws used to
adjust the alignment of the crystals to the SiPMs on the readout board (left);
HPK SiPMs soldered onto a readout board (middle); single bar assembly (right).

###  2.3 Readout electronics

Customized electronic boards were used to apply the bias and perform the
readout of the SiPMs. In the three-bar setup, each SiPM signal was amplified
with a Gali74+ccchttps://www.minicircuits.com/pdfs/GALI-74+.pdf low noise
amplifier, filtered with a 500 MHz low-pass filter and then split into two
paths. One of the two signals was further amplified by about 44 dB using two
Gali52+dddhttps://www.minicircuits.com/pdfs/GALI-52+.pdf amplifiers in cascade
to provide a saturated waveform with a steep rising edge. This highly
amplified signal was used to extract the time of arrival using discrimination
with a very low threshold, while the other unsaturated signal was used to
measure the deposited energy, as described in the next section.

For the single bar readout, each SiPM signal was amplified and filtered by the
same first stage amplifier and 500 MHz low-pass filter that is used for the
three-bar assembly described above. The signal was split and one output was
read out directly to measure the signal amplitude while the second output was
further amplified by a second stage Hamamatsu C5594 amplifier with a gain of
36 dB and used to measure the MIP arrival time.

A CAEN V1742 digitizer [4] hosted in a VME crate was used for the readout of
all the waveforms: two for each SiPM under test and one for the MCP-PMT used
as time reference. The digitizer was triggered by TTL-level signals
originating from the trigger counter and configured to operate at a sampling
frequency of about 5.12 GSample/s, providing waveform measurements consisting
of 1024 samples spanning a 200 ns wide time window.

##  3 Data analysis methods

###  3.1 Pulse shape analysis

The waveforms corresponding to the signals of each SiPM from both the low and
high gain amplifiers were analyzed to extract the signal amplitude and the
time at which the MIP crossed the sensor, respectively. Typical pulses are
shown in Figure 5.

(a) SiPM (low gain)

(b) SiPM (high gain)

(c) MCP-PMT

Figure 5: Example of pulses from a LYSO:Ce bar coupled to SiPMs from the (a)
low gain and (b) high gain amplifier, and (c) a pulse from the MCP-PMT.

Firstly, a subtraction of the pedestal is performed on the low gain waveforms.
For each waveform, the pedestal is calculated as the average value of the
samples in the time interval $[1.0,t_{\text{rise}}-5.0]$ ns, where
$t_{\text{rise}}$ is the time corresponding to the maximum slope of the signal
on the rising edge. The integral of the low gain pulse, after the subtraction
of the pedestal, is computed in a time window of $[-10,+80]$ ns around the
time of a threshold at 50% amplitude on the rising edge of the pulse. The
pulse integral, hereafter referred to as signal amplitude, is found to give a
slightly better estimation of the light output than the peak value of the
signal (i.e. the amplitude of the sample with maximum deviation from the
baseline) and is used to select MIPs passing through the crystals and to apply
amplitude walk corrections (discussed in Section 3.4).

For the saturated pulses from the high gain channels, a baseline restoration
algorithm similar to the one implemented in the BTL ASIC for the mitigation of
the SiPM dark count noise is applied: pulses are processed such that the
waveform is inverted and delayed and then added to the original pulse as
$f(t)-f(t-D)$, where $f(t)$ is the original pulse and $D$ is the delay [5]. In
this way, baseline fluctuations on the leading edge of the pulse before
saturation are reduced. This approach is also effective in removing the low
frequency ($\sim$10 MHz) noise observed in this data set. The time of arrival
is then extracted from the resulting pulse using a leading edge
discrimination. A linear interpolation between the first sample below and the
first sample above the discrimination threshold value is performed to extract
the time of arrival at that value of the threshold with better accuracy.
Delays between 200 ps and 600 ps were found to provide optimal time
resolution. A delay of 400 ps was used in this analysis. Figure 6 shows the
pulse of Figure 5 (b) in the high gain channel after the invert and delay
procedure; the inverted and delayed pulse is added and the sum is normalized
to the delay, D = 400 ps, as $s(t)=(f(t)-f(t-D))/D$, representing the
derivative of the original pulse $f(t)$.

Figure 6: Left: example of pulse from a LYSO:Ce bar coupled to SiPMs recorded
at high gain after applying the invert and delay procedure described in the
text. The inverted and delayed pulse is normalized to the delay, D = 400 ps,
as $s(t)=(f(t)-f(t-D))/D$, representing the derivative of the original pulse
$f(t)$. On the right, a zoom of the same pulse in a range of few ns near the
leading rising edge is reported.

###  3.2 Micro Channel Plate photomultiplier as time reference

An MCP-PMT was used to measure the reference time. An example of a recorded
pulse from the MCP-PMT is shown in Figure 5 (c). Since the 5 GSample/s
sampling frequency of the digitizer is mismatched to the 200 ps rise time of
the device, we degraded the bandwidth of the signal and then estimate the MIP
time of arrival by performing a Gaussian interpolation of the digitized pulse.
A Gaussian interpolation is performed around the sample with maximum deviation
from the baseline using two samples before and two samples after the maximum.
The time and the amplitude of the MCP-PMT signals are taken as the mean and
the maximum of the fit function, respectively. A radial non-uniformity (up to
about 60 ps) of the MCP-PMT time response was observed as a function of the
distance of the MIP impact point from the centre of the MCP-PMT. A similar
effect has also been reported in other studies [6]. This non-uniformity was
studied in dedicated runs in which the relative position of the MCP-PMT and of
a single crystal bar was varied. An empirical correction is derived by
parametrizing the dependence of the MCP-PMT time as a function of the distance
from the MCP-PMT centre with a fourth-order polynomial. This correction was
applied to the entire data set. The time resolution of the MCP-PMT, after
these corrections, was estimated by measuring the coincidence time resolution
of two identical MCP-PMTs exposed to the beam and was found to be about 12 ps.
The impact of the MCP-PMT time resolution on the evaluation of the timing
performance of the tested BTL sensor prototypes is discussed in Section 3.4

###  3.3 Event selection

Events with one track reconstructed from hits in the silicon planes [2] and
with an energy deposit in the crystal bar and an MCP-PMT hit compatible with a
MIP are selected. An example of an amplitude distribution measured from a SiPM
coupled to a crystal bar with beam at normal incidence with respect to the bar
long axis is shown in Figure 7 (a). Events with the amplitude of each SiPM in
the range [0.8, 5] MPV, where MPV corresponds to the most probable value of
the amplitude distribution, are retained for the analysis.

Additional requirements are applied on the time of arrival of the MIP and on
the impact point position of the track to exclude backgrounds due to non-beam
particles. Beam-related SiPM signals are recorded in a well-defined time
window, with their absolute time being between 35 ns and 50 ns (Figure 7 (b)).
Events outside this window are due to dark counts, badly reconstructed hits,
or out of time beam particles, and are therefore discarded. The track impact
point position is selected by looking at the probability to find a signal from
the SiPMs with an amplitude above 0.8$\cdot$MPV, as a function of the track
coordinates $x$ and $y$ as measured by the tracker. The coordinate system is
defined such that, with beam at normal incidence, the $z$-axis runs along the
beam direction, the $x$-axis along the crystal longest axis and the $y$-axis
along the crystal shorter axis. The precise location of the bar can be well
determined in the region of the $xy$ plane where the probability to pass the
minimum amplitude and time range selection criteria is close to 1 (Figure 7
(c)). Events outside this region, due to mis-reconstructed tracks or cases
when the synchronization between the tracking system and the VME was lost, are
rejected.

(a)

(b)

(c)

Figure 7: From left to right: (a) measured amplitude distribution for a MIP
traversing a 3 mm thick crystal bar coupled to HPK SiPMs, with amplitudes
normalized to the MPV; (b) distribution of the absolute arrival times, with
red dotted lines representing the in time selection range applied in the data
analysis; (c) probability to have a SiPM signal with an amplitude compatible
with an energy deposit from a MIP crossing a 3 mm thick crystal bar, as a
function of the $x$ and $y$ track coordinates: the $xy$ region where the
probability is close to 1 allows one to determine the precise location of the
bar.

Good in-time MCP-PMT hits are selected by requiring their amplitude to be
between 50 mV and 250 mV and their absolute arrival times between 40 ns and 62
ns. The width of the fit to the MCP-PMT signal (discussed in section 3.2) and
RMS of the samples around the baseline before the MCP-PMT pulse are also used
to refine the selection of good MCP-PMT pulses to reject abnormal pulses and
poorly measured times due to noise.

###  3.4 Definition of time resolution

The time of a MIP incident on a crystal bar is estimated as the average
between the times of arrival measured at the two ends, $t_{left}$ and
$t_{right}$, and is compared to the time measured by the MCP-PMT, $t_{MCP}$:

$\Delta t_{bar}=t_{average}-t_{MCP}=\frac{1}{2}(t_{left}+t_{right})-t_{MCP}$
(3.1)

The time resolution of the bar is then obtained from a Gaussian fit to the
distribution of $\Delta t_{bar}$ after subtracting in quadrature the time
resolution of the MCP-PMT:

$\sigma_{t_{average}}=\sqrt{\sigma_{\Delta t_{bar}}^{2}-\sigma_{t_{MCP}}^{2}}$
(3.2)

Since the times of arrival are derived using a leading edge discrimination
approach at a fixed threshold, a dependence of the times of arrival on the
amplitude of the signals is expected. The observed variation of the times of
arrival across the entire range of measured amplitudes amounts to several
hundreds of picoseconds (Figure 8 (a)) and needs to be corrected to obtain the
optimal time resolution. This correction is referred to as "amplitude walk
correction". Amplitude walk corrections are derived from data by studying the
mean value of $t_{left(right)}-t_{MCP}$, where $t_{left(right)}$ is the time
measurement from the high gain channel (as described in Section 3.1) and
$t_{MCP}$ is the time measured by the MCP-PMT after corrections for the track
position dependence (Section 3.2), as a function of the amplitude of the
pulses from the low gain channel. This dependence is parametrized by fitting
the profiles with a sum of powers of logarithmic functions and a correction is
applied event-by-event by subtracting the value of the fitted function from
the raw value of $t_{left(right)}-t_{MCP}$. The effect is larger for higher
values of the leading edge threshold, therefore corrections are derived for
each threshold individually. As the crystal dimensions affect the light
propagation, amplitude walk corrections are also derived independently for
each sensor configuration. An example of measured amplitude dependence of the
time of arrival is shown in Figure 8 (a).

At a fixed position along the crystal bar, the bar time resolution can also be
estimated as half of the width of the difference $t_{diff}=t_{left}-t_{right}$
between the times of arrival measured at the two bar ends as

$\frac{1}{2}\sigma_{t_{diff}}=\frac{1}{2}\sqrt{\sigma_{t_{left}}^{2}+\sigma_{t_{right}}^{2}}=\sigma_{t_{average}}$
(3.3)

The relation of Eq. 3.3 holds if there are no correlated sources of
fluctuations between the $t_{left}$ and $t_{right}$ measurements. While
fluctuations due to photostatistics and SiPM noise are uncorrelated between
the two SiPMs, fluctuations due to electronics noise may be correlated. This
is the case for the three-bar setup used in this beam test. As our goal is to
characterize the timing performance of the sensors (i.e. crystal+SiPMs), the
time resolution estimated from a Gaussian fit of the $t_{diff}/2$ distribution
is used for most of the results that will be presented in the following.

The quantity $t_{diff}$ has a strong dependence on the impact point position
of the MIP along the bar, as shown in Figure 8. This dependence is fitted with
a linear function and the result of the fit is used to correct $t_{diff}$ on
an event-by-event basis. For the configuration with bars coupled to HPK SiPMs
(Figure 8 (b)), the slope of this linear function is about 15 ps/mm, which is
slightly larger than twice the reciprocal of the light speed in LYSO (1/v =
n/c $\sim$ 6.1 ps/mm, with n = 1.82), suggesting that the time response is
mostly determined by optical photons with a quasi-direct path to the SiPMs.
For the single bar setup with FBK SiPMs (Figure 8 (c)) a larger slope (about
18 ps/mm) is measured. The difference between the two values is attributed
both to differences in the light propagation mechanism related to the
dimension of the SiPM active area relative to the crystal end surface and to
the different optimal operation threshold (as discussed in Section 4.1). The
larger active area of the FBK SiPMs allows one to collect light coming out at
wider angles, thus with longer optical paths.

(a)

(b)

(c)

Figure 8: (a) Example of measured amplitude dependence of the MIP time of
arrival measured by one channel of the three-bar setup: the range of
amplitudes, extending down to about 0.1$\cdot$MPV, corresponds to MIPs
traversing a crystal bar rotated by 45∘ around its longitudinal axis, with MPV
being the most probable value of the amplitude distribution for a crossed
crystal thickness of 4.2 mm; a global offset was added on the vertical axis in
such a way that the average value of the time difference between the SiPM and
the MCP-PMT is equal to 0. (b) Time difference between two SiPMs as a function
of the track impact point position for a 3 mm thick bar read out by HPK SiPMs
using a leading edge discrimination threshold of 20 mV, and (c) for a 4 mm
thick bar read out by FBK SiPMs using a leading edge discrimination threshold
of 100 mV.

##  4 Results

###  4.1 Time resolution at different thresholds

The time resolution was first studied by varying the leading edge threshold
used to extract the MIP time of arrival. Figure 9 shows the time resolution,
estimated using the $t_{diff}$ method, as function of the threshold for the
three bars coupled to HPK SiPMs operated at Vbias= 72 V and for the single 4
mm thick bar readout by FBK SiPMs at Vbias= 43 V. These operating voltages
correspond to approximately an OV of 6 V and 36% PDE in both configurations.

Figure 9: Time resolution as a function of the leading edge discrimination
threshold for the three $3\times 3\times 57$ mm3 LYSO:Ce bars coupled to HPK
SiPMs (left) and for the $3\times 4\times 57$ mm3 LYSO:Ce bar coupled to FBK
SiPMs (right).

The time resolution as a function of the leading edge threshold is the result
of two main contributions: one from stochastic fluctuations in the time of
arrival of the photons, which increases as a function of the threshold, and
one from the noise, which decreases with increasing threshold; the
contribution from the noise, given by the noise divided by the derivative of
the pulse ($\sigma_{V}/(dV/dt)$), reduces at larger thresholds because the
derivative $dV/dt$ is larger, as shown in Figure 6. The combination of the two
contributions results in a minimum in the time resolution which corresponds to
the optimal operating threshold. In these configurations, the optimal
threshold is found to be around 20 mV for HPK SiPMs and 100 mV for FBK SiPMs.
We attributed the higher value of the optimal threshold in the second case to
the longer optical path in the wider crystal and to the larger noise
contribution in the FBK setup, which is uncorrelated between SiPMs at the two
bar ends and is due both to the larger SiPM capacitance and to the smaller
amplification of the readout electronics.

At these thresholds, a time resolution of 28.4$\pm$0.4 ps and 26.4$\pm$0.3 ps
is achieved for HPK and FBK SiPMs, respectively. At very low thresholds the
time resolution is affected by the noise and can reach values larger than 40
ps for a threshold below 10 mV. With increasing threshold, a degradation of
the time resolution is observed due to the increase of the photostatistics
contribution. For FBK SiPMs, this deterioration is less pronounced than for
HPK SiPMs. Signals from FBK SiPMs are larger because the crystal bar is
thicker, and the light collection efficiency is higher due to the larger SiPMs
area with respect to the crystal surface. This results in a time resolution
which shows a broad minimum and is almost flat for higher thresholds where the
noise contribution becomes negligible.

In all the results discussed in the following, the time resolution obtained at
the optimal threshold will be used, unless otherwise stated.

###  4.2 Spatial uniformity of the sensor response

The mean value of the signal amplitude measured at each bar end for a 3 mm
thick bar coupled to HPK SiPMs and a 4 mm thick bar coupled to FBK SiPMs is
shown in Figure 10 as a function of the track impact point position along $x$.
In the first setup, a variation of the single end amplitude up to about
$\pm$5% ($\sim$0.3%/mm) on the visible portion of the bar length is observed.
For the second setup, a much larger slope of the light collection of the
single end is observed ($>$1%/mm). Possible factors that can explain this
difference are differences in the crystal properties and preparation, such as
the crystal dimensions, the wrapping of the crystals, the optical coupling
between the crystals and the SiPMs, and the different ratio of the SiPM active
area to the crystal surface in the two configurations. The average of the
amplitudes of left and right SiPMs, which is proportional to the total light
collected, is instead in both cases rather uniform across the bar, with
variations within 5%.

Figure 10: Mean signal amplitude of the individual SiPMs and average of the
two for a $3\times 3\times 57$ mm3 LYSO:Ce bar coupled to HPK SiPMs (left) and
for a $3\times 4\times 57$ mm3 LYSO:Ce bar coupled to FBK SiPMs (right) as a
function of the hit position on the crystal bar. Amplitudes are normalized to
the most probable value of the signal amplitude distribution. In the right
plot, the distance of the track impact point from the bar center is shown on
the horizontal axis, as two sets of data with the bar in different positions
were combined to cover a larger range of hit positions along the bar.

The time of arrival measured at a single bar end depends on the distance
between the point where the scintillation photons are emitted and the SiPM,
due to the propagation time of optical photons within the crystal. On the
contrary, the average of the times measured at the two ends of a bar provides
a uniform time response, as shown in Figure 11. A small residual non-
uniformity is observed in $t_{average}-t_{MPC}$ across the bar length of
approximately 50 ps for the setup with HPK SiPMs and 20 ps for the one with
FBK SiPMs. This remaining non-uniformity can be due to both the intrinsic non-
linearity of the crystal bar response (related to self-absorption in LYSO and
effects due to the crystal wrapping) and to an imperfect correction of the
non-uniformity of the MCP-PMT response. An additional effect that would bring
a different dependence of $t_{average}-t_{MCP}$ on the impact point position
in the two configurations could be a different tilt of the MCP-PMT with
respect to the bar. Overall the impact of this residual non-uniformity on the
global time resolution of the bar is marginal, as will be shown later.

Figure 11: Mean value of the times of arrival from left and right SiPMs and
their average as a function of the MIP impact point for a $3\times 3\times 57$
mm3 LYSO:Ce bar coupled to HPK SiPMs (left) and for a $3\times 4\times 57$ mm3
LYSO:Ce bar coupled to FBK SiPMs (right). The different range of values of the
track impact points $x$ in the two configurations is due to the selection of
events with spatial overlap between the crystal bar and the MCP-PMT.

The time resolution for various impact point positions of the tracks along the
$x$ direction is reported in Figure 12 for $t_{left}$, $t_{right}$,
$t_{average}$ and $t_{diff}/2$ at about 6 V OV. A local bar time resolution of
about 30 ps and 25 ps is achieved for a $3\times 3\times 57$ mm3 LYSO:Ce bar
coupled to HPK SiPMs and for a $3\times 4\times 57$ mm3 LYSO:Ce bar coupled to
FBK SiPMs, respectively. The better performance in the configuration with the
FBK SiPMs can be ascribed to the larger energy deposited due to thicker
crystal and larger light collection efficiency. The combination of the two
SiPM measurements in $t_{average}$ improves the time resolution by about
$\sqrt{2}$ with respect to the individual SiPM, since the dominant stochastic
fluctuations from photostatistics are uncorrelated between the two ends. The
time resolution obtained using $t_{diff}/2$ in the setup with HPK SiPMs is
slightly better than the time resolution obtained using $t_{average}$ because,
as mentioned above, the contribution of the correlated electronics noise gets
cancelled in $t_{diff}$.

Figure 12: Time resolution of the left and right SiPMs, their average, and
half of the time difference as a function of the MIP impact point for a
$3\times 3\times 57$ mm3 LYSO:Ce bar coupled to HPK SiPMs (left) and for a
$3\times 4\times 57$ mm3 LYSO:Ce bar coupled to FBK SiPMs (right). For
$t_{left}$, $t_{right}$, $t_{average}$, the estimated contribution from the
resolution of the MCP-PMT (12 ps) was subtracted in quadrature.

A comparison between the global and local time resolutions is shown in Figure
13. The global time resolution is obtained using a Gaussian fit to the
distribution of $t_{diff}/2$, after applying amplitude walk and position
dependence corrections (as described in Section 3.4), for beam tracks with
impact point at any location along the portion of the bar illuminated by the
beam, while the local time resolution is for tracks with impact point in a 2
mm wide spot. The values of the global and local time resolutions are
comparable for both the configuration with HPK SiPMs and the one with FBK
SiPMs. As no effects from the MCP-PMT response are present when using the
variable $t_{diff}$, we conclude than any residual intrinsic non-linearity of
the bar response along its length introduces a negligible contribution to the
global time resolution.

Figure 13: Global and local time resolution for a $3\times 3\times 57$ mm3
LYSO:Ce bar coupled to HPK SiPMs (left) and for a $3\times 4\times 57$ mm3
LYSO:Ce bar coupled to FBK SiPMs (right). The local time resolution is for
tracks with impact point in a 2 mm wide spot at 1 cm from the bar end.

Exploiting the dependence of the single SiPM time response on the impact point
position of the track along the bar, this sensor layout is also capable of
providing a measurement of the impact point with few millimetres resolution.
As shown in Figure 8, the difference between the times of arrival measured at
the two ends, $t_{diff}=t_{left}-t_{right}$, is strongly correlated with the
impact point of the MIP along the bar. The impact point can therefore be
obtained by dividing the measured $t_{diff}$ (with only amplitude walk
corrections applied) by the slope, $k$, of Figure 8, which represents the
variation of the $t_{diff}$ mean value as a function of $x$. In general, the
impact point position resolution is given by
$\sigma_{x}=\sigma_{t_{diff}}/k=2\sigma_{t_{average}}/k$. The distribution of
the residuals between the estimated impact point position $x_{reco}$ and the
impact point position measured by the tracker, $x_{track}$, is reported in
Figure 14 for data collected at about 6 V OV. A resolution of 4 mm and 2.8 mm
is achieved for the configuration with HPK and FBK SiPMs, respectively. No
dependence of the residuals on the impact point position is observed (Figure
14 (right)). The better spatial resolution measured with the bar with FBK
SiPMs is the result of both a better time resolution and of the steeper slope
$k$, as shown in Figure 8. This method is valid for high $p_{T}$ tracks in CMS
that are at close to normal incidence on the crystal bars.

Figure 14: Distribution of the residuals between the impact point $x_{reco}$
estimated from the time difference between the times of arrival measured at
the two bar ends and the impact point $x_{track}$ measured by the tracking
system for a 3$\times$3$\times$57 mm3 LYSO:Ce bar coupled to HPK SiPMs and
3$\times$4$\times$57 mm3 LYSO:Ce bar coupled to FBK SiPMs. A fit with a
Gaussian function is superimposed (left). The distribution of the residuals as
a function of the impact point position is shown on the right.

###  4.3 Dependence of the time resolution on the signal amplitude

In this section the dependence of the time resolution on the amplitude of the
signals is discussed. We describe the effects of crystal thickness, SiPM over-
voltage and the dependence on the energy deposited in the crystals.

####  4.3.1 Role of the crystal thickness

The crystal thickness traversed by a MIP determines the energy deposited in
the crystal and therefore the number of signal photo-electrons. The energy
deposited by a MIP in a LYSO:Ce crystal follows a Landau distribution with the
most probable value (MPV) of 0.86 MeV/mm [1] and thus a total energy deposit
of about 2.6 MeV for a MIP at normal incidence in a 3 mm thick crystal. In the
design of the MTD barrel detector the crystal thickness is optimized to limit
the material budget in front of the CMS electromagnetic calorimeter. In
particular, three different crystal thicknesses of 2.4, 3.0 and 3.75 mm are
used, as described in [1]. For this reason, the signal amplitude and time
resolution were studied for crystals of different thicknesses in a range that
mimics the one for the BTL final design.

Data were taken using individual crystal bars of dimensions $3\times t\times
57$ mm3, with $t$ = 2, 3, 4 mm, and read out by FBK SiPMs. FBK SiPMs were used
in this study because their active area is large enough to read out the entire
face of all the crystals with some margin, thus reducing uncertainties in
light collection efficiency due to alignment of the crystal with the SiPM. The
setup is configured such that MIPs impact at normal incidence with respect to
the crystal bar longitudinal axis. Events with impact point position of the
track in a 1 cm wide region around the bar centre are selected for this
analysis. The operating bias voltage for this measurement was 72 V,
corresponding to about 6 V over the breakdown voltage. As shown in Figure 15
(left), the MIP peak most probable value scales approximately linearly with
the crystal thickness in the tested range. The measured time resolutions for
individual SiPMs, the averaged arrival time and the time difference divided by
two are reported in Figure 15 (right). The scaling of the time resolution with
the crystal thickness follows a behaviour that is consistent with the inverse
of the square root of the thickness. This suggests that the time resolution is
dominated by the photostatistics contribution, $\sigma^{phot}\propto
1/\sqrt{N_{ph}}$, due to stochastic fluctuations in the time of arrival of the
photons detected at the SiPM.

Figure 15: Most probable value of the MIP signal amplitude normalized to the
MPV for 3 mm crystal thickness (left) and time resolution (right) as a
function of the crystal thickness for $3\times\text{t}\times 57$ mm3 (t = 2,
3, 4 mm) LYSO:Ce bars coupled to FBK SiPMs.

####  4.3.2 Role of the over-voltage

The over-voltage has a direct impact on the signal amplitude because it
determines the SiPM photon detection efficiency (PDE). It also affects other
parameters like the SiPM gain, noise, cross-talk and single-photon time
resolution (SPTR), which in turn can impact the time resolution.

Figure 16 shows the time resolutions as a function of the over-voltage for the
three bars coupled to HPK SiPMs (Figure 16 (left)) and for the single 4 mm
thick bar read out by FBK SiPMs (Figure 16 (right)). Figure 17 shows the same
time resolutions as in Figure 16 but as a function of the PDE. Each point
corresponds to the time resolution at the optimal threshold. A pure
photostatistics contribution to the time resolution would scale as the inverse
of the square root of the number of signal photoelectrons (proportional to the
PDE) and would give $\alpha=$ 0.5, while a pure SiPM noise contribution would
scale as the inverse of the number of photoelectrons, resulting in
$\alpha=$1\. Here, the coefficient $\alpha$ is found to be close to 0.5 for
the configuration with FBK SiPMs and between 0.5 and 1 for the one with HPK
SiPMs. In the first case, it suggests that photostatistics is the dominant
contribution. The departure from the simple $1/\sqrt{PDE}$ behaviour of the
HPK SiPMs configuration indicates that both the photostatistics and noise
terms are contributing to the time resolution.

Figure 16: Time resolution as a function of the over-voltage for the three
$3\times 3\times 57$ mm3 LYSO:Ce bars coupled to HPK SiPMs (left) and for the
$3\times 4\times 57$ mm3 LYSO:Ce bar coupled to FBK SiPMs (right).

Figure 17: Time resolution as a function of the photon detection efficiency
for the three $3\times 3\times 57$ mm3 LYSO:Ce bars coupled to HPK SiPMs
(left) and for the $3\times 4\times 57$ mm3 LYSO:Ce bar coupled to FBK SiPMs
(right). A fit to data of a power law function in the form $f(PDE)\propto
1/(PDE)^{\alpha}$ is superimposed.

####  4.3.3 Dependence on the energy deposition in the crystals

We studied also the dependence of the time resolution on the signal amplitude
at fixed over-voltage. For this study, the data taken with bars rotated at 45∘
in the $yz$ plane were used, spanning a large range of crystal thicknesses
crossed by the MIP and providing therefore a data set with a large range of
energy depositions. The transverse cross-section of the bars with respect to
the beam direction in this configuration is sketched in Figure 18 (left). The
distribution of the signal amplitudes for different impact points of the
tracks along the $y$ direction is shown in Figure 19 (left). Here, the
amplitude is defined as the sum of the signal amplitudes measured by each SiPM
in the bar normalized to the most probable value of the distribution for the
maximum crossed thickness.

Figure 18: Sketch of the LYSO:Ce bars orientation with respect to the beam in
two configurations: transverse cross section of the bars for a rotation in the
$yz$ plane (left); rotation in the $xz$ plane with the beam forming an angle
of incidence $\theta$ with respect to the bar longitudinal axis (right).

The time resolution estimated using $t_{diff}/2$ is shown in Figure 19 in
different bins of amplitude for the central bar of the three-bar assembly. The
leading edge threshold for the timing measurement is the same for all the
amplitudes and is set to the value that minimizes the overall time resolution.

Figure 19: Left: Distribution of the signal amplitude, normalized to the MPV,
for different impact point positions of the tracks along the $y$ direction for
a bar rotated by 45∘ in the $yz$ plane; amplitudes decrease linearly with the
crossed crystal thickness and therefore with the shift in the $y$ direction
from the impact point corresponding to the maximum crossing path length.
Right: time resolution as a function of the amplitude of the signal in a
$3\times 3\times 57$ mm3 crystal bar coupled to HPK SiPMs. The dependence is
shown for a discrimination threshold of 20 mV.

The dependence on the signal amplitude, $A$, is fitted with two types of
functions: a power law plus a constant term

$f_{1}(A)=\frac{a}{A^{\alpha}}\oplus c$ (4.1)

and the sum in quadrature of a stochastic, noise and constant term in the
form:

$f_{2}(A)=\frac{s}{\sqrt{A}}\oplus\frac{n}{A}\oplus c$ (4.2)

The power law exponent of Eq. 4.1 is found to be $\alpha$ = 0.87. The fact
that the measured coefficient is between 0.5 and 1 suggests that
photostatistics and noise terms both contribute to the time resolution. The
value of $\alpha$ is found to be dependent on the value of the leading edge
discrimination threshold, with $|\alpha|$ increasing for increasing
thresholds, because of the different balance between the stochastic and noise
contributions. Similarly, the values of the stochastic $s$ and noise $n$
parameters of Eq. 4.2 are found to be comparable ($s=18\pm 4$ ps and $n=16\pm
9$ ps).

A constant term $c$ of about 13$\pm$1 ps is measured. A contribution to this
constant term arises from the signal digitization step and is estimated to be
of the order of 5 ps. Another contribution comes from tails in the energy
deposition due to secondary particles that yield a different energy deposition
profile along the bar.

###  4.4 Reconstruction of the time of arrival and impact point of a MIP
crossing two bars

Since tracks at high pseudorapidity in CMS will cross more than one bar, we
also characterized the response of the sensors in a configuration where the
crystal bars are rotated in the $yz$ plane in such a way that a signal from
the MIP is generated in adjacent bars. This study aims at characterizing both
the time and the position resolution of the sensors in this configuration. For
this measurement, the tilting angle of the three-bar array is about 45∘
(Figure 18 (right)). When the signal is shared, a weighted average of the
times of arrival measured on the individual bars can be used to obtain the
optimal time resolution, and a weighted average of the position of the bars
with hits can provide a precise determination of the MIP impact point.

In this configuration, the MIP deposits energy either in one bar only or in
two bars, depending on the impact point along $y$ (Figure 18). Events with
amplitude in the range [0.1, 5.0] MPV are selected, where MPV is the most
probable value of the distribution of the sum of the amplitudes from all the
SiPMs and is representative of the total light output. The signal arrival
times from individual bars are first calculated as the average of the times
measured at the two ends in the individual bar after applying amplitude walk
corrections. Then, a weighted average is calculated as

$t_{comb}=\frac{\sum w_{i}t_{i}}{\sum w_{i}}$ (4.3)

with weights $w_{i}=1/\sigma_{i}^{2}$, where $\sigma_{i}$ is the expected time
resolution of the individual bar. The value $\sigma_{i}$ is assumed to be
proportional to $1/A_{i}^{\alpha}$, with $A_{i}$ being the energy measured in
the $i$-th bar and using a value $\alpha$ = 0.87 (as discussed in Section
4.3.3). The energy deposited in a crystal bar is estimated from the sum of the
amplitudes of the signals measured at the two bar ends. Combined times for the
left and right ends are calculated separately using the same formula, and used
to estimate the time resolution using the time difference method.

Figure 20 (left) shows the time resolution, estimated using
$\sigma_{t_{diff}}/2$, as a function of the track impact point along $y$ for
the three individual bars and for the weighted average computed as in Eq. 4.3.
The MIP track crosses adjacent bars for impact point positions along the $y$
axis between 22.5 and 24 mm and between 25.2 and 26.7 mm. The time resolution
in the region where there is an overlap between two bars improves when using
the combination of the times of arrival measured in adjacent bars with respect
to the single bar only. We observe a modulation of the combined resolution as
a function of the impact point. This is related to the presence of a large
(850 $\mu$m) gap between the crystal bars in the setup, which is determined by
the size of the non-active area of the SiPMs.

A simple model, shown in Figure 20, describes the measurements well. The model
assumes energy depositions per track in each bar proportional to the length of
its trajectory through the crystal. The time resolution is estimated using the
fit function shown in Figure 19. In the model, gaps between bars are set to
850 $\mu$m as measured in the lab and the rotation angle is set to 40∘ to
match the distance between the minima of the measured resolution at the centre
of each bar.

The size of the gap between crystals was scanned in simulation. If the actual
gap between bars is kept below 200 $\mu$m, the combination of time
measurements from neighbouring crystals allows one to maintain a nearly
uniform time resolution across the gap, with MIPs crossing that region having
the same time resolution of MIPs passing through a single crystal at maximum
thickness. In the final BTL design, the use of very thin (about 80 $\mu$m)
layers of Enhanced Specular Reflector (ESR) between crystals is foreseen to
reduce this effect to a negligible level as shown in Figure 20 (right).

Figure 20: Left: time resolution as a function of the beam impact point on the
three-bar array with HPK SiPMs, for individual bars (colored solid lines) and
for the weighted average of the arrival time in two adjacent bars (solid black
line); the crystal barr array is rotated by an angle of about 45∘ in the $xz$
plane with respect to the beam direction; measurements are shown as dots and
compared to the model predictions represented by the lines. Right: the model
prediction for an array of five bars, each identical to the middle bar in the
left plot, is shown for three values of the inter-crystal gap size.

The position of a track crossing one or more bars can be estimated as a
weighted average

$v_{\mathrm{track}}=\frac{\sum A_{i}v_{i}}{\sum A_{i}}$ (4.4)

where $A_{i}$ is the signal amplitude in the i-$th$ bar and $v_{i}$ is the
coordinate of the bar centre along the plane of the bars. The position
resolution can be estimated by comparing $v_{\mathrm{track}}$ with the track
position measured with the silicon telescope and corrected by taking into
account the beam angle, $v_{\mathrm{telescope}}$. Figure 21 (left) shows the
distribution of $\Delta=v_{\mathrm{track}}-v_{\mathrm{telescope}}$. The long
tails of the distributions are due to inefficiencies of the track
reconstruction and assignment of random beam tracks to bar hits; this
background is estimated with a Gaussian fit of the tails of the distribution,
as shown by the red line. After subtracting the background and taking into
account the uncertainty on the track $y$ position measured from the telescope,
the estimated position resolution for tracks at an angle of about 40∘ is
$0.56\pm 0.04$ mm. This estimation is in agreement with simulations, as shown
in Figure 21 (right). The position resolution depends on the angle of
incidence of the particle and improves with smaller gaps between bars. For an
inter-crystal gap below 200 $\mu$m a resolution below 1 mm is expected for all
angles and about 0.4 mm at the angles corresponding to the edge of the BTL
angular coverage ($\theta$ = 25.6∘ for a pseudorapidity $|\eta|$ = 1.48).

Figure 21: Left: distribution of the difference between the track position
measured with the telescope and the position estimated from the weighted
average of the position of the bars; the red line represents the background
estimated with a Gaussian fit of the tails of the distribution. Right:
simulation results for the position resolution as a function of beam angle for
an array of bars with 3 $\times$ 3 mm2 cross section and gaps of 0.85, 0.20,
and 0.08 mm, respectively, shown as lines, and measured position resolution
(marker) for 3 $\times$ 3 mm2 cross section with 0.85 mm gaps.

###  4.5 Dependence of the time resolution on the MIP impact angle

As low transverse momentum particles will be bent in the CMS magnetic field
and will impact the crystal bar with non-normal incidence, we have also
studied the dependence of the time resolution on the angle of incidence of the
MIP on the crystal bar. In fact, while the average crystal thickness in the
MTD is 3 mm, the average MPV of the energy deposit from minimum bias events in
CMS, is expected to be about 4.2 MeV due to the path length for bending tracks
within the crystal volume (instead of 2.6 MeV as would be the case for
particles at normal incidence). This study is performed using data in which
the three-bar array with HPK SiPMs was rotated by an angle $\theta$ with
respect to the beam direction in the $xz$ plane, as represented in Figure 18
(right). Several angle values were tested between 90∘, corresponding to normal
incidence, to about 15∘, the smallest achievable angle given the mechanical
constraints of our setup. This set of angles spans a large portion of the
range of slant thicknesses expected in CMS, where very low $p_{T}$ (0.7-2 GeV)
charged particles strongly bent by the magnetic field can enter the crystal
with an angle as small as $\sim$10∘.

The time resolution, estimated with $t_{diff}/2$, was measured as a function
of the slant thickness $t/\sin(\theta)$ and is reported in Figure 22. It
ranges between 28 ps for normal incidence (3 mm slant thickness) and 15 ps for
a slant thickness of about 11 mm, corresponding to $\theta=$15.8∘. We observe
that the time resolution improves with the slant thickness as $1/\sqrt{t}$.
This means that the time resolution is largely dominated by the increased
light output, while the wider spread of energy depositions, both in position
along the bar and in a broader time range, for smaller angles doesn’t
introduce appreciable effects that degrade it. In addition, the asymmetry in
the signal amplitude measured at the left and right SiPMs of a bar, defined as
$(A_{L}-A_{R})/(A_{L}+A_{R})$, does not change by more than 5% when varying
the impact angle of the MIP on the crystal bar. This is shown on the left plot
of Figure 22. Similarly, no difference in the pulse shape of the left and
right SiPM was observed. These observations suggest that the contribution to
the light signal from the detected Cherenkov photons, which is expected to
vary with the angle, has an impact smaller than 5% on the overall sensor
performance. A fit to data in the form $f(t)=a/\sqrt{t}\oplus c$, reported in
Figure 22 (right), yields a constant term $c$ of about 7 ps, which can be
ascribed to the effect of the signal digitization.

Figure 22: Left: asymmetry in the signal amplitude measured at the left and
right SiPMs of a bar, defined as $(A_{L}-A_{R})/(A_{L}+A_{R})$, as a function
of the MIP impact angle. The asymmetry is estimated for tracks with impact
point position at the left end of the bar. Right: time resolution of the
individual bars in the three-bar array with HPK SiPMs as a function of the
slant thickness obtained by tilting the setup by angles down to about 15∘ to
the crystal axis. The slant thickness at each angle is estimated by scaling
the crystal thickness (3 mm) by the ratio of the amplitude MPV at a given
angle to the MPV at normal incidence.

##  5 Discussion

A local time resolution of about 28 ps for LYSO:Ce $3\times 3\times 57$ mm3
crystal bars coupled to $3\times 3$ mm2 HPK SiPMs and 25 ps for a $3\times
4\times 57$ mm3 bar coupled to $5\times 5$ mm2 FBK SiPMs are measured by
combining the times of arrival at two bar ends. The combination of the timing
of the two SiPMs results in an improvement by a factor $\sqrt{2}$ in time
resolution over the individual ends at a fixed location along the bar, and in
a uniform response along the crystal bar. By comparing the global and local
time resolutions, we conclude that any residual non-linearity of the time
response along the bar results in a negligible contribution to the overall
time resolution.

The dependence of the time resolution on the signal amplitude was studied as a
function of some key parameters of the sensors, like the crystal thickness,
the SiPM over-voltage and PDE, and of the energy deposited in the crystals. As
expected, the time resolution improves with the increase in light output. For
sufficiently high thresholds, when the noise contribution becomes negligible,
the time resolution is dominated by stochastic fluctuations and scales with
the inverse of the square root of the signal amplitude.

The study of the timing performance as a function on the MIP angle of
incidence shows no degradation of the time resolution due to the time and
spatial spread of the energy depositions along the bar and demonstrates the
benefit of the increased light output related to the increased slant
thickness.

A proof of the performance with energy sharing between adjacent bars was
given: no degradation is expected, provided sufficiently small ($<$ 200
$\mu$m) inter-crystal gaps.

Finally, besides the timing capabilities, a resolution of few millimetres on
the impact point along the crystal bar long axis can be achieved with this
sensor layout by exploiting the time difference between the times of arrival
measured at two bars ends. In the orthogonal direction, a good spatial
resolution can also be obtained: for tracks crossing adjacent bars, the track
impact position can be estimated as an average of the position of the bars
with hits, weighted by the hit amplitudes, and a precision below 1 mm can be
achieved for all angles of incidence.

The 28 ps time resolution measured for normal incidence on a 3 mm thick bar,
which corresponds to 2.6 MeV energy deposition, can be extrapolated to about
22 ps for the 4.2 MeV MPV energy deposition expected in the BTL final design.
Additional contributions from clock, digitization, electronics and noise, as
reported in [1], are expected to bring the time resolution to about 30 ps at
the beginning of the detector operation.

The results achieved with these tests indicate that this sensor design with
double-end readout is suitable for the measurement of the arrival time of
minimum ionizing particles with a resolution of about 30 ps for the upgraded
CMS experiment.

##  6 Summary

In this work, we present a comprehensive characterization of the timing
performance of sensor prototypes for the CMS barrel timing layer consisting of
elongated LYSO:Ce crystal bars of dimensions $3\times t\times 57$ mm3, read
out at both ends with Silicon Photomultipliers (SiPMs). We demonstrate that
such a sensor layout can provide a uniform time response with a time
resolution better than 30 ps and spatial resolution of a few millimetres for
single minimum ionizing particles. In particular, a time resolution of 22 ps
is achieved for the 4.2 MeV MPV energy deposition expected in the MTD barrel
timing layer. Two different types of SiPMs were tested using the 120 GeV
proton beam line at the Fermilab Test Beam Facility: $3\times 3$ mm2 SiPMs
from Hamamatsu and $5\times 5$ mm2 SiPMs from FBK, allowing to better
understand the role of SiPM parameters on the performance achieved. The timing
performance of the sensors was characterized under different over-voltages and
angles of incidence of the MIP on the crystal bar, that are representative of
the conditions under which the MTD detector will operate. The time resolution
was also measured as a function of different parameters like the crystal
thickness, the photon detection efficiency and the leading edge discrimination
threshold used to extract the time of arrival of the MIP. The data collected
allowed us to understand and parametrize the contribution of different effects
to the time resolution providing a coherent description of the sensor timing
performance. These results represent an important step forward for the use of
crystal and SiPM technology in large-area timing detectors, demonstrating that
the target time resolution for the MIP Timing Detector of 30 ps with
unirradiated SiPMs can be achieved with the present technology and sensor
layout.

## Acknowledgments

We are grateful to the FNAL accelerator teams for providing excellent beam
quality for these tests. We acknowledge the FTBF team for the support provided
during data taking. We would like to thank Lorenzo Uplegger for his support
with the tracking. We are also thankful to Todd Nebel and Jim Wish for their
help in promptly providing mechanical solutions to arrange our setup on the
beam line.

We thank the technical and administrative staff at the CMS institutes for
their contributions to the success of the CMS effort. We acknowledge the
support provided to the MTD project by the following funding agencies: CERN;
Academy of Finland, MEC, and HIP (Finland); CEA (France); NKFIH, OTKA-131991
(Hungary); INFN (Italy); NRF-2008-00460, 2018R1A6A1A06024970, 2020R1A2C1012322
(Republic of Korea); LAS (Lithuania); FCT (Portugal); MON, Rosatom, RAS, RFBR,
and NRC KI (Russia); SEIDI, CPAN, PCTI, and FEDER (Spain); Swiss Funding
Agencies (Switzerland); DOE and NSF (USA);

## References

  * [1] CMS Collaboration.  A MIP Timing Detector for the CMS Phase-2 Upgrade.  Technical Report CERN-LHCC-2019-003. CMS-TDR-020, CERN, Geneva, 2019.
  * [2] S. Kwan et al.  The pixel tracking telescope at the Fermilab Test Beam Facility.  Nucl.Instrum.Meth.A, 811:162–169, 2016.
  * [3] Y. Musienko, S. Reucroft, and J. Swain.  The gain, photon detection efficiency and excess noise factor of multi-pixel Geiger-mode avalanche photodiodes.  Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, 567(1):57 – 61, 2006.  Proceedings of the 4th International Conference on New Developments in Photodetection.
  * [4] CAEN V1742: 32+2 Channel 12bit 5 GS/s Switched Capacitor Digitizer.  https://www.caen.it/products/v1742/.
  * [5] A. Gola, C. Piemonte, and A. Tarolli.  The DLED Algorithm for Timing Measurements on Large Area SiPMs Coupled to Scintillators.  IEEE Transactions on Nuclear Science, 59(2):358–365, 2012.
  * [6] J. Bortfeldt et al.  Timing Performance of a Micro-Channel-Plate Photomultiplier Tube.  Nucl.Instrum.Meth.A, 960(163592), 2020.