# The EXTRA-BL4S experiment for the measurement of the energy and angular
distributions of transition radiation X-rays

M. N. Mazziotta 11footnotetext: Corresponding authors. 0000-0001-9325-4672 F.
Loparco  0000-0002-1173-5673 A. Anelli  M. M. Belviso  A. Buquicchio  E. V.
Cassano  M. De Cosmo  P. Ginefra  M. L. Martulli  C. Picci  D. Picicci  R. D.
Soriano  A. P. Tatulli  G. Tripaldella  V. M. Zupo  M. F. Muscarella  S.
Turbacci  M. Boselli  0000-0003-3540-8228 C. B. da Cruz E Silva
,22footnotetext: Now at LIP - Laboratório de Instrumentação e Física
Experimental de Partículas Avenida Prof. Gama Pinto 2, Complexo
Interdisciplinar (3is), 1649-003 Lisboa, Portugal 0000-0002-1231-3819 M. Joos
0000-0001-8184-5598 and P. Schütze  0000-0003-4802-6990

###### Abstract

We have designed and implemented an experiment to measure the angular
distributions and the energy spectra of the transition radiation X-rays
emitted by fast electrons and positrons crossing different radiators. Our
experiment was selected among the proposals of the 2021 Beamline for Schools
contest, a competition for high-school students organized every year by CERN
and DESY, and was performed at the DESY II Test Beam facility area TB21, using
a high-purity beam of electrons or positrons with momenta in the range from
$1$ to $6\hbox{$\,{\rm GeV/c}$}$. The measurements were performed using a
$100\hbox{$\,{\rm\mu m}$}$ thick silicon pixel detector, with a pitch of
$55\hbox{$\,{\rm\mu m}$}$. Our results are consistent with the expectations
from the theoretical models describing the production of transition radiation
in multilayer regular radiators.

##  1 Introduction

In recent years, high-school physics curricula increasingly include topics
related to modern high-energy physics and particle detectors. Universities and
research centers promote several programs to bring high-school students in
touch with modern physics and the scientific research. The Liceo Scientifico
“A. Scacchi” in Bari has taken part in such projects for years, and in 2021
the school promoted the participation of a team of students of the $12^{th}$
and $13^{th}$ grade in the Beamline for Schools (BL4S) competition.

BL4S is organized by CERN in collaboration with DESY, and offers to groups of
high-school students the unique opportunity to propose a scientific experiment
at a particle accelerator facility and to win a trip to perform it. Because of
the maintenance of CERN accelerators, the experiment was performed at the DESY
II Test Beam facility in Hamburg. The students, coordinated by their physics
teachers and under the supervision of experienced researchers from the Physics
Department of the Bari University and from the INFN Unit in Bari, won the
competition.

The goal of the experiment conceived by the team was to study the transition
radiation emitted by fast electrons and positrons crossing different kinds of
radiators. This paper provides a short presentation of the BL4S competition
and presents the experiment and the result obtained by the team during their
beam time in Hamburg in September 2021.

##  2 The BL4S competition

Beamline for Schools (BL4S) [1] is a physics competition organised by CERN and
DESY, which invites high-school students from all over the world to propose an
experiment to be performed at a particle accelerator. Each team has to write
an original scientific proposal, explaining the theoretical background of the
selected topic, and describing both the procedure to carry it out at a test
beam facility and the results that they expect to find. A jury of experts,
including scientists of CERN and DESY, review the proposal and select two
teams (three from 2022 on) that win a trip to a fully equipped beam line of a
particle accelerator.

From 2014 to 2018 the winning experiments took place at the test beam area of
the CERN Proton Synchrotron (PS) accelerator. In 2019 the competition moved to
the DESY II Test Beam Facility (Hamburg, Germany) [2]. The partnership between
CERN and DESY allowed BL4S to continue during the three-year long shutdown of
the CERN accelerator complex for upgrade and maintenance.

The competition is structured in several preparatory phases, which include
conferences and meetings with the organisers. Once the competition is
announced, usually in Autumn, interested teams start preparing their
proposals. Teams can include students either from the same school or from
different schools. Having teams representing two schools or more is not
unusual. During the proposal preparation, students are involved in an intense
research project. After the conception and design of their experiment, the
participants must write a well structured proposal and submit it on time. The
students are not alone in this process, but they are guided by their coaches,
who provide them with details on particle physics and teach them the necessary
technical skills. Team coaches can be teachers, parents or scientists of local
universities. It is important that students are well aware of each scientific
detail of the proposed experiment, so that the theoretical background is clear
and solid. The students are required to write down in detail how they intend
to use the particle beam for their measurements and which equipment and
detectors they need. Moreover, participants often complement their theoretical
hypothesis with computer simulations. In fact, it is fundamental that students
acquire the rudimentary programming skills that will be required in case of
victory. Lastly, conclusions must contain the team’s expectations and
motivation, which play a significant role in the jury’s decision. The BL4S
organisers are always available to answer questions that the teams might have
during the preparation of their proposals. Many teams contact them to discuss
the feasibility of their experiments or practical problems that they
encounter.

In the final phase of the competition, a jury consisting of more than 50
volunteers selects the teams that are invited to a research institute to
perform their experiment together with support scientists. Prior to the visit,
the winning teams work remotely with the BL4S scientists to refine their
experiments and perform a detailed planning.

The beam time of the winning teams usually happens just after the summer, and
the students have 12 full days of access to the experimental area to perform
their measurements, supervised by the support scientists. During their stay,
they work as a team of professional scientists would do and they complement
their scientific experience with visits and lectures.

After taking the data at the beam line, the teams are encouraged to analyse
their data to answer the scientific question of the initial proposal, and to
write a paper. During this phase, the team members stay in close contact with
the BL4S support scientists and the team coaches.

##  3 The EXTRA experiment

The EXTRA (Electron X-ray Transition RAdiation) experiment is designed to
study the transition radiation (TR) [3] emitted by fast electrons and
positrons crossing different radiators.

Highly relativistic particles crossing the boundary between materials with
different dielectric constants can produce TR in the X-ray region. However,
since the yield of TR photons emitted at a single interface is considerably
small (it is of the order of the fine structure constant $\alpha\approx
1/137$), multiple boundaries are needed to enhance the X-ray production.
Periodic radiators, consisting of stacks of thin foils of dielectric material
separated by thicker air gaps, are commonly used in transition radiation
detectors (TRDs) [4].

The main features of the TR emitted by a periodic radiator depend on the
kinematic properties of the radiating particles and on the radiator
properties. They can be summarized as follows [5]:

  1. 1.

The effective TR photon emission starts at a threshold Lorentz factor, which
is given by $\gamma_{thr}=d_{1}\omega_{1}/c$, where $d_{1}$ is the thickness
of the foils, while $\omega_{1}$ is the plasma frequency of the foil material.

  2. 2.

The TR emission increases with the Lorentz factor $\gamma$ until it reaches
saturation at $\gamma_{sat}=\gamma_{thr}\sqrt{d_{2}/d_{1}}$, where $d_{2}$ is
the thickness of the air gaps.

  3. 3.

Most of the TR energy is emitted near the energy
$\hbar\omega_{max}=\hbar\omega_{1}^{2}d_{1}/2\pi c$.

  4. 4.

The angular distribution of TR photons exhibits a few maxima and extends up to
$\theta_{max}=\sqrt{1/\gamma^{2}+\omega_{1}^{2}/\omega^{2}}$.

Figure 1: Schematic view of the experimental setup.

Figure 2: Pictures of the experimental setup.

We have designed an experimental setup to measure the energy spectra and the
angular distributions of the TR X-rays emitted by fast electrons and positrons
crossing different radiators. Similar measurements were performed in the past
at the CERN SPS with beams of $20\hbox{$\,{\rm GeV/c}$}$ electrons and of
$120$, $180$ and $290\hbox{$\,{\rm GeV/c}$}$ muons, using silicon strip
detectors [6], silicon pixel detectors [7, 8] and GaAs pixel detectors [9, 8].
Parallel to the measurements, an effort to develop accurate Monte Carlo
simulations of the TR process is being carried out [10]. One of the goals of
these activities is that of exploiting TR for the identification of charged
hadrons in the $\,{\rm TeV}$ energy region [11]. In this region all hadrons
have Lorentz factor exceeding the typical threshold values for TR production
(usually $\gamma_{thr}\sim 500\div 1000$), and the simultaneous measurement of
the energies and of the emission angles of TR X-rays can help to discriminate
among different hadron species.

Our measurements were performed at the DESY II Test Beam Facility [2] area
TB21, using a beam of either electrons or positrons with momenta in the range
from $1$ to $6\hbox{$\,{\rm GeV/c}$}$. A scheme of the setup is shown in Fig.
1, while pictures of the setup are shown in Fig. 2. The radiator is followed
by a Timepix3 assembly containing a thin silicon pixel sensor, which is used
to detect the TR X-rays. A downstream beam telescope, composed by an array of
six silicon pixel detectors, is used to reconstruct the tracks of the beam
particles [12]. A set of two trigger scintillators, located downstream of the
last plane of the beam telescope, is used for triggering the data acquisition.

Radiator | Foil/gap material | d1 ($\mu$m) | d2 ($\mu$m) | Nf
---|---|---|---|---
EXTRA | polyethylene/air | 23 | 500 | 150
INFN | polyethylene/air | 25 | 300 | 155
CERN | polyethylene/air | 25 | 240 | 190
Table 1: Parameters of radiators used in the beam test: $d_{1}$ and $d_{2}$ are the thickness of the foils and the gap respectively; $N_{f}$ is the number of foils.  Radiator | distance ($\,{\rm cm}$) | Beam particle | Beam momenta ($\,{\rm GeV/c}$)
---|---|---|---
EXTRA | $40.5$ | $e^{-}$ | $1,2,3,4,5,6$
$88.0$ | $e^{-}$ | $1,2,3,4,5,6$
$132.0$ | $e^{-}$ | $1,2,3,4,5,6$
INFN | $88.9$ | $e^{-}$ | $1,2,3,4,5,6$
CERN | $88.4$ | $e^{-}/e^{+}$ | $1,2,3,4,5$
Table 2: Summary of the data taking configurations. For each radiator the beam
particle, their momenta and the distance between the radiator and the X-ray
detector are reported.

In our experiment we used three different radiators, which in the following
will be labelled as "EXTRA", "INFN" and "CERN" respectively. Their features
are summarized in Tab. 1. In particular, the EXTRA radiator was assembled for
this measurement by the students at the Liceo Scientifico "A. Scacchi" in
Bari. Fig. 3 shows some picture taken during the assembly of this radiator.
The INFN and CERN radiators were borrowed from the Bari INFN Group and were
used in a beam test campaign performed in 2006 [13].

With these radiators, several measurements were performed, changing the beam
composition and momentum, and the distance between the radiator and the X-ray
detector. The different data taking configurations are summarized in Tab. 2.

Figure 3: Assembly of the EXTRA radiator at the Liceo Scientifico "A.
Scacchi".

The TR X-rays were detected by a $100\hbox{$\,{\rm\mu m}$}$ thick silicon
sensor, bump-bonded to a Timepix3 readout chip [14], consisting of a pixel
matrix of $256\times 256$ pixels with a pitch of $55\hbox{$\,{\rm\mu m}$}$.
This silicon detector assembly was placed such that the sensor faces the
radiator to mitigate prior absorption in the readout chip. The sensor of the
assembly with the ID W5_E2 was operated at a bias voltage of $-21\hbox{$\,{\rm
V}$}$ to ensure full depletion [15].

The pixel pitch of the silicon sensor and its distance from the radiator
determine the minimum detectable angular separation $\theta_{min}$ of TR
X-rays from the direction of the radiating particles, as they should be
separated by at least one pixel. Its value is in fact given by
$\theta_{min}\gtrsim w/d$, where $w=55\hbox{$\,{\rm\mu m}$}$ is the pixel
pitch and $d$ is the distance of the silicon detector from the radiator. We
also remark that a fraction of the TR photons emitted at angles
$\theta\lesssim 3w/d$ from the particles will be suppressed, since the
particles can yield detectable signals on clusters of a few adjacent pixels.
The configurations with larger distances allow to detect smaller angular
separations; however, due to the X-ray absorption in the radiator and in the
air gap between the radiator and the sensor, the number of detected TR X-rays
will decrease with the distance from the radiator and the angular resolution
will deteriorate due to multiple Coulomb scattering of the primary particles
in air.

While the TR X-rays are likely absorbed by the front sensor (the photoelectric
absorption probability for $10\hbox{$\,{\rm keV}$}$ X-rays in a
$100\hbox{$\,{\rm\mu m}$}$ thick silicon layer is $\sim 54\%$ [16]), the
radiating charged particles traverse the detector and leave an ionization
track in the detectors of the beam telescope, which consists of an array of
six regularly spaced silicon pixel detectors. By minimizing the space in-
between the sensor planes, the scattering in air is limited, thus enabling a
track resolution of a few $\,{\rm\mu m}$ extrapolated to the Timepix3 detector
[12], which is more than sufficient for an identification of the charged
particle among two or more clusters in the Timepix3 detector with cluster
distances larger than a pixel pitch.

Finally, the two scintillators, approximately shadowing the size of the
telescope sensor planes and located at the end of the beam line, are used for
triggering the data acquisition.

The data acquisition was performed using the software framework EUDAQ2 [17],
which integrates the control and readout of the Timepix3 assembly and the beam
telescope, and features a graphical user interface for the configuration of
connected devices, starting and stopping runs and data storage. An AIDA TLU
[18] was used to form a trigger signal as a coincidence of the signals from
the two scintillators while enabling a busy-handshake with the detectors.

##  4 Data analysis

###  4.1 Conversion & Clustering

The raw data contains a collection of hit pixels per detector plane per
trigger, which defines a so-called "event", including the corresponding pixel
addresses; for the data from the Timepix3 assembly, the corresponding
information on the energy deposit, in form of a digitised signal, is also
stored, while for the beam telescope no charge information is recorded. The
collected data are converted to the ROOT TTree format [19] using the data
analysis framework Corryvreckan [20]. In addition, this software performs a
clustering procedure, which identifies adjacent hit pixels and connects them
to form a so-called "cluster" under the hypothesis that pixel hits in one
cluster are caused by a single incident particle. The cluster center, as an
estimation on the incidence position of the particle, is calculated either as
the center-of-gravity using the charge information, or as the arithmetic mean
of the pixel hit positions in case of binary hit information.

The energy calibration of the silicon pixel detector is performed assuming
that the most probable energy loss of $5\hbox{$\,{\rm GeV/c}$}$ electrons
crossing a $100\hbox{$\,{\rm\mu m}$}$ thick silicon layer is
$25.41\hbox{$\,{\rm keV}$}$. This value has been calculated using a dedicated
Monte Carlo simulation for the calculation of the energy losses of charged
particles in thin silicon absorbers [21, 22].

###  4.2 Detector alignment procedure

The positions of the clusters in each silicon detector are evaluated in the
local detector reference frame, with the $z$-axis oriented along the beam
direction and the $x-y$ plane corresponding to the detector plane, with the
origin in the center of the detector. In the global reference frame the
$z$-axis is also directed along the beam direction, and the detectors are
disposed on planes parallel to the $x-y$ plane, with their centers at the
coordinates $(x_{0}^{i},y_{0}^{i},z_{0}^{i})$. Due to mechanical tolerances in
the assembly of the detectors, the coordinates $(x_{0}^{i},y_{0}^{i})$ are
slightly misaligned with respect to the reference values $(0,0)$.

A dedicated alignment run has been therefore performed to evaluate the
coordinates $(x^{i}_{0},y^{i}_{0})$ of the centers of the silicon detectors
(the index $i=0$ refers to the Timepix3 sensor, while the indices $i=1\ldots
6$ refer to the detectors of the beam telescope). The alignment run has been
performed removing the radiator from the beam line and using $5\hbox{$\,{\rm
GeV/c}$}$ electrons.

We have implemented an iterative alignment procedure selecting a sample of
events with only one cluster in each silicon detector. This choice is aimed to
select events with only one electron track across all the detectors. In the
first iteration we assume $x_{0}^{i}=0$ and $y_{0}^{i}=0$ for all detectors.
We fit all the tracks with a straight line and, for each track, we evaluate
the residuals in each detector as $r_{x}^{i}=x^{i}-x^{i}_{fit}$ and
$r_{y}^{i}=y^{i}-y^{i}_{fit}$, where $(x^{i},y^{i})$ and
$(x^{i}_{fit},y^{i}_{fit})$ are respectively the true and fitted positions of
the cluster in the $i$-th detector. We then build the distributions of the
residuals $r_{x}^{i}$ and $r_{y}^{i}$ and, in the next iteration, we set
$x_{0}^{i}=-\mu_{x}^{i}$ and $y_{0}^{i}=-\mu_{y}^{i}$, where $\mu_{x}^{i}$ and
$\mu_{y}^{i}$ are the average values of these distributions. The iterative
procedure is terminated when $|\mu_{x}^{i}|<1\hbox{$\,{\rm\mu m}$}$ and
$|\mu_{y}^{i}|<1\hbox{$\,{\rm\mu m}$}$ for all detectors. Convergence is
reached after the second iteration.

Figure 4: Distributions of the residuals in the silicon detector equipped with
the Timepix3 chip after the alignment procedure.

Fig. 4 shows the distributions of the residuals in the silicon detector
equipped with the Timepix3 chip after the alignment procedure. The RMS of the
residual distributions in both the $x$ and $y$ views are of about
$10\hbox{$\,{\rm\mu m}$}$.

Figure 5: Distributions of the direction cosines of the electron tracks in the
silicon detector and in the beam telescope in the alignment run.

Fig. 5 shows the distributions of the direction cosines of the electron tracks
in the alignment run. We see that the average values of the direction cosines
$c_{x}$ and $c_{y}$ are slightly different from zero. This result implies that
the $z$-axis of our reference frame is not perfectly aligned with the
direction of the beam. The tilt angle can be estimated from the average value
of $c_{z}$, and is of about $5\hbox{$\,{\rm mrad}$}$. Finally, from the values
of the RMS of the distributions of $c_{x}$ and $c_{y}$ we can deduce that the
beam divergence is of about $1\hbox{$\,{\rm mrad}$}$ in both the $x$ and $y$
directions.

###  4.3 Data selection and analysis

As discussed in Sec. 3, several runs in different configurations have been
taken, by changing the beam composition and momentum, the radiator and its
distance from the silicon pixel detector.

In each of these runs we have selected events with at least one cluster in the
silicon pixel sensor and at least 3 clusters in different detectors of the
beam telescope. This choice is motivated by the need of identifying, among the
clusters in the silicon sensor, the one originated by the ionization energy
deposit of the beam particle and those eventually originated by the absorption
of TR X-rays produced in the upstream radiator.

Figure 6: Distribution of the total number of clusters in the beam telescope
for all the runs performed with electrons crossing the EXTRA radiator, placed
at a distance of $88.9\hbox{$\,{\rm cm}$}$ from the silicon pixel sensor.

Fig. 6 shows the distribution of the total number of clusters in the detectors
of the beam telescope for all the runs performed with electrons crossing the
EXTRA radiator, which was placed at a distance of $88.9\hbox{$\,{\rm cm}$}$
from the silicon pixel sensor. As expected, the distribution is peaked at 6
clusters, corresponding to clean electron tracks, yielding one cluster in each
detector. Events with less than 6 clusters can be originated from
inefficiencies of some detectors in the beam telescope or from beam particles
which do not cross all the telescope planes. Events with more than 6 clusters
can be originated from delta rays accompanying the primary electron track or
from TR X-rays passing through the upstream silicon sensor and being absorbed
in any detector of the silicon telescope. We also see two peaks, at 12 and 18
clusters respectively, which include less than $1\%$ of the total number of
events, and which likely correspond to double and triple electron tracks.

The clusters in the detectors of the beam telescope are used to reconstruct
the tracks of the beam particles in the telescope. To select events with
single electron (positron) tracks, we require less than $10$ clusters in the
beam telescope. Candidate tracks are built by selecting all the possible
cluster combinations with only one cluster per plane of the telescope. The
clusters of each candidate track are then fitted with a straight line and the
$\chi^{2}$ of the fit is evaluated. The track with the best $\chi^{2}$ is then
selected.

Figure 7: Distribution of the distances of the "particle clusters" from the
track in the silicon sensor for the runs performed with electrons crossing the
EXTRA radiator, placed at a distance of $88.9\hbox{$\,{\rm cm}$}$ from the
silicon pixel sensor.

Once the track of the radiating particle in the beam telescope is
reconstructed, we evaluate the coordinates $(x_{track},y_{track})$ of its
intersection with the upstream silicon pixel sensor. Then, if more than one
cluster is found in the sensor, the cluster nearest to the track is associated
to the particle ("particle cluster"), while other clusters are associated to
possible TR X-rays ("X-ray clusters"). Clearly, if only one cluster is found
in the silicon pixel sensor, it is associated to the particle and no X-rays
are detected. Fig. 7 shows the distribution of the distances of the particle
clusters in the silicon pixel sensor from the track reconstructed in the beam
telescope obtained in the runs performed with electrons crossing the EXTRA
radiator. We see that $94\%$ of the particle clusters are within
$100\hbox{$\,{\rm\mu m}$}$ from the track and only $0.1\%$ of them are at
distances above $500\hbox{$\,{\rm\mu m}$}$.

Figure 8: Summary of the results obtained in the runs with the EXTRA radiator
at $40.5\hbox{$\,{\rm cm}$}$ from the Timepix3 detector. Top panel:
distribution of the relative positions of the TR photons (X-ray clusters) with
respect to the electrons (particle clusters); middle panel: distribution of
X-ray energies as a function of their angular separation from the electrons;
bottom panel: electron and X-ray energy distributions.

Figure 9: Summary of the results obtained in the runs with the EXTRA radiator
at $88\hbox{$\,{\rm cm}$}$ from the Timepix3 detector. Top panel: distribution
of the relative positions of the TR photons (X-ray clusters) with respect to
the electrons (particle clusters); middle panel: distribution of X-ray
energies as a function of their angular separation from the electrons; bottom
panel: electron and X-ray energy distributions.

Figure 10: Summary of the results obtained in the runs with the EXTRA radiator
at $132\hbox{$\,{\rm cm}$}$ from the Timepix3 detector. Top panel:
distribution of the relative positions of the TR photons (X-ray clusters) with
respect to the electrons (particle clusters); middle panel: distribution of
X-ray energies as a function of their angular separation from the electrons;
bottom panel: electron and X-ray energy distributions.

##  5 Results

In Figs. 8, 9 and 10 the results obtained in the runs with the EXTRA radiator
are summarized. The plots in each figure correspond to the configurations with
the silicon detector placed at the distances of $40.5\hbox{$\,{\rm cm}$}$,
$88\hbox{$\,{\rm cm}$}$ and $132\hbox{$\,{\rm cm}$}$ from the radiator
respectively. The plots are built selecting events with the particle cluster
inside a square of a $3\times 3\hbox{$\,{\rm mm^{2}}$}$ area, in the centre of
the Timepix3 detector. All the distributions shown in the above plots are
normalized to the total number of selected events.

The top panels of each figure show the distributions of the relative positions
of the TR X-rays (evaluated from the X-ray clusters) with respect to the
radiating electron (evaluated from the particle cluster). As expected, TR
photons tend to accumulate in rings centered on the position of the radiating
particle and the number of photons per electron increases with the beam
momentum (and consequently with the Lorentz factor of the radiating
particles).

The central panels show the distributions of the TR X-ray energies as a
function of their angular separation from the radiating particle. Most X-rays
are emitted at angles $\theta\lesssim 2\hbox{$\,{\rm mrad}$}$ from the
radiating particle, with energies peaked at energies $<10\hbox{$\,{\rm
keV}$}$. A second peak of X-rays emitted at angles $\sim 3.5\hbox{$\,{\rm
mrad}$}$ and with the same energies as the first peak can also be seen, and it
becomes more evident as the beam momentum increases.

Finally, the bottom panels show the energy distributions of the absorbed TR
X-rays compared with the distributions of the energies deposited by the parent
electrons in the Timepix3 detector. As discussed in Sec. 4.1, the energy
losses of the electrons follow Landau distributions with a most probable value
of $25.4\hbox{$\,{\rm keV}$}$, while X-ray energies are peaked at less than
$10\hbox{$\,{\rm keV}$}$. We see that the area of the X-ray energy spectra
increases with increasing electron momentum. This behaviour is expected since
the spectra are normalized to the total number of electrons and the TR yield
increases with the Lorentz factor of the radiating particle.

Figure 11: Average number of TR photons as a function of electron beam
momentum for the three radiator types and for the different distances from the
Timepix3 detector. The dashed lines show the predictions for the different
configurations. The results obtained in a run without radiator and in two runs
with "dummy" radiators are also shown.

A summary of the results obtained in all the configurations explored is shown
in Fig. 11. The average number of detected TR X-rays per electron is shown as
a function of the beam momentum. We see that for all configurations the number
of detected photons increases with the beam momentum and saturates above
$4\hbox{$\,{\rm GeV/c}$}$. This behavior is expected, since the threshold
Lorentz factor for all radiators is $\gamma_{thr}\simeq 10^{3}$ and the
saturation Lorentz factors are in the range $4\div 5\times 10^{3}$. Comparing
the results obtained with the EXTRA radiator in the different configurations
we see that the average number of detected TR X-rays decreases when the
radiator-detector distance is increased. The increase of the distance causes
an increase of the X-ray absorption in the air gap between the radiator and
the detector, which is not compensated by the lower minimum detectable angle
between the photons and the radiating particles. We remark here that the
results shown in Fig. 11 referred to the CERN radiator have been obtained from
a joint analysis of the data samples collected with both the electron and
positron beams (see Tab. 2). This choice is motivated by the fact that the
separate analyses of the electron and positron data samples yield the same
results. This feature was expected, since the properties of TR are independent
of the sign of the charge of the radiating particle.

The experimental results shown in Fig. 11 are compared with the predictions
obtained by folding the TR yield, evaluated with the theoretical formulae for
regular radiators [4, 23] with the X-ray absorption probabilities in the air
gap between the radiator and the Timepix3 detector and in the silicon layer of
the detector. The theoretical curves seem to be in a reasonable agreement with
the experimental results.

Finally, we have performed some control runs to check our results. A run with
$5\hbox{$\,{\rm GeV/c}$}$ electrons without any radiator was performed to
evaluate the possible contamination to the detected TR signal from
bremsstrahlung photons produced in the upstream materials and accompanying the
beam particles and the possible contamination from noisy pixels. In this run
we found about $0.03$ X-rays per electron; in addition, since all X-ray
clusters are found very close to the particle cluster, the contamination from
noisy pixels can be considered negligible. We also performed two additional
runs with $6\hbox{$\,{\rm GeV/c}$}$ electrons, in which we replaced the
radiator with some "dummy" radiators: in particular, we used a set of paper
towels, which were arranged in a stack simulating a regular radiator, and a
piece of sponge, which simulates an irregular radiator333Irregular radiators
made of foams or fiber mats are sometimes used in TRDs.. In both cases we
observed a TR signal of about $0.17$ X-rays per electron, indicating that TR
can be generated also at everyday objects. At the same time it should be noted
that the design and quality of a radiator is crucial to optimise the photon
yield and thus the signal in corresponding detector systems.

##  6 Conclusions

In the framework of the BL4S competition we have designed and implemented an
experiment to measure the TR emitted by fast electrons and positrons crossing
different kind of radiators. The measurement has been performed at the DESY II
Test Beam Facility area TB21, using electrons and positron beams with momenta
up to $6\hbox{$\,{\rm GeV/c}$}$. We have measured the energy spectra and the
angular distribution of the TR X-rays using a $100\hbox{$\,{\rm\mu m}$}$ thick
pixel silicon detector, with a pitch of $55\hbox{$\,{\rm\mu m}$}$. The
experimental results are well reproduced by the theoretical curves obtained
from standard TR models.

BL4S has offered the students the chance to be actively involved in all the
aspects of an experimental research: during the preparation of the proposal,
they have learned how to design an experimental setup, optimizing the
detectors available for the measurement; after their proposal was selected,
they have been involved in the design and in the assembly of their own
radiator; then, at DESY, they had the chance to run a real beam test; finally,
they have taken part to the analysis of the data collected in the test.
However, the most important educational result of this experience is that the
students learned how to apply the scientific approach not only in the field of
research, but also to the solution of everyday life challenges.

## Acknowledgments

The measurements leading to these results have been performed at the Test Beam
Facility at DESY Hamburg (Germany), a member of the Helmholtz Association
(HGF). The Timepix3 assembly used for the measurements within this publication
is provided by the CLICdp group at CERN.

The members of the EXTRA team thank the CERN and DESY support scientists, the
beamline scientists, the volunteers and the BL4S organisers who helped them
during the preparation and the implementation of their experiments. All the
scientists involved in the competition dedicated a lot of their time to answer
all the questions the students had, giving them precious advises for their
future career. The team was really pleased to find such wonderful people, who
showed them what unconditional love for science really means.

A big thank to the Teomizli team from Mexico, the other winning team of the
BL4S 2021. Meeting peers from the other side of the world and work with them
as a unique team of scientists has been an enriching opportunity.

Beamline for Schools is an education and outreach project funded by the CERN &
Society Foundation and supported by individual donors, foundations and
companies. In 2021, the project was funded by the Wilhelm and Else Heraeus
Foundation. Additional contributions have been received from the Arconic
Foundation, Amgen Switzerland AG, and the Ernest Solvay Fund managed by the
King Baudouin Foundation.

The EXTRA team also acknowledges financial support from CERN and DESY for
their participation to the beam test campaign.

The EXTRA team thanks B. Fanti, I. Iusco, D. Ricchiuti and all the personnel
of the Liceo Scientifico “A. Scacchi” for their support to the project
activities.

## References

  * [1] Sarah Aretz, Cristóvão Beirão da Cruz e Silva, Markus Joos, Paul Schütze, and Marcel Stanitzki.  An Overview of the CERN Beamline for Schools Competition.  The Physics Educator, 02(01):2050001, 2020.
  * [2] R. Diener, J. Dreyling-Eschweiler, H. Ehrlichmann, I.M. Gregor, U. Kötz, U. Krämer, N. Meyners, N. Potylitsina-Kube, A. Schütz, P. Schütze, and M. Stanitzki.  The DESY II test beam facility.  Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, 922:265–286, 2019.
  * [3] V. L. Ginzburg and I. M. Frank.  Radiation of a uniformly moving electron due to its transition from one medium into another.  J. Phys. (USSR), 9:353–362, 1945.
  * [4] X. Artru, G. B. Yodh, and G. Mennessier.  Practical Theory of the Multilayered Transition Radiation Detector.  Phys. Rev. D, 12:1289, 1975.
  * [5] P.A. Zyla et al.  Review of Particle Physics.  PTEP, 2020(8):083C01, 2020.  and 2021 update.
  * [6] J. Alozy et al.  Identification of particles with Lorentz factor up to $10^{4}$ with Transition Radiation Detectors based on micro-strip silicon detectors.  Nucl. Instrum. Meth. A, 927:1–13, 2019.
  * [7] J. Alozy et al.  Studies of the spectral and angular distributions of transition radiation using a silicon pixel sensor on a Timepix3 chip.  Nucl. Instrum. Meth. A, 961:163681, 2020.
  * [8] F. Dachs et al.  Transition radiation measurements with a Si and a GaAs pixel sensor on a Timepix3 chip.  Nucl. Instrum. Methods Phys. Res. A, 958:162037, 2020.
  * [9] J. Alozy et al.  Registration of the transition radiation with GaAs detector: Data/MC comparison.  J. Phys. Conf. Ser., 1690(1):012041, 2020.
  * [10] A. A. Savchenko et al.  Fine structure of angular distribution of x-ray transition radiation from multilayered radiator in Geant4.  JINST, 15(06):C06024, 2020.
  * [11] Michael Albrow.  A Very Forward Hadron Spectrometer for the LHC and Cosmic Ray Physics.  PoS, EDSU2018:048, 2018.
  * [12] Jansen, Hendrik, Spannagel, Simon, Behr, Jörg, Bulgheroni, Antonio, Claus, Gilles, Corrin, Emlyn, Cussans, David, Dreyling-Eschweiler, Jan, Eckstein, Doris, Eichhorn, Thomas, Goffe, Mathieu, Gregor, Ingrid Maria, Haas, Daniel, Muhl, Carsten, Perrey, Hanno, Peschke, Richard, Roloff, Philipp, Rubinskiy, Igor, and Winter, Marc.  Performance of the EUDET-type beam telescopes.  Eur. Phys. J. Techn. Instrum., 3(1):7, 2016.
  * [13] M. Brigida et al.  Beam test results with a reduced scale Silicon Transition Radiation Detector prototype.  Nucl. Instrum. Meth. A, 577:519–522, 2007.
  * [14] T Poikela, J Plosila, T Westerlund, M Campbell, M De Gaspari, X Llopart, V Gromov, R Kluit, M van Beuzekom, F Zappon, V Zivkovic, C Brezina, K Desch, Y Fu, and A Kruth.  Timepix3: a 65K channel hybrid pixel readout chip with simultaneous ToA/ToT and sparse readout.  Journal of Instrumentation, 9(05):C05013–C05013, may 2014.
  * [15] Niloufar Alipour Tehrani.  Test-beam measurements and simulation studies of thin pixel sensors for the CLIC vertex detector.  PhD thesis, ETH Zurich, Zurich, 2017.
  * [16] Berger M., Hubbell J., Seltzer S., Coursey J., and D. Zucker.  Xcom: Photon cross section database.  http://physics.nist.gov/xcom.
  * [17] Y. Liu, M.S. Amjad, P. Baesso, D. Cussans, J. Dreyling-Eschweiler, R. Ete, I. Gregor, L. Huth, A. Irles, H. Jansen, K. Krueger, J. Kvasnicka, R. Peschke, E. Rossi, A. Rummler, F. Sefkow, M. Stanitzki, M. Wing, and M. Wu.  EUDAQ2—A flexible data acquisition software framework for common test beams.  Journal of Instrumentation, 14(10):P10033–P10033, oct 2019.
  * [18] P. Baesso, D. Cussans, and J. Goldstein.  The AIDA-2020 TLU: a flexible trigger logic unit for test beam facilities.  Journal of Instrumentation, 14(09):P09019–P09019, sep 2019.
  * [19] R. Brun and F. Rademakers.  ROOT: An object oriented data analysis framework.  Nucl. Instrum. Meth. A, 389:81–86, 1997.
  * [20] D. Dannheim, K. Dort, L. Huth, D. Hynds, I. Kremastiotis, J. Kröger, M. Munker, F. Pitters, P. Schütze, S. Spannagel, T. Vanat, and M. Williams.  Corryvreckan: a modular 4D track reconstruction and analysis software for test beam data.  Journal of Instrumentation, 16(03):P03008, mar 2021.
  * [21] Hans Bichsel.  Straggling in Thin Silicon Detectors.  Rev. Mod. Phys., 60:663–699, 1988.
  * [22] M. Brigida et al.  A new Monte Carlo code for full simulation of silicon strip detectors.  Nucl. Instrum. Meth. A, 533:322–343, 2004.
  * [23] C. W. Fabjan and W. Struczinski.  Coherent Emission of Transition Radiation in Periodic Radiators.  Phys. Lett. B, 57:483–486, 1975.


# RatingScaleReduction package: stepwise rating scale item reduction without
predictability loss

by Waldemar W. Koczkodaj and Alicja Wolny–Dominiak

###### Abstract

This study presents an innovative method for reducing the number of rating
scale items without predictability loss. The “area under the receiver operator
curve method” (AUC ROC) is used to implement in the RatingScaleReduction
package posted on CRAN
111https://cran.r-project.org/web/packages/RatingScaleReduction/index.html.
Several cases have been used to illustrate how the stepwise method has reduced
the number of rating scale items (variables).

Keywords: rating scale, receiver operator characteristic, ROC, AUC, scale
reduction.

##  1 Introduction

Rating scales (also called assessment scale and ”the scale” in this study) are
used to elicit data about quantitative entities. Often, predictability of
rating scales (also called “assessment scales”) can be improved. Rating scales
often use values “1 to 10” and some rating scales may have over 100 items
(questions) to rate. Sometimes, the scale is called a survey or a
questionnaire. A questionnaire is a tool for data collection while a survey
may not necessarily be conducted by questionnaires since some surveys may be
conducted by interviews or by data gathered from web pages. In fact, the main
and very important distinction between the scale and both questionnaires and
surveys is that the scale is used for assessment (as in “the scale of
disaster”) while questionnaires and surveys may be used to only collect data.
In other words, some kind of “summation procedure” must be provided for
questionnaires or surveys to become rating scales.

The recent popularity of rating scales is due to various “Customer Reviews” on
the Internet where five stars are often used instead of ordinal numbers.
However, the most important examples of rating scales are questionnaires used
in examinations. We may risk a statement that the accelerated progress of
granting academic degrees can be linked to a better use of rating scales.

Rating scales are predominantly used to express our subjective assessments
such as “on the scale 1 to 5 express your preference”: “strongly agree to
strongly disagree” (with 3 as ’neutral” preference). The importance of
subjectivity processing probably inspired introduction of the idea of bounded
rationality, proposed by Herbert A. Simon (the Nobel Prize winner), as an
alternative basis for the mathematical modeling of decision making. It is
often expressed as “good enough is perfect” and gained popularity in the
software industry where frequent updates are common. Objective data are more
commonly used in so called “strict sciences” while processing subjectivity is
still under development. It is worth noticing that the objectivity is
illusive. Often, the difference between subjectivity and objectivity is a
matter of an arbitrary decision. For example, an item listed for sale for, let
us say, 100,000 monetary units, will be very likely sold for 99,999 of such
units if such an offer is made. If so, one may also not resist accepting
99,998 monetary units and so on. Setting a limit (so called, “the bottom
line”) is often a highly subjective decision. Scales help in many cases, but a
large number of items (questions) is often a discouragement for its use.

It seems that one of the first successful rating scale reduction (RSR) took
place in [13]. The 17-item Hamilton Rating Scale for Depression (HAM-D17) was
used to derive even a more reduced version (Ham-D7) with seven items.
According to [10], “The clinical utility of the HAM-D17 is hampered, in part,
by the length of time required to administer the interview and by the lack of
inter-rater reliability.”

##  2 Heuristic algorithm

A heuristic is, in essence, a simplified method for solving a problem more
quickly when well-established methods fail to find a sound algorithm. Usually,
this is achieved by the “good enough is perfect” approach, mentioned in the
introduction, as characterization of the heuristic solution. Heuristics are
expected to produce a reasonable solution when a time frame and accuracy are a
problem. The “good enough”solution for an urgent problem is commonly practiced
in computer science. It is usually not the best solution to our problem but it
may still be of great value. For example, the traveling salesman problem
(TSP), often formulated as “find the shortest possible route to visit each
city exactly once and return to the origin city”, cannot be solved for 50 or
more cities by verifying all possible combinations since the total number of
such combinations would easily exceeds the number of atoms in the entire
Universe. Using heuristics, we can solve TSP for millions of cities with the
accuracy of a small fraction of 1%. Most heuristics produce results by
themselves but many are used in conjunction with optimization algorithms to
improve their efficiency (e.g., differential evolution).

In our case, the number of possible combinations for a rating scale with 100
items (which is not uncommon) is a “cosmic number” hence the complete search
must be ruled out. Computing the area under the receiver characteristic curve
for all items is the basis for our heuristic. Common sense dictates that the
contribution of the individual items to the overall value of the area under
the receiver characteristic curve needs to be somehow utilized. We have
decided on a stepwise heuristic. Certainly, the results need to be verified
and used only if the item reduction is substantial.

##  3 The package description

Rating scales are used to elicit data about qualitative entities (e.g.,
research collaboration). This study presents an innovative method for reducing
the number of rating scale items without predictability loss. The “area under
the receiver operator curve method” (AUC ROC) is used. The presented method
has reduced the number of rating scale items (variables) to 28.57% (from 21 to
6) making over 70% of collected data unnecessary.

Results have been verified by two methods of analysis: Graded Response Model
(GRM) and Confirmatory Factor Analysis (CFA). GRM revealed that the new method
differentiates observations of high and middle scores. CFA proved that the
reliability of the rating scale has not deteriorated by the scale item
reduction. Both statistical analysis evidenced usefulness of the AUC ROC
reduction method.

Rating scales (also called assessment scale) are used to elicit data about
quantitative entities. Often, the predictability of rating scales (also called
“assessment scales”) could be improved. Rating scales often use values: “1 to
10” and some rating scales may have over 100 items (questions) to rate. Other
popular terms for rating scales are: survey and questionnaire, although a
questionnaire is a method of data collection while survey may not necessarily
be conducted by questionnaires. Some surveys may be conducted by interviews or
by analyzing web pages. Rating itself is very popular on the Internet for
“Customer Reviews” where five stars (e.g., Amazon.com) are often used instead
of ordinal numbers. One may regard such rating as a one item rating scale.

In computer science and mathematical optimization, a heuristic is a technique
designed for solving a problem for finding an approximate solution when
classic methods fail to find any exact solution. Often, finding such methods
is achieved by trading completeness, accuracy, or optimality, for speed.

The main objective of a heuristic is to produce a solution that is good enough
to solve our problem. The solution may not be the “best” solution and it may
only approximate the solution since the optimal solution may require a
prohibitively long time. The traveling salesman problem and virus scanning are
probably the most recognized problems where the need for using heuristics is
evident. In both cases, the complete search for the optimal search would take
thousands of years using the fastest computers built. One of the shortest
heuristics may be “22/7” as an approximation of $\Pi$ constant with two
decimal points (3.14), as it is easier to remember and sometimes easier to
use.

Herbert A. Simon originally was the proponent of bounded rationality. In
practice, it means that human judgments are based on heuristics. He is the
only person who received both the Nobel prize and the Turing prize.

Data collected by a rating scale with a fixed number of items (questions) are
stored in a table with one decision (in our case, binary) variable. The
parametrized classifier is usually created by total score of all items. The
outcome of such rating scales is usually compared to external validation
provided by assessing professionals (e.g., grant application committees).

Our approach not only reduces the number of items, but also sequences them
according to the contribution to predictability. It is based on the Receiver
Operator Characteristic (ROC), which gives individual scores for all examined
items. The term “receiver operating characteristic” (ROC), or “ROC curve” was
coined for a graphical plot illustrating the performance of radar operators
(hence “operating”). A binary classifier represented absence or presence of an
enemy aircraft. It was used to plot the fraction of true positives out of the
total actual positives (TPR = true positive rate) vs. the fraction of false
positives out of the total actual negatives (FPR = false positive rate).
Positive instances (P) and negative instances (N) for some condition are
computed and stored as four outcomes of a 2 contingency table or confusion
matrix, as follows:

Table 1: The confusion matrix True Positives | False Positives
---|---
False Negative | True Negative

Each patient either has or does not have the disorder. The screening outcome
can be positive (classifying a patient as having the disorder) or negative
(classifying the patent as not having the disorder). The screening results for
each patient may or may not match the subject’s actual status.

In means that in the medical terminology we may have:

  * •

TP = true positive: patient is correctly identified as having the disorder,

  * •

FP = false positive: patient is incorrectly identified as having the disorder,

  * •

TN = true negative: patient with no disorder is correctly identified as not
having the disorder,

  * •

FN = false negative: patent with the disorder is incorrectly identified as not
having the disorder.

In simple terms, positive = identified and negative = rejected hence:

True positive = correctly identified examples
False positive = incorrectly identified examples
True negative = correctly rejected examples
False negative = incorrectly rejected examples

In assessment and evaluation research, the ROC curve is a representation of a
“separator” (or decision) variable. The decision variable is usually: “has a
property” or “does not have a property” or has some condition to meet
(pass/fail). The frequencies of positive and negative cases of the diagnostic
test vary for the “cut-off” value for the positivity. By changing the “cut-
off” value from 0 (all negatives) to a maximum value (all positives), we
obtain the ROC by plotting TPR (true positive rate also called sensitivity)
versus FPR (false positive also called specificity) across varying cut-offs,
which generate a curve in the unit square called an ROC curve.

According to [2], the area under the curve (the AUC or AUROC) is equal to the
probability that a classifier will rank a randomly chosen positive instance
higher than a randomly chosen negative one (assuming the ’positive’ rank
higher than ’negative’).

This can be seen as follows: the area under the curve is given by (the
integral boundaries are reversed as large T has a lower value on the x-axis)

$A=\int_{\infty}^{-\infty}y(T)x^{\prime}(T)dT=$
$\int_{\infty}^{-\infty}TPR(T)FPR^{\prime}(T)dT=$
$\int_{-\infty}^{\infty}TPR(T)P_{0}(T)dT=\langle TPR\rangle$

The angular brackets denote average from the distribution of negative samples.
AUC is closely related to the Mann-Whitney U test which tests whether
positives are ked higher than negatives. It is also equivalent to the Wilcoxon
test of ranks.

ROC method is implemented by many R packages including: pROC [11] and ROCR
[12]. There is also one interesting web application easyROC [3] giving
possibility to compute the confusion matrix and plot the curve on-line. The
RatingScaleReduction package expands this analysis to carry out the procedure
of rating scale reduction.

A package for preprocessing “messy” data into a form is easily analyzed within
R is presented in [8]. In [15], the new R package sbtools enables users direct
access to the advanced online data functionality provided by ScienceBase, the
U.S. Geological Survey’s online scientific data storage platform. It can be
used for harvesting other data sets.

##  4 Rating scale stepwise reduction procedure

The procedure follows the heuristic algorithm represented by Fig. 2.
Technically, it is an algorithm since the flowchart, represented by Fig. 2,
shows the finite namer of steps. It is, however, a heuristic algorithms since
the optimality of the presented approach cannot be guaranteed (as pointed out
in Section 2). However, the common sense dictates to select the “best”
attribute and keep adding to it the next “best” attribute where the “best” has
the meaning of the area under the curve (AUC) value since it is the
universally accepted criterion for classifiers (in statistical classification
and machine learning). A rating scale total is a classifier. The
classification in this study is regarded as an instance of supervised
learning. Briefly, it requires a training set of correctly identified examples
(observations) with the external evaluation. In our case, a trained
professional is needed to determine if a subject (a screened psychiatric
patient) had a mental disorder or not). An algorithm that implements a
concrete classification is called as a classifier. The most common way of
doing i by a rating scale is using the total of all items. Some of them may be
negative (e.g., in the Oxford Happiness Questionnaire, see [4]).

Figure 1: Rating scale stepwise reduction heuristic algorithm

In the RatingScaleReduction, the implemented algorithm (when reduced to its
minimum) uses a loop for all attributes (with the class excluded) to compute
AUC. Subsequently, attributes are sorted in the ascending order by AUC. The
attribute with the largest AUC is added to a subset of all attributes
(evidently, it cannot be empty since it is supposed to be the minimum subset S
of all attributes with the maximum AUC). We continue adding the next in line
(according to AUC) attribute to the subset S checking AUC. If it decreases, we
stop the procedure. There are a lot of checking (e.g., if the dataset is not
empty or full of replications) involved. These steps are implemented in
startAuc, totalAuc oraz rsr functions of the package.

Before running the RSR procedure the data set should be analysed to detect
replicated examples and so-called ”gray” examples. One example may be
replicated $m$ times, where $m$ is the total number of examples, so that there
are no other examples. Such situation would deviate computations and should be
early detected. Ideally, no example should be replicated but if the
replication rate is small, we can proceed to computing AUC. There is no
generally acceptable “golden rule” for the level or replication rate. Moreover
the data may contain gray examples which should also be detected, gray example
is an example for which there are another examples in the data set having
identical values on all attributes but different decision. This analysis of
data set can by carry out using functions: diffExamples and grayExamplesN,
grayExamples.

The important problem after the scale reduction by RSR procedure is to check
for the possible inclusion of the next attribute in the reduced rating scale
by maximizing AUC of all included items. In a highly unlikely scenario, all
attributes will be included in the reduced (that is, non reduced) set of
items. The reduced rating scale of one attribute may be created if there is an
identifying attribute. To test the inclusion the function CheckAttr4Inclusion
is available in the package.

##  5 RatingScaleReduction: overview of the package functions

The RatingScaleReduction package implements the above-stated stepwise
procedure using two functions of the pROC package: roc and roc.test. It works
on the data as the matrix or data.frame containing columns of attributes and
one decision column with two categories, e.g. (0,1). The rows in data.frame
represents examples in the sample. All attributes and the decision vector must
be numeric. There are to groups of functions available in the package. Because
the essence of the procedure is to set the attributes in the correct order
good practice is to enter their name using e.g. colnames in R. The first group
are dedicated to carry out the RSR procedure:

  1. 1.

startAuc(attribute, D) – compute the AUC values of every single attribute in
the rating scale.

  2. 2.

totalAuc(attribute, D, plotT=FALSE) – sort AUC values in the ascending order
and compute AUCs of running total of first $k$ attributes, $k=1,...,n$, where
$n$ is the number of attributes. Setting the argument plotT as TRUE the plot
of new AUC values is created. The horizontal line marks the max new AUC.

  3. 3.

rsr(attribute, D, plotRSR=FALSE) – the main function of the package reducing
the rating scale according the procedure illustrated by Fig. 1. Setting the
argument plotRSR as TRUE the plot of ROC curve of the sum of attributes in
reduced rating scale is created.

Additionally, the package provides second group of functions to support the
reduction procedure:

  1. 1.

CheckAttr4Inclusion(attribute, D) – carry out a statistical tests for a
difference in AUC of two correlated ROC curves: ROC1 of the sum of attributes
from reduced rating scale and ROC2 of this sum plus the next ordered
attribute. The function uses The function roc.test from the pROC is used and
all implemented tests are available, in particular delong and bootstrap.

  2. 2.

diffExamples(attribute) – search replicated examples in the data and return
the number of different examples and the number of duplicates.

  3. 3.

grayExamples(attribute, D) – produce the list of pairs of examples having
identical values on all attributes. The decision value and attributes are
produced for every pair in the data set, so the list clearly shows all gray
examples.

  4. 4.

grayExamplesN(attribute, D, N) – produce a list of examples and the numbers of
examples $j$ in the data set having identical values on all attributes for the
given example $N$.

###  5.1 Data sets used for verification and working with the package

The examples presents the capabilities of the RatingScaleReduction package.
The full code is available for download from
https://github.com/woali/RatingScaleReduction/example_Rj.r.

###  5.2 The first demonstration example: rating scale reduction

We consider the data BDI data set used in [7]. It is a rating scale for
depression BDI (Beck Depression Inventory) with 21 attributes in our
relational database. The goal in this example is to show how to reduce the BDI
rating scale in the use of three main functions of the package. The data.frame
we work on contains 21 columns with attributes and one additional column as a
decision (reality). The sample is represented with 561 examples (instances).

We start with analysis of the AUC of all 21 individual attributes of BDI scale
by the use of the function totalAuc setting the argument plotT as TRUE.

    > tauc.bdi <-totalAuc(attribute, D, plotT=TRUE)

    > tauc.bdi$summary

                AUC one variable AUC running total
    BDI_1         0.7250092         0.7250092
    BDI_14        0.7074013         0.7765412
    BDI_7         0.7014889         0.7945490
    BDI_9         0.7004614         0.8095300
    BDI_10        0.6972253         0.8131352
    BDI_15        0.6920635         0.8221669
    BDI_17        0.6742833         0.8205426
    BDI_8         0.6679833         0.8192322
    BDI_20        0.6669989         0.8198782
    BDI_5         0.6584779         0.8211333
    BDI_3         0.6556663         0.8210840
    BDI_4         0.6511874         0.8211394
    BDI_13        0.6489172         0.8210779
    BDI_12        0.6367909         0.8195583
    BDI_2         0.6312046         0.8186908
    BDI_19        0.6292851         0.8181125
    BDI_18        0.6100283         0.8160699
    BDI_6         0.6100037         0.8138489
    BDI_16        0.6059370         0.8116525
    BDI_11        0.5973422         0.8110680
    BDI_21        0.5874677         0.8117263

The R output shows the tauc.bdi$summary AUC of every single attribute in the
second column, sorted in the ascending order. The running total of AUCs is in
the thrird column. The initially selected variable (BDI_1) for the first row
is the attribute with the largest AUC. Subsequently, we add to it the variable
with the largest AUC of the remaining attributes. The process continues while
the last attribute of the scale is added.

Values in the running total (from the top to the current variable) are checked
for growing. Evidently, the value 0.725 in the first row is the same for the
running total as for the single variable (BDI_1). However, the value in the
third row (0.795) is not for variable 7 but the total of variables BDI_1,
BDI_14, and BDI_7. In particular, the value (0.812) in in the last row is for
the total of all variables. Their line plot can be easily created by setting
the totalAuc parameter plotT as TRUE. The plot for our BDI scale is
illustrated by Fig. 2. The curve peek is for variable #6 which is 15.

Figure 2: A stepwise AUC reduction method (example)

Printing the value tauc.bdi$item we receive the attribute labels in an
ascending order.

    > tauc.bdi$item

    "BDI_1"  "BDI_14" "BDI_7"  "BDI_9"  "BDI_10" "BDI_15"
    "BDI_17" "BDI_8"  "BDI_20"
    "BDI_5"  "BDI_3"  "BDI_4"  "BDI_13" "BDI_12" "BDI_2"
    "BDI_19" "BDI_18" "BDI_6"
    "BDI_16" "BDI_11" "BDI_21"

As illustrated by Fig. 2, the value of AUC of the selected subset of
attributes is increasing by adding the first six attributes labeled DBI_1,
DBI_17, DBI_7, DBI_9, DBI_10, and DBI_15. There is a slight decline by adding
the variable DBI_16. For this reason, the reduction procedure is terminated
after the first six attributes are added. The function rsr reduce the scale
automatically assuming the truncation point as the attribute that first
reaches the maximum AUC. AUC is a real value between 0 and 1. It is 0.5 for
random data but hardly ever reaches 1 since , in reality, there are always
“gray examples” in sizable data.

    > rsr.bdi <-rsr(attribute, D, plotRSR=TRUE)
    The criteria: Stop first MAX AUC
    > rsr.bdi$rsr.auc

    [1] 0.7250092 0.7765412 0.7945490 0.8095300 0.8131352 0.8221669

    $rsr.label
    [1] "BDI_1"  "BDI_14" "BDI_7"  "BDI_9"  "BDI_10" "BDI_15"

    $summary
    AUC one variable AUC running total
    BDI_1         0.7250092         0.7250092
    BDI_14        0.7074013         0.7765412
    BDI_7         0.7014889         0.7945490
    BDI_9         0.7004614         0.8095300
    BDI_10        0.6972253         0.8131352
    BDI_15        0.6920635         0.8221669

Setting the rsr parameter plotRSR as TRUE the function generate plot
illustrated by Fig. 3.

Figure 3: AUC of $BDI_{1}+BDI_{14}+BDI_{7}+BDI_{9}+BDI_{10}+BDI_{15},$ for DBI
scale

We assume that by selecting the “best” attribute in a loop, we are able to
reduce the number of attributes for the best preventiveness. In our case,
having the largest AUC is the “best” criterion. Adding the next “best”
attribute to the selected attribute from the subset of the remaining
attributes until AUC of all selected attributes decreases is the main idea of
our heuristic. So far, each and every rating scale has been reduced.

###  5.3 Second illustrative example: using the entire RSR procedure

Le us consider the data set Hepatitis analyzed in [9] and located at
http://archive.ics.uci.edu/ml/. It has 20 attributes and 312 examples used in
[1]. The goal is to illustrate how our entire RSR procedure may be used. To
reduce Hepatitis data set, we use the following attributes hepato:

    > names(att[,-d3])
     "time"     "status"   "trt"      "age"      "sex"
     "ascites"  "spiders"
     "edema"    "bili"     "chol"     "albumin"  "copper"
     "alk.phos" "ast"
      "trig"     "platelet" "protime"  "stage"

The following steps are needed:

  * •

detect duplicates and gray example in data,

  * •

reduce rating scale,

  * •

check possible inclusion.

By executing this code:

    > diffExamples(att[,-d3])
    $total.examples
    [1] 276

    $dif.examples
    [1] 276

    $dup.examples
    [1] 0

we detect no duplicate in data set.

Subsequently, we analyze the data set to detect gray examples for (status,
sex, spiders, stage) attributes. Gray examples are located by the use of
data.frame and a function called gray.ex.

Working on a full data set is a time-consuming process since it requires all
pair comparisons to be analyzed. A short optimization procedure is used for
attributes in two categories. The code below shows how to list by gray
examples by comparing the subset of attributes (status, sex, spiders, stage)
using the function grayEcamplesN. The key issue is to properly modify the
data.

    > gray.ex <-c()
    > df1 <-unique(data.frame(hepato, status, sex, spiders, stage))

    > for (i in 1:nrow(df1)){
    > ex <-grayExamplesN(df1[,2:ncol(df1)], df1[,1], i)$examp
    > if (nrow(ex)>1){
    > gray.ex <-rbind(gray.ex, ex[1,])}}
    > colnames(gray.ex) <-names(df1)

    > gray.ex
        hepato status sex spiders stage
    1        1      0   1       1     4
    2        1      0   1       1     3
    3        0      0   2       0     4
    6        1      0   1       0     3
    7        0      0   1       0     3
    8        0      0   1       1     2
    9        0      0   1       1     4
    15       1      0   1       0     4
    22       1      0   2       0     2
    23       0      0   1       0     2
    29       1      0   1       0     2
    42       0      0   2       0     3
    48       1      0   1       1     2
    53       0      0   1       0     4
    54       0      0   1       1     3
    57       1      0   2       0     3
    70       1      0   2       0     4
    80       0      0   2       0     2
    93       1      1   1       0     3
    98       0      1   1       0     3
    111      1      1   1       0     4
    139      0      1   1       0     2
    253      1      1   1       0     2
    256      0      1   1       0     4

The R print the list of gray examples. It means that every example from list
corresponds to (not listed) examples having identical attributes, but
different decision.

To reduce this rating scale, we execute the functions: totalAuc and rsr.

    > totalAuc(att[,-d3], hepato, plotT=TRUE)
    > rsr(att[,-d3], hepato, plotRSR=TRUE)

After reduction, the scale contains two attributes: stage, bili. The plot in
Fig. 4 illustrates AUC of the reduced rating scale and the proper ROC curve.

Figure 4: Left panel: AUC of individual attributes of Hepatitis data set;
Right panel: AUC of stage+bili for hepato scale

Finally, we examine the scale for possible inclusion using deLong and boostrap
tests as in the function roc.test from the pROC.

    > ROC1 <-CheckAttr4Inclusion(att[,-d3], hepato,
    + method=c("delong"), alternative=c("two.side"))

    > ROC2 <-CheckAttr4Inclusion(att[,-d3], hepato,
    + method=c("bootstrap"), alternative=c("two.side"))

    > summ
              Z statistics    p-value
    DeLong        1.986207 0.04701039
    bootstrap     1.945672 0.05169413

The p-value = 0.04701 shows that according to deLong test the null hypothesis
“H0: true difference in AUC is equal to 0” should not be rejected in favor of
the alternative. Fig. 5 illustrates two tested ROC curves.

Figure 5: ROC curves for the original and reduced rating scales for the
Hepatitis data set

##  6 The potential application targets

Rating scales are by far more important contributors to science that we can
address them by this study. Most examinations for granting scientific degrees
are rating scales of various shapes and forms. Simplifying them (or reducing
in size) is needed since we are subjected to more and more examination for so
needed certifications.

In bioinformatics, reporting trade-off in sensitivity and specificity, by
using a Receiver Operating Characteristic (ROC) curve, is becoming a common
practice. ROC plot has the sensitivity on the $y$ axis, against the false
discovery rate (1- specificity) on the $x$ axis. ROC curve plot provides a
visual tool to determine the boundary limit (or the separation threshold) of a
subset (or a combination) of scale items for the potentially optimal
combination of sensitivity and specificity. The area under the curve (AUC) of
the ROC curve indicates the overall accuracy and the separation performance of
the rating scale. It can be readily used to compare different item subsets. As
a rule of thumb, the fewer the scale items used to maximize the AUC of the ROC
curve, the better.

World Health Organization estimates are included behind selected rating scales
for mental disorder. Rating scales are of considerable importance for
psychiatry where they are predominately used for screening patients for mental
disorders such as:

  * •

depression (see [7]) which affects 60 million people worldwide according to
[14],

  * •

bipolar affective disorder (60 million people),

  * •

dementia and cognitive impairment (47.5 million people)

  * •

schizophrenia (21 million people),

  * •

autism and autism spectrum disorder(e.g., [5])

  * •

mania and bipolar disorder,

  * •

addiction,

  * •

personality and personality disorders,

  * •

anxiety,

  * •

ADHD;

and many other disorders.

Usually, there are many scales for each mental disorder. The most important
for screening are global scales. Reducing these global rating scales makes
they more usable as indicated in [7]. World Health Organization Media Centre
reports that“depression and anxiety disorders cost the global economy US $1
trillion each year” and it is no longer a local problem.

##  7 Conclusions

The presented method has reduced the number of the rating scale items
(variables) to 28.57% from the original number of items (from 21 to 6). It
means that over 70% of collected data was unnecessary. It is not only an
essential budgetary saving, as the data collection is usually expensive and
may easily go into hundreds of thousands of dollars, but the excessive data
collection may contribute to the data collection error increase. The more data
are collected, the more errors may occur occur since a lack of concentration
and boredom are realistic factors.

By using the proposed AUC ROC reduction method, the predictability has
increased by approximately 0.5%. It may seem insignificant. However, for a
large population, it is of considerable importance. In fact, [14] states that:
“Taken together, mental, neurological and substance use disorders exact a high
toll, accounting for 13% of the total global burden.”

The proposed use of AUC for reducing the number of rating scale items, as a
criterion, is innovative and applicable to practically all rating scales.
System R code is posted on the Internet (RatingScaleReduction) for the general
use as a R package. Certainly, more validation cases would be helpful and the
assistance will be provided to anyone who wishes to try this method using
his/her data.

Future plans include using the presented method for financial data analyzed
but the real aim our collaborative effort is towards psychiatric scales. The
reduced scales can be further enhanced by the method described in [5] and [6].

## Acknowledgments

The first author has been supported in part by the Euro Research grant “Human
Capital”.

The authors would also like to express appreciation to Amanda Dion-Groleau,
B.A. Honors Psycholgie, (Laurentian University, Psychology), Tyler D. Jessup
(Laurentian University, Computer Science), and Grant O. Duncan, Team Lead,
Business Intelligence, Integration and Development, Health Sciences North,
Sudbury, Ontario, Canada) for the editorial improvements.

## References

  * [1] Bojan Cestnik, Igor Kononenko, Ivan Bratko, et al.  Assistant 86: A knowledge-elicitation tool for sophisticated users.  In EWSL, pages 31–45, 1987.
  * [2] Tom Fawcett.  Roc graphs: Notes and practical considerations for researchers.  Machine learning, 31(1):1–38, 2004.
  * [3] Dincer Goksuluk, Selcuk Korkmaz, Gokmen Zararsiz, and A Ergun Karaagaoglu.  easyroc: An interactive web-tool for roc curve analysis using r language environment.  The R Journal, 2016.
  * [4] Peter Hills and Michael Argyle.  The oxford happiness questionnaire: A compact scale for the measurement of psychological well-being.  Personality and individual differences, 33(7):1073–1082, 2002.
  * [5] Tamar Kakiashvili, Waldemar W. Koczkodaj, and Marc Woodbury-Smith.  Improving the medical scale predictability by the pairwise comparisons method: Evidence from a clinical data study.  Computer Methods and Programs in Biomedicine, 105(3):210–216, 2012\.
  * [6] Waldemar W Koczkodaj.  Statistically accurate evidence of improved error rate by pairwise comparisons.  Perceptual and Motor Skills, 82(1):43–48, 1996.
  * [7] W.W. Koczkodaj, T. Kakiashvili, A. Szymańska, J. Montero-Marin, R. Araya, , J. Garcia-Campayo, K. Rutkowski, and D. Strzałka.  How to reduce the number of rating scale items without predictability loss?  Scientometrics, 2, 2017.
  * [8] Thomas J Leeper.  Crowdsourced data preprocessing with r and amazon mechanical turk.  The R Journal, 8(1):276–288, 2016.
  * [9] M. Lichman.  UCI machine learning repository, 2013.
  * [10] Roger McIntyre, Sidney Kennedy, Michael Bagby, and David Bakish.  Assessing full remission.  Journal of psychiatry & neuroscience: JPN, 27(4):235, 2002.
  * [11] Xavier Robin, Natacha Turck, Alexandre Hainard, Natalia Tiberti, Frédérique Lisacek, Jean-Charles Sanchez, and Markus Müller.  proc: an open-source package for r and s+ to analyze and compare roc curves.  BMC Bioinformatics, 12:77, 2011.
  * [12] Tobias Sing, Oliver Sander, Niko Beerenwinkel, and Thomas Lengauer.  Rocr: visualizing classifier performance in r.  Bioinformatics, 21(20):3940–3941, 2005.
  * [13] W. Velden, M. & Clark.  Reduction of rating scale data by means of signal detection theory.  Psychophysics, 25(6):517–518, 1979.  doi:10.3758/BF03213831.
  * [14] WHO.  Mental disorders fact sheet.  “www.who.int/mediacentre/factsheets/fs396/en/”, April, 2016.
  * [15] Luke A Winslow, Scott Chamberlain, Alison P Appling, and Jordan S Read.  sbtools: A package connecting r to cloud-based data for collaborative online research.  The R Journal, 8(1):387–398, 2016.

Waldemar W. Koczkodaj
Computer Science
Ramsey Lk Rd.
Laurentian University
Sudbury, Ontario P3E 2C6
Canada
<EMAIL_ADDRESS>
http://cs.laurentian.ca/wkoczkodaj/info

Alicja Wolny–Dominiak
Department of Statistical and Mathematical Methods in Economics
University of Economics in Katowice
ul. 1 Maja 50
40–287 Katowice
Poland
<EMAIL_ADDRESS>
web.ue.katowice.pl/woali