Simulating a Positron Converter in Bmad

John Mastroberti, Indiana University, Bloomington, IN, 47405, USA

David Sagan and Jim Shanks, Cornell University, Ithaca, NY, 14850, USA

A model to describe the output particle distribution generated
by particles impinging on a planar target has been
developed. This model was developed to simulate positron
production in the Cornell CESR Linac but can be applied to targets using different types
of particles.

To model a specific converter target, the output particle distribution is first
simulated by tracking particles using the Geant4 toolkit
which models the fundamental physics of the conversion process. The Monte Carlo
distribution from Geant4 is fitted to a set of functions and the function
coefficients are saved for use in simulations. Using
the fit functions in a simulation is not only more portable but is also more than an order of magnitude faster than running Geant4.
This model has been successfully incorporated into the simulation toolkit.

The output position, angular orientation, and momentum distribution is modeled.
Preliminary support for modeling the polarization distribution is also

§ INTRODUCTION

A common method of producing positrons for use in an accelerator is through the
use of a positron converter<cit.>. This is
typically a slab of heavy metal, such as tungsten, located in a Linac which
is bombarded with electrons with energies of order $100$ MeV. The
electrons emit photons via bremsstrahlung which in turn decay to $e^+ e^-$
pairs via the reaction

\begin{align*}
e^- + Z \rightarrow e^- + Z + \gamma \rightarrow e^- + Z + e^- + e^+.
\end{align*}

After the converter, the electrons and any other particles produced can be filtered off with a dipole
magnet effectively “converting" a beam of incoming electrons into a beam
of outgoing positrons.

Both bremsstrahlung from electrons traveling through the field of a nucleus
and the subsequent $e^+ e^-$ pair production have been discussed at length in
the literature<cit.>.
Many of the
theoretical treatments of bremsstrahlung and $e^+ e^-$ pair production are
reviewed by Tsai<cit.>.
Analytic expressions for both the bremsstrahlung radiation
spectrum and secondary electron energy spectrum as functions of target
thickness have been derived<cit.>, and the problem has been treated
across a wide range of incoming electron energies<cit.>.  The
bremsstrahlung cross section has been tabulated for a wide range of
electron energies and target materials<cit.>.
Recently, computer codes have been developed to calculate the spectra and
angular distributions of bremsstrahlung<cit.>, although
these are designed to work at low energies ($< 3$ MeV).

Despite the extensive study of bremsstrahlung and pair production that exists
in the literature, there is no closed form description of the kinematics of
positrons produced by bremsstrahlung from electrons impinging on a target.
This problem has been treated numerically<cit.>, but there is no
existing machinery to incorporate these results into a full accelerator
simulation.  These prior efforts also do not address the production of
polarized positrons from a beam of polarized electrons, a topic which is of
current interest<cit.>.

In response, the authors have developed a converter model and this model
has been incorporated in the <cit.> accelerator toolkit. The model is designed to be flexible, accommodating any converter material and thickness as well as
supporting arbitrary incoming and outgoing
particle species. The model is also fast to use since the expensive physics calculations are performed
only once when the converter model coefficients are being calculated. This is especially
important when designing a machine with a converter as optimization simulations
are typically time intensive.

One challenge in constructing the model was how to handle the complexity coming from the fact that the outgoing
distribution has five dimensions (two spatial and three momentum) along with the incoming particle momentum, and orientation.
Additionally, the target thickness and particle spin are potential degrees of freedom.

§ THE CONVERTER MODEL

The first step is to simulate the conversion process and
calculate conversion probability coefficients.
The coefficients can then be used to generate particles
as part of a simulation of the machine the converter is a part of.

The coefficient calculation has two parts. The first part
is to simulate the quantum electrodynamic<cit.>
interactions of the incoming particles with the converter
to produce bremsstrahlung photons and subsequent positrons.
The distribution of the outgoing particles calculated in this first step is then used to calculate the coefficients
for the expressions used to model the distribution.

§.§ Coordinate System

For concreteness, it will be assumed that the impinging beam is composed of
electrons and the outgoing beam composed of positrons. The model itself is
species agnostic.

Consider an electron incident on the upstream face of a positron converter of
thickness $T$, with momentum $p_-$ perpendicular to the face of the converter
as depicted in Figure <ref>. The $(\bfu, \bfv, \bfs)$ coordinate system
(Figs. <ref> and <ref>) is defined with $\bfs$ perpendicular
to the converter surface and the coordinate origin at the point where the electron
would emerge if it was undeflected through the converter.
Positrons produced in the converter
emerge from the downstream face with some radial displacement $r$ from the origin and at an
angle $\theta$ relative to the $\bfu$ axis as depicted in Figure <ref>.
Define the rotated ($\bfx$, $\bfy$, $\bfs$) coordinate system so that
$\bfx$ is in the direction of $\bfr$.

By symmetry, $\theta$ must be uniformly distributed from 0 to $2\pi$.
The kinematic properties of the produced positrons which must be
modeled are its radial displacement $r$ and outgoing momentum $\mathbf{p}_+$. The outgoing
momentum is described in terms of its magnitude, $p_+$, and the
slopes along the $\bfx$ and $\bfy$ directions:

\begin{equation}
\frac{dx}{ds} = \frac{p_x}{p_s}, \qquad
\frac{dy}{ds} = \frac{p_y}{p_s}
\end{equation}

Positron converter coordinate system (side view).

Positron converter coordinate system (view of the downstream face).

§.§ Probability Distributions

At the downstream surface, positrons produced in the converter are described by the distribution

\begin{align}
P \left( p_+, r, \dxds, \dyds ; p_-, t \right),
\end{align}

$P$ is the probability, per incoming electron, of producing an outgoing positron with momentum magnitude
$p_+$, radial displacement $r$, and momentum orientation $(dx/ds, dy/ds)$. $P$ will have a dependence on the converter thickness
$t$ and incoming momentum $p_-$ (non-normal incidence is discussed below).
The converter thickness dependence is important if the
converter thickness is allowed to vary in a simulation
where the positron yield is to be maximized. To model the thickness and electron momentum dependence, the distribution $P$
is calculated at a number of user defined thicknesses $t$ and
electron momenta $p_-$. Interpolation is then used when generating particles with the model.

For a given $t$ and $p_-$, the integral of $P$ over the space $(p_+, r, dx/ds, dy/ds)$ will be the number of positrons $n_+$
produced per electron

\begin{align}
\int P \left( p_+, r, \dxds, \dyds \right)
\, dp_+ \, dr \, d \! \left( \dxds \right) d \! \left( \dyds \right) & = n_+
\end{align}

To simplify the computations, $P$ is decomposed into two

\begin{align}
P \left( p_+, r, \dxds, \dyds \right) & = P_1 \!
\left( p_+, r \right) \, P_2 \left( \dxds, \dyds ; p_+, r \right)
\end{align}

where $P_1$ is the probability, per incident electron, of producing a positron with momentum magnitude
$p_+$ and radial displacement $r$, and $P_2$ is the probability, for a given $p_+$ and $r$, of producing
a positron with momentum orientation $(dx/ds, dy/ds)$. $P_1$ is normalized to $n_+$ and $P_2$ is
normalized to one.

The $P_1$ probability distribution is characterized by a two dimensional lookup table that gives $P_1$
at specific values of $p_+$ and $r$. $P_2$ is approximated using a heuristically derived skewed Lorentzian distribution

\begin{align}
P_2 \left( \dxds, \dyds; p_+, r \right) & =
A \frac{1 + \beta \dxds}{1 + \alpha_x^2 \left( \dxds - c_x \right)^2 + \alpha_y^2 \left( \dyds \right)^2}
\label{eq:cauchy}
\end{align}

where, as explained below, $\beta$, $\alpha_x$, $\alpha_y$ and $c_x$ will be characterized as functions of $p_+$ and $r$,
and $A$ is calculated from the normalization of $P_2$.

To calculate coefficients, a program to simulate the passage of particles through matter is needed. This
program was developed using the Geant4<cit.> particle physics library. A large number of electrons incident upon the
converter are simulated and the produced positron distribution is recorded.
From this, a fitting program is used to construct a two dimensional binned table of output probability $P_1$ as a function of
$p_+$ and $r$. The user may specify values for $p_+$ and $r$ to bin at or specify
the desired number of bins in both dimensions to use. If the latter option is made, the fitting program will space the bins
to give roughly equal number of particles in each bin. An example
is shown in Figure <ref>. The probability of having a positron
outside of the range of the table is taken to be zero. Inside the range of the table,
the value of $P_1(p_+, r)$ is found by linear interpolation of the table data.

For the $P_2$ modeling, for each of the $(p_+, r)$ bin points for $P_1$, a least squares fit is made using eq:cauchy and
the distribution of $dx/ds$ and $dy/ds$ for particles in the bin.
Since the distribution in Equation <ref> is not normalizable over the entire
$(dx/ds, dy/ds)$ plane, as part of the fit, the fitting program calculates values for $dx/ds_\Min$, $dx/ds_\Max$,
and $|dy/ds|_\Max$, outside of which the value of $P_2$ is taken to be zero.
These cuts are made so that $P_2$ covers 95% of the positrons generated by Geant4.
The values of $c_x$, $\alpha_x$, $\alpha_y$, $\beta$, $dx/ds_\Min$, $dx/ds_\Max$,
and $|dy/ds|_\Max$ are obtained by a fit of the Geant4 data in each $(p_+, r)$ bin with
the value of $A$ fixed by the unit normalization of $P_2$.

Once tables for the parameters $c_x$, $\alpha_x$, etc. are calculated at the bin points, the tables are used for each of these parameters to fit the parameters as a function of $p_+$ and $r$. The fit for a given parameter is has two pieces. Above a user definable threshold in $p_+$, a two dimensional fit in $(p_+, r)$ is made of the form:

\begin{equation}
f(p_+) g(r) \exp(k_1 p_+ + k_2 r) + C
\end{equation}

where $f(p_+)$ and $g(r)$ are third-order polynomials. Below the threshold, for each $p_+$ in the $P_1$ table, a 1D fourth-order polynomial fit in $r$ is made.
The 1D fits are performed for small values of $p_+$ because equation 6 tends to fit the Geant4 data poorly when $p_+ \lesssim 10 \mbox{ MeV}$.
Since most produced positrons are produced in this region, it is desirable for the $P_2$ parameter fits to be more detailed when $p_+$ is small.

Examples of the $P_1$ and $P_2$ distributions obtained from the
Geant simulation are shown in Figures <ref> and <ref>
The fit given in equation <ref> models the data from Geant quite successfully, both when the parameters are obtained by direct fit to the data and when the parameters from the second order fits are used.
The average $\chi^2$ value for the direct fits is about 0.11, while the average $\chi^2$ for the second order fits is about 0.14.
The bins where the fit performs the worst have a $\chi^2$ of about 1 for the direct fits, while the maximum $\chi^2$ for the second order fits is about 2.2.

$P_1(p_+, r)$ for incoming electrons with $p_- c = 250$ MeV and a
tungsten target of thickness $T = 6.35$ mm.

$P_2 \left( \dxds, \dyds ; p_+, r \right)$ for incoming electrons
with $p_- c = 250$ MeV and a tungsten target of thickness $T = 6.35$ mm, and
outgoing positrons with $p_+ c = 5.15$ MeV and $r = 0.37$ mm.  The purple
points indicate data obtained directly from the Geant simulation, while the
green curve shows the fit to the data.

The Geant4 simulation and accompanying formalism assume that the incoming electrons
are normally incident on the target. To adjust for a non-normal impinging electron when
generating positrons with the converter model,
the coordinate system is rotated so that the $\bfs$ axis is aligned along the incoming electron axis.
The effective converter thickness used is adjusted to be $T \sec\phi$ where $\phi$ is the angle from the normal, as this is the
distance from the electron beam's entry point on the converter's upstream face
to it's exit point on the downstream face.
The distributions of positron momentum $p_+$, radial displacement $r$, and outgoing angle $\theta$ are shown in Figures <ref>-<ref> for various incoming angles $\phi$. Up to 5 degrees off-normal
there is very little deviation from the on-axis results. For a machine like the Cornell CESR Linac, where the impinging beam has an
angular spread well less than 1 degree, the simple treatment of off-normal is a good approximation.

Relative discrepancy of the distributions of positron momenta
for $\phi = 1$ and 5 degrees compared to the distribution for $\phi = 0$.

Relative discrepancy of the distributions of positron radial displacements $r$
for $\phi = 1$ and 5 degrees compared to the distribution for $\phi = 0$.

Relative discrepancy of the distributions of positron angle $\theta$
for $\phi = 1$ and 5 degrees compared to the distribution for $\phi = 0$.

§ SPIN TRACKING

Polarization transfer from the incoming electrons to the outgoing positrons has
also been modeled.  Any incoming polarization $\mathbf{S}_-$ may be specified,
and histograms describing $S_x$, $S_y$, and $S_z$ of the produced positrons as
functions of $p_+$ and $r$ are produced. In the Geant4 simulations done for this paper,
it is observed that only
the longitudinal polarization of the incoming electrons is ever transferred to
the produced positrons; the produced positrons always have $S_x$ and $S_y$
essentially zero regardless of the incoming electron polarization.

Recently, the PEPPo collaboration<cit.> has published experimental
results of polarized positron production from a polarized electron beam via
bremsstrahlung, to which the spin tracking results from this simulation can be
compared.  This simulation yields polarization transfer efficiencies
significantly higher than those observed experimentally for outgoing positrons
with $p_+ c$ less than 5 MeV.  In contrast, the simulation's results for higher
momentum positrons agrees closely with experiment. The discrepancy is likely
due to approximations made by the Geant4 library in computing polarization
transfer during the pair production step. While this disagreement shows that the
spin tracking components of this simulation should not be relied on for
accuracy at present, all of the infrastructure is in place to handle spin
tracking.  When more accurate spin tracking methods become available in Geant,
this simulation, and the converter element, will be ready to use their

§ THE BMAD CONVERTER ELEMENT

is a software toolkit for the simulation of high energy charged particles
and X-rays. To make use of the converter model,
a converter element has been added to which uses the model coefficients
as described above allowing
for such things as a start to end simulation of the Cornell CESR Linac.
The model coefficients are stored within a lattice input file which makes
converter simulation portable to any based program.

For each incoming electron, randomly pulls values of $p_+$, $r$,
$\dxds$, and $\dyds$ from the interpolated probability distributions.
Outgoing positrons are assigned a weighting factor proportional
to the likelihood of producing a positron from the given incoming electron.
This weighting factor is then used with any statistical analysis.

Agreement between direct output from Geant4 and emulated output from is quite good, with the distribution of particles from coming within a
few percent of the Geant distributions.  This is illustrated in Figure
<ref>. simulates the positron converter
approximately 20 times faster than the direct Geant4 physics simulations.

As the figure shows, the relative discrepancy between the distributions
of positrons produced by Geant and agree within a few percent under most circumstances.
The two scenarios where this is not the case are for the lowest energy bins, and for bins
with large $p_+$ and large $r$. The discrepancy in the first case is due to the fact that does not produce positrons with $p_+$ below the value of $p_+$ used as the center of the first bin.
This causes to produce less low-energy positrons than Geant. This discrepancy is acceptable,
as virtually all of these low-energy positrons would be lost within a short distance from the converter.
The discrepancy for large $p_+$ and $r$ bins is also acceptable since very few positrons are produced
in this region.

Comparison of the converter element with the Geant simulation.
The surface plot shows the difference between the $P_1$ distributions generated by and by Geant, with the difference scaled by the maximum value of the $P_1$ distribution from Geant.
The color plot shows the value of the $P_1$ distribution from Geant, indicating the region where obtaining accurate results from is most important.

§ CONCLUSION

The converter model presented in this paper, when applied to positron production,
provides simulation results that
are in close agreement to Geant4's direct simulation of bremsstrahlung
radiation and pair production.  A small sacrifice in accuracy is made in return
for a significant increase in simulation speed, making this method appropriate
for use in lattice simulation and optimization software such as .

One of the primary motivators for this work was the desire to fully model the
CESR linac lattice.  Work is currently underway to use the model
developed here. A first goal in this endeavor is to assess the agreement
between the model of the linac as a whole the against the linac's experimentally
observed beam.  Once good agreement is established, can be
used to optimize the lattice design, including quadrupole and solenoid tuning.
Other future applications could include the optimization of element positioning
on the linac beam line.

§ ACKNOWLEDGMENTS

Many thanks to Vardan Khachatryan for helping with running of Geant.
This work is supported by National Science Foundation award number

Missing 'biblatex' package
The bibliography requires the 'biblatex' package.

booktitleSLAC LC02
titleCESR's Positron Source

abstractConversion of an electron beam to a positron beam is obtained at the end of the first four accelerating guides of the Frascati linac, using an electron current of 200 mA peak, at 85 MeV, having a pulse width of 4 μs and a repetition rate variable between 3 and 250 pps. Two different fixed targets have been designed: a high average power target used to produce a high duty cycle (4 μs, 250 pps) positron beam for nuclear physics experiments, and a high positron production efficiency, low average power target, used for injection of positrons into the storage ring, Adone, when a beam repetition rate of only 3 pps is allowed, but a high positron yield is required, to reduce the ring filling time.
journaltitleNuclear Instruments and Methods
titlePositron converter for the Frascati linear accelerator

American Physical Society

journaltitlePhys. Rev.
titleThick-Target Bremsstrahlung and Target Considerations for Secondary-Particle Production by Electrons

American Physical Society

journaltitlePhys. Rev. A
titleExact Screened Calculations of Atomic-Field Bremsstrahlung

journaltitleNuclear Instruments and Methods in Physics Research Section B: Beam Interactions with Materials and Atoms
titleBremsstrahlung spectra from electron interactions with screened atomic nuclei and orbital electrons

journaltitleAtomic Data and Nuclear Data Tables
titleBremsstrahlung energy spectra from electrons with kinetic energy 1 keV–10 GeV incident on screened nuclei and orbital electrons of neutral atoms with Z = 1–100

journaltitleComputer Physics Communications
titleBREMS: A program for calculating spectra and angular distributions of bremsstrahlung at electron energies less than 3 MeV
Electron scattering,Atomic-field bremsstrahlung,Bremsstrahlung spectrum,Bremsstrahlung angular distribution

journaltitleAtomic Data and Nuclear Data Tables
titleShape functions and singly differential cross sections of bremsstrahlung at electron energies from 10 eV to 3 MeV for Z = 1–100
Atomic-field bremsstrahlung,Bremsstrahlung spectrum,Bremsstrahlung angular distribution,Bremsstrahlung shape function

American Physical Society

journaltitleRev. Mod. Phys.
titlePair production and bremsstrahlung of charged leptons

Cornell University

titleIncreasing the positron capture efficiency of the CESR linac injector
Physics: Elementary Particles and High Energy

American Physical Society

journaltitlePhys. Rev. Lett.
titleProduction of Highly Polarized Positrons Using Polarized Electrons at MeV Energies

journaltitleNuc. Instrum. and Methods in Phys Research A
note<http://www.lepp.cornell.edu/ dcs/bmad>
titleBmad: A relativistic charged particle simulation library

number8, sec. 34.4
titleReview of Particle Physics

abstractGeant4 is a toolkit for simulating the passage of particles through matter. It includes a complete range of functionality including tracking, geometry, physics models and hits. The physics processes offered cover a comprehensive range, including electromagnetic, hadronic and optical processes, a large set of long-lived particles, materials and elements, over a wide energy range starting, in some cases, from 250eV and extending in others to the TeV energy range. It has been designed and constructed to expose the physics models utilised, to handle complex geometries, and to enable its easy adaptation for optimal use in different sets of applications. The toolkit is the result of a worldwide collaboration of physicists and software engineers. It has been created exploiting software engineering and object-oriented technology and implemented in the C++ programming language. It has been used in applications in particle physics, nuclear physics, accelerator design, space engineering and medical physics.
journaltitleNuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment
titleGeant4—a simulation toolkit
Simulation,Particle interactions,Geometrical modelling,Software engineering,Object-oriented technology,Distributed software development


# Immersive Visual Analysis of Cello Bow Movements

Frank Heyen<EMAIL_ADDRESS>0000-0002-5090-0133 ,  Yannik
Kohler<EMAIL_ADDRESS>VISUS, University of
StuttgartStuttgartGermany ,  Sebastian Triebener<EMAIL_ADDRESS>Hochschule für Musik und Darstellende Kunst StuttgartStuttgartGermany ,
Sebastian Rigling<EMAIL_ADDRESS>0000-0002-2732-6276 and  Michael Sedlmair<EMAIL_ADDRESS>stuttgart.de 0000-0001-7048-9292 VISUS, University of
StuttgartStuttgartGermany

(2022)

###### Abstract.

We propose a 3D immersive visualization environment for analyzing the right
hand movements of a cello player. To achieve this, we track the position and
orientation of the cello bow and record audio. As movements mostly occur in a
shallow volume and the motion is therefore mostly two-dimensional, we use the
third dimension to encode time. Our concept further explores various mappings
from motion and audio data to spatial and other visual attributes. We work in
close cooperation with a cellist and plan to evaluate our prototype through a
user study with a group of cellists in the near future.

Immersive analytics, virtual reality, cello, tracking, visualization, motion
guidance

††copyright: ccbysa††journalyear: 2022††conference: CHI Conference on Human
Factors in Computing Systems; April 29–May 5, 2022; New Orleans, LA,
USA††booktitle: CHI Conference on Human Factors in Computing Systems (CHI
’22), April 29–May 5, 2022, New Orleans, LA, USA††ccs: Human-centered
computing Visualization††ccs: Human-centered computing Human computer
interaction (HCI)††ccs: Human-centered computing Visual analytics††ccs:
Applied computing Interactive learning environments

(a) A one-to-one mapping that shows the motion exactly as it was.

(b) Mapping time to a spatial dimension allows seeing longer recordings less
cluttered.

(c) Simultaneous display of the current motion and a recording.

Figure 1.  Comparison of visual encodings for bow movements. Here, color and
thickness both indicate movement speed. We plan to also visualize played
notes, loudness, and differences between recordings or live input and
recording.

Comparison of three examples for visual encodings. In all three images, a long
gray bar represents the current position and orientation of the cello bow.
From left to right: a) A 3D path that shows the last few seconds of hand/bow
movement. The color and thickness varies and encodes the movement speed, thin
and red for fast, thick and blue for slow. b) The same path as in a), but
stretched away from the observer and therefore more spaced out. c) The same
path as in b), with another path next to it, that looks roughly similar.

##  1\. Introduction

Sheet music for cello pieces usually lacks instructions for how notes should
be played. Deciding on the exact bow movements is therefore up to the
musicians themselves. Since there might be very different ways to play the
same piece, the analysis and comparison of different musicians’ playing can
reveal their strategies and differences and might help musicians try other
movements and thereby improve or vary their playing.

Instrument playing is a physical activity in which positions and scales of
movements have a meaning connected to our physical world. We therefore decided
to use concepts from immersive and situated analytics for our design (Marriott
et al., 2018). When looking at the bow movements of a cellist, we note that
they are mostly restricted to a two-dimensional plane. This fact can be
leveraged for visualization, as it allows us to map time to space, following
an important visualization principle (Ware et al., 2006). We thus use the
third dimension by letting a line indicating the path of motion “fly away”
from the observer.

Our approach works for both live and post hoc analysis and can show multiple
recordings of the same or different players. Besides analysis, our display
could be used for artistic performances, where the audience sees our 3D
representation of the musicians’ movements over time. Our current design uses
virtual reality for ease of development, but future work will explore
extensions to augmented reality, which would allow musicians and audience to
see both instruments and visualization in the same space.

The authors of this paper have different backgrounds in the areas of
visualization, immersive analytics, and general human-computer interaction.
During our design, we collaborate closely with a cellist and follow guidelines
from design study methodology (Sedlmair et al., 2012), such as committing to
regular meetings with this domain expert.

In summary, we contribute a new interface for immersive analysis of cellists’
bow movements to allow studying and comparing movement patterns of one or more
musicians. We explore possible data features we can measure or compute and
then map to a range of visual markers and encodings. Our current prototype
supports data collection and basic visualization. Future work includes further
design exploration and an evaluation with a group of cellists.

##  2\. Related Work

A recent survey on music visualization (Khulusi et al., 2020) shows that
despite a range of different applications such as visualization of instrument
hardware, audio (Foote, 1999), sheet music (Miller et al., 2019b; Miller et
al., 2019a; Fürst et al., 2020), artist networks, and listening histories
(Baur et al., 2010), there has still been little work in this domain. One task
that has not been addressed much in this area is the analysis and visual
comparison (Gleicher et al., 2011) of motion and exercise recordings. As data
collected from playing music is personal to the musician, we consider our
approach to be personal visualization (Huang et al., 2015).

Closest to our work are motion guidance visualizations. These systems show
animated 3D paths that users need to follow along when learning motions such
as dancing (Yu et al., 2020) or physiotherapy exercises (Tang et al., 2015).
Our primary tasks are more analytical, and as such we are interested in
visualizations that can still provide motion guidance but at the same time
allow for a better overview over entire motion sequences. To that end, we
found inspiration in Ware et al.’s work (Ware et al., 2006), in which they —
instead of using animation — map the 3D movements and behavior of whales onto
paths that are augmented with textures and glyphs.

Work in the field of human-computer interaction addressed tasks such as
assessing musicians’ playing and providing respective feedback. Augmented
reality, for instance, has been used to display information on drum kits
(Yamabe et al., 2011) and guitars (Löchtefeld et al., 2011). _Let’s frets!_
(Marky et al., 2021) uses LEDs and touch sensors on the fretboard to display
and check exercises. Other approaches (Karolus et al., 2018; Yuksel et al.,
2016) use sensor measurements in order to adapt the tempo or difficulty of
exercises, or to allow users to control effects (Karolus et al., 2020).
Colored sheet music (Asahi et al., 2018; Hori et al., 2019; Smith and
Johnston, 2008) can highlight mistakes and compare student’s and teacher’s
recordings. _Strummer_ (Ariga et al., 2017) teaches users to play guitar
chords and uses audio to check for correctness. _Soloist_ (Wang et al., 2021)
creates overviews of instructional videos for guitar players and extracts the
played notes from audio through _CREPE_ (Kim et al., 2018), so users can
visually compare their playing to the instructor’s. Our work focuses on
different tasks, data, and instruments and extends this set of previous work.

Figure 2.  We currently track position and orientation of the bow through a VR
controller fixed to it.

A cellist wears a head-mounted display and plays the cello using a bow, which
has a VR controller fixed to it.

##  3\. Design & Implementation

Inspired by Munzner’s nested model (Munzner, 2009), we structure this section
into use cases and tasks, tracking and data, and visualization. Our current
design uses virtual reality (VR) to allow for more immersion (Marriott et al.,
2018) and better embedding (Willett et al., 2017) than desktop interfaces, and
a less distracting environment (Satkowski and Dachselt, 2021) and easier
development than an augmented reality (AR) interface. We display sheet music
virtually and our domain expert confirmed that seeing the instrument is not
important. An extension to AR is planned for future work.

###  3.1. Use Cases & Tasks

There are multiple educational use cases that we plan to support: In music
schools, our approach could be used to record a teacher’s motion that a
student then plays back while trying to follow, to learn the recommended
movements. Recordings could also be analyzed and compared retrospectively, for
instance to find incorrect or very different sections. Students or self-taught
learners who own VR hardware could also do this at home once recordings made
by others are available. Without access to others’ recordings, students could
still analyze their own playing and detect possible difficulties.

There are also uses cases apart from education: During a live performance, the
musicians’ movements could be visualized for the audience, for example through
individual head-mounted displays or a 3D cinema canvas and glasses. Once
recorded, the movements themselves could be displayed as an art installation,
played back together with audio, and interacted with by visitors.

###  3.2. Tracking & Data

Our implementation uses the Unity game engine for rendering 3D real-time
visualization and an Oculus Rift S headset with controllers for output and
interaction. For our current prototype, we taped one of the usually hand-held
controllers to the bow (Figure 2), but we plan to switch to OptiTrack for our
user study to reduce the bow’s weight. We track the controller’s 3D position
and orientation 80 times per second (one sample each 12.5 ms) and store this
information together with a time stamp. Additionally, we record audio and
extract played notes with CREPE (Kim et al., 2018).

Table 1 summarizes the data features that we collect or compute for each time
frame. For note and loudness, we can also compute the difference between
actual and expected and visualize this value. In case we have multiple paths,
we can compare them through differences at a certain point in time.

Table 1. Data summary Feature | Source | Example visual encodings
---|---|---
time stamp | time | 1D space, animation, label
3D position | motion | 1D, 2D, or 3D space
3D orientation | motion | texture, material, shape
speed of movement | motion | color, thickness, texture
speed of rotation | motion | color, thickness, texture
pitch / played note | audio | color, label
loudness | audio | color, thickness, texture
expected note | sheet music | color, label
expected loudness | sheet music | color, thickness, texture

###  3.3. Visualization

Having these data features, we can now choose from a range of visual
attributes to map them to. Our current design supports mapping the motion over
time to 3D, such that the motion path is “flying away” from the user in a
direction of choice, which is usually roughly orthogonal to the playing motion
(1(b)). The speed of this 3D translation can be adapted. When set to zero, the
motion is played back exactly as it was recorded (1(a)).

To show other data features together with time and space, we change the path’s
appearance at each time step. For example, we can vary the color and thickness
based on the movement’s speed or direction, or the audio’s pitch or loudness.
Further ideas for encodings include the path’s shape (spiky, smooth, zigzag,
…) or texture and material (stripes, roughness, reflectance, …), similar to
related work (Ware et al., 2006).

For comparison between multiple recordings or a recording and the current hand
motion, we can simply juxtapose or superimpose them (1(c)). Another
interesting option would be to explicitly encode the differences (Gleicher et
al., 2011), which is non-trivial for time-varying motion data and is left for
future work at the moment. One approach would be a mapping from the Euclidean
distance between the expected and actual position to a color. We could also
apply encodings from previous motion guidance research (Yu et al., 2020), such
as “rubber bands” connecting the actual position to the recommended one. A
mapping to 2D paths in combination with multiples views could show the motion
from different perspectives and make comparison easier than the 3D view that
suffers from occlusion and perspective distortion, especially when displaying
more than two paths. Such comparative visualizations would allow, for example,
to study different playstyles, such as playing a piece in a simplified versus
the original way. Showing these paths side by side should clearly reveal
differences.

Besides our visualization, we show a virtual representation of the bow for
reference and plan to add a virtual cello. Sheet music can be displayed
virtually as well, for use cases where the head-mounted display is worn during
playing.

##  4\. Conclusion

We propose a concept for visual immersive analysis of cello players’ bow
movements and explore different design aspects such as data collection,
feature extraction, and visual encodings. During our design study process
(Sedlmair et al., 2012), we collaborate with a cellist as domain expert. Our
current prototype already supports tracking and visualizing movement and speed
in a 3D path’s color and thickness.

Future work includes further exploration of the visual design space and an
evaluation of effectiveness and usability in different use cases, which we
plan to conduct as a user study with a group of cellists. We further plan to
extend our approach to augmented reality, so players and audience are able to
see each other and the instruments.

###### Acknowledgements.

Funded by Deutsche Forschungsgemeinschaft (DFG, German Research Foundation)
under Germany’s Excellence Strategy - EXC 2075 - 390740016, and by Cyber
Valley (InstruData project).

## References

  * (1)
  * Ariga et al. (2017) Shunya Ariga, Masataka Goto, and Koji Yatani. 2017.  Strummer: An Interactive Guitar Chord Practice System. In _International Conf. Multimedia and Expo (ICME)_. IEEE, 1057–1062.  https://doi.org/10.1109/ICME.2017.8019338
  * Asahi et al. (2018) Shota Asahi, Satoshi Tamura, Yuko Sugiyama, and Satoru Hayamizu. 2018.  Toward a High Performance Piano Practice Support System for Beginners. In _Asia-Pacific Signal and Information Processing Association Annual Summit and Conf. (APSIPA ASC)_. 73–79.  https://doi.org/10.23919/APSIPA.2018.8659463
  * Baur et al. (2010) Dominikus Baur, Frederik Seiffert, Michael Sedlmair, and Sebastian Boring. 2010.  The Streams of Our Lives: Visualizing Listening Histories in Context.  _IEEE Trans. Visualization and Computer Graphics (TVCG)_ 16, 6 (2010), 1119–1128.  https://doi.org/10.1109/TVCG.2010.206
  * Foote (1999) Jonathan Foote. 1999\.  Visualizing Music and Audio Using Self-Similarity. In _Proc. Seventh ACM International Conf. Multimedia (MULTIMEDIA)_. ACM, 77–80.  https://doi.org/10.1145/319463.319472
  * Fürst et al. (2020) Daniel Fürst, Matthias Miller, Daniel A. Keim, Alexandra Bonnici, Hanna Schäfer, and Mennatallah El-Assady. 2020.  Augmenting Sheet Music with Rhythmic Fingerprints. In _5th Workshop Visualization for the Digital Humanities (VIS4DH)_. IEEE, 14–23.  https://doi.org/10.1109/VIS4DH51463.2020.00007
  * Gleicher et al. (2011) Michael Gleicher, Danielle Albers, Rick Walker, Ilir Jusufi, Charles D. Hansen, and Jonathan C. Roberts. 2011.  Visual Comparison for Information Visualization.  _Information Visualization_ 10, 4 (2011), 289–309.
  * Hori et al. (2019) Matsuto Hori, Christoph M. Wilk, and Shigeki Sagayama. 2019\.  Piano Practice Evaluation and Visualization by HMM for Arbitrary Jumps and Mistakes. In _53rd Annual Conf. Information Sciences and Systems (CISS)_. 1–5.  https://doi.org/10.1109/CISS.2019.8692813
  * Huang et al. (2015) Dandan Huang, Melanie Tory, Bon Adriel Aseniero, Lyn Bartram, Scott Bateman, Sheelagh Carpendale, Anthony Tang, and Robert Woodbury. 2015\.  Personal Visualization and Personal Visual Analytics.  _IEEE Trans. Visualization and Computer Graphics (TVCG)_ 21, 3 (2015), 420–433.  https://doi.org/10.1109/TVCG.2014.2359887
  * Karolus et al. (2020) Jakob Karolus, Annika Kilian, Thomas Kosch, Albrecht Schmidt, and Paweł W. Wozniak. 2020\.  Hit the Thumb Jack! Using Electromyography to Augment the Piano Keyboard. In _Proc. ACM Designing Interactive Systems Conf. (DIS)_. ACM, 429–440.  https://doi.org/10.1145/3357236.3395500
  * Karolus et al. (2018) Jakob Karolus, Hendrik Schuff, Thomas Kosch, Paweł W. Wozniak, and Albrecht Schmidt. 2018\.  EMGuitar: Assisting Guitar Playing with Electromyography. In _Proc. 2018 Designing Interactive Systems Conf. (DIS)_. ACM, 651–655.  https://doi.org/10.1145/3196709.3196803
  * Khulusi et al. (2020) Richard Khulusi, Jakob Kusnick, Christofer Meinecke, Christina Gillmann, Josef Focht, and Stefan Jänicke. 2020.  A Survey on Visualizations for Musical Data.  _Computer Graphics Forum (CGF)_ (2020).  https://doi.org/10.1111/cgf.13905 arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1111/cgf.13905
  * Kim et al. (2018) Jong W. Kim, Justin Salamon, Peter Li, and Juan P. Bello. 2018\.  CREPE: A Convolutional Representation for Pitch Estimation. In _International Conf. Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 161–165.  https://doi.org/10.1109/ICASSP.2018.8461329
  * Löchtefeld et al. (2011) Markus Löchtefeld, Sven Gehring, Ralf Jung, and Antonio Krüger. 2011.  GuitAR: Supporting Guitar Learning through Mobile Projection. In _CHI ’11 Extended Abstracts on Human Factors in Computing Systems (CHI EA)_. ACM, 1447–1452.  https://doi.org/10.1145/1979742.1979789
  * Marky et al. (2021) Karola Marky, Andreas Weiß, Andrii Matviienko, Florian Brandherm, Sebastian Wolf, Martin Schmitz, Florian Krell, Florian Müller, Max Mühlhäuser, and Thomas Kosch. 2021.  Let’s Frets! Assisting Guitar Students During Practice via Capacitive Sensing. In _Proc. 2021 CHI Conf. Human Factors in Computing Systems (CHI)_. ACM, Article 746, 12 pages.  https://doi.org/10.1145/3411764.3445595
  * Marriott et al. (2018) Kim Marriott, Falk Schreiber, Tim Dwyer, Karsten Klein, Nathalie Henry Riche, Takayuki Itoh, Wolfgang Stuerzlinger, and Bruce H. Thomas. 2018.  _Immersive Analytics_.  Springer LNCS 11190.
  * Miller et al. (2019a) Matthias Miller, Alexandra Bonnici, and Mennatallah El-Assady. 2019a.  Augmenting Music Sheets with Harmonic Fingerprints. In _Proc. ACM Symp. Document Engineering (DocEng)_. ACM, Article 17, 10 pages.  https://doi.org/10.1145/3342558.3345395
  * Miller et al. (2019b) Matthias Miller, Hanna Schäfer, Matthias Kraus, Marc Leman, Daniel A. Keim, and Mennatallah El-Assady. 2019b.  Framing Visual Musicology through Methodology Transfer.  abs/1908.10411 (2019).  arXiv:1908.10411 http://arxiv.org/abs/1908.10411
  * Munzner (2009) Tamara Munzner. 2009\.  A Nested Model for Visualization Design and Validation.  _IEEE Trans. Visualization and Computer Graphics (TVCG)_ 15, 6 (2009), 921–928.  https://doi.org/10.1109/TVCG.2009.111
  * Satkowski and Dachselt (2021) Marc Satkowski and Raimund Dachselt. 2021.  Investigating the Impact of Real-World Environments on the Perception of 2D Visualizations in Augmented Reality. In _Proc. 2021 Conf. on Human Factors in Computing Systems (CHI)_ _(CHI ’21)_. ACM, Article 522, 15 pages.  https://doi.org/10.1145/3411764.3445330
  * Sedlmair et al. (2012) Michael Sedlmair, Mariah Meyer, and Tamara Munzner. 2012\.  Design Study Methodology: Reflections from the Trenches and the Stacks.  _IEEE Trans. Visualization and Computer Graphics (TVCG)_ 18, 12 (2012), 2431–2440.  https://doi.org/10.1109/TVCG.2012.213
  * Smith and Johnston (2008) Graeme Smith and Andrew Johnston. 2008.  Interactive Software for Guitar Learning. In _Australasian Computer Music Conf. (ACMC)_. Australasian Computer Music Association.
  * Tang et al. (2015) Richard Tang, Xing-Dong Yang, Scott Bateman, Joaquim Jorge, and Anthony Tang. 2015.  Physio@Home: Exploring Visual Guidance and Feedback Techniques for Physiotherapy Exercises. In _Proc. ACM Conf. Human Factors in Computing Systems (CHI)_ _(CHI ’15)_. ACM, 4123–4132.  https://doi.org/10.1145/2702123.2702401
  * Wang et al. (2021) Bryan Wang, Meng Yu Yang, and Tovi Grossman. 2021.  Soloist: Generating Mixed-Initiative Tutorials from Existing Guitar Instructional Videos Through Audio Processing. In _Proc. 2021 CHI Conf. Human Factors in Computing Systems (CHI)_. ACM, Article 98, 14 pages.  https://doi.org/10.1145/3411764.3445162
  * Ware et al. (2006) C. Ware, R. Arsenault, M. Plumlee, and D. Wiley. 2006\.  Visualizing the Underwater Behavior of Humpback Whales.  _Computer Graphics and Applications (CG &A)_ 26, 4 (2006), 14–18.  https://doi.org/10.1109/MCG.2006.93
  * Willett et al. (2017) W. Willett, Y. Jansen, and P. Dragicevic. 2017.  Embedded Data Representations.  _IEEE TVCG_ 23, 1 (2017), 461–470.  https://doi.org/10.1109/TVCG.2016.2598608
  * Yamabe et al. (2011) Tetsuo Yamabe, Hiroshi Asuma, Sumire Kiyono, and Tatsuo Nakajima. 2011.  Feedback Design in Augmented Musical Instruments: A Case Study with an AR Drum Kit. In _17th International Conf. Embedded and Real-Time Computing Systems and Applications (RTCSA)_ , Vol. 2. IEEE, 126–129.  https://doi.org/10.1109/RTCSA.2011.27
  * Yu et al. (2020) Xingyao Yu, Katrin Angerbauer, Peter Mohr, Denis Kalkofen, and Michael Sedlmair. 2020.  Perspective Matters: Design Implications for Motion Guidance in Mixed Reality. In _International Symposium on Mixed and Augmented Reality (ISMAR)_. IEEE, 577–587.  https://doi.org/10.1109/ISMAR50242.2020.00085
  * Yuksel et al. (2016) Beste F. Yuksel, Kurt B. Oleson, Lane Harrison, Evan M. Peck, Daniel Afergan, Remco Chang, and Robert J.K. Jacob. 2016.  Learn Piano with BACh: An Adaptive Learning Interface That Adjusts Task Difficulty Based on Brain State. In _Proc. 2016 CHI Conf. Human Factors in Computing Systems (CHI)_. ACM, 5372–5384.  https://doi.org/10.1145/2858036.2858388