# Music Enhancement with Deep Filters:
A Technical Report for The ICASSP 2024 Cadenza Challenge

###### Abstract

In this challenge, we disentangle the deep filters from the original
DeepfilterNet and incorporate them into our Spec-UNet-based network to further
improve a hybrid Demucs (hdemucs) based remixing pipeline. The motivation
behind the use of the deep filter component lies at its potential in better
handling temporal fine structures. We demonstrate an incremental improvement
in both the Signal-to-Distortion Ratio (SDR) and the Hearing Aid Audio Quality
Index (HAAQI) metrics when comparing the performance of hdemucs against
different versions of our model.

Index Terms—  Hearing Aids, Music Enhancement

##  1 Introduction

The ICASSP 2024 Cadenza Challenge [1] focuses on improving music perception
for individuals with hearing aids. While general audio enhancement and speech
enhancement are well-researched topics and are advancing rapidly with the rise
of deep learning techniques [2, 3], enhancement techniques specifically
targeting people with hearing disabilities have received less attention.

As shown in Figure 1, in this challenge, the enhancement pipeline separates
the mixtures at the hearing aid’s microphones (mixtures-at-mic) into drums,
bass, other and vocal stems using pre-trained baseline systems such as hdemucs
[4], and outputs an enhanced remix given a set of gains
$\\{G_{\text{drums}},G_{\text{bass}},G_{\text{other}},G_{\text{vocal}}\\}$ and
the listener’s characteristic (NALR). Note that per challenge rules, our
design does not have access to the preprocessing stages before the mixture-at-
mic.

The remix is then compared against the ground-truth remix. This ground-truth
remix is generated by processing individual ground truth stems from the
MUSDB18 dataset [5] in the same manner as the predicted stems. The evaluation
metric used in this challenge is HAAQI [6]. Unlike conventional metrics such
as SDR, HAAQI focuses more on the envelope and temporal fine structure. It has
been shown to better predict perceptual quality from the perspective of
hearing aid listeners [6].

To further enhance this pipeline, we adopt Spec-UNet [7] as a fine-tuning
network. This allows us to leverage the solid separation performance of the
baseline hdemucs.

Additionally, rather than outputting a complex ratio mask (cRM) at the end of
the fine-tuning network, we borrow the concept of deep filters from [8, 9] as
they take temporal information into account by filtering the spectrograms with
a fixed-length window along the time axis. This approach turns out to better
align with HAAQI, whose computation stresses temporal fine structures.

Fig. 1: Cadenza Challenge Pipeline. The circled numbers 1, 2, and 3 indicate
the components of the input to our designed model. The model output then
replaces the role of component 3 and is compared against the ground truth
remix.

##  2 Theory

Fig. 2: Our Model Architecture. The dot represents vector inner product. ’pre-
NALR’ corresponds to the stereo remix in locations 1 and 2, while ’NALRed’
corresponds to the remix in location 3 in Figure 1.

###  2.1 Model Input, Architecture and Loss

As depicted in Figure 1 and Figure 2, our model takes a concatenation of the
STFT of mixture-at-mic, gain-adjusted remix, and NALRed remix as input, which
is of shape $\mathbb{C}^{B\times C_{m}\times T\times F}$, where $B$ represents
the batch size, $C_{m}=6$ the audio channels from pre-NALR and NALRed inputs,
$T$ the time axis, and $F$ the frequency axis.

The data then flows through the Spec-UNet. The final layer produces a data
shape of $\mathbb{C}^{B\times 32\times T\times F}$, at which point we have the
option to further reduce the channel count to 1 to generate a complex ratio
mask or to some value $N$ to utilize it as a deep filter of order $N$, as
explained in the following section. The final output shape is
$\mathbb{C}^{B\times 2\times T\times F}$.

Finally, we perform inverse STFT, and its output is trained against the
ground-truth remix using the mean absolute error loss.

###  2.2 Deep Filter

Formally, given a complex ratio mask (cRM) produced by the network, we perform
the following operation:

$\displaystyle O[t,f]=I[t,f]*M[t,f]\text{ for }\forall t,f$ (1)

Here, disregarding the stereo channel dimension, $O\in\mathbb{C}^{T\times F}$
represents the final output, $I\in\mathbb{C}^{T\times F}$ represents the
NALRed part of the input, and $M\in\mathbb{C}^{T\times F}$ is our mask.

For the deep filter, as explained in [9], we first unfold the input $I$ along
the time axis with a kernel length of $N$, obtaining
$I^{\prime}\in\mathbb{C}^{T\times F\times N}$. Then, we let the network
produce a tensor $M^{\prime}\in\mathbb{C}^{N\times T\times F}$, and our output
can be computed as follows:

$\displaystyle O[t,f]=I^{\prime}[t,f,:]\cdot M^{\prime}[:,t,f]\text{ for
}\forall t,f$ (2)

Here, the dot represents a vector dot product.

##  3 Experiments

###  3.1 Datasets and Experimental setup

We utilize the MUSDB18 dataset for the training and the evaluation of our
model. The training set contains 100 music tracks with four stems (vocals,
drums, bass and other). And the test set contains 50 music tracks. No data
augmentation is performed, and additional datasets are not utilized. For the
hyperparameters of Spec-UNet, we use the window size and the FFT size of 2048
with the hop size of 441. Since all of our inputs share a sampling rate of
44100 Hz, our output remains consistent with that. Regarding the deep filter,
we set the unfolding kernel length to $N=5$. Attempting higher kernel lengths
and experimenting with two-dimensional filters produced similar performance.

In terms of general training hyperparameters, we use a batch size of 36.
During training, we randomly extract 2-second clips from the pool of songs.
The learning rate is set to 1e-3. All methods are implemented in PyTorch and
trained on an NVIDIA A6000 GPU, with a maximum epoch of 100. Our model weighs
approximately 138M. With mostly convolutional layers, a single pass of
training completes in less than a day.

###  3.2 Results and Discussion

We present the results in Table 1. In the second row, we use DeepFilterNet as
the enhancement network and retrain it without further adjusting its ERB
(equivalent rectangular bandwidth) front end, potentially contributing to its
suboptimal performance. In the last row, we only incorporate the deep filter
operations from the DeepFilterNet into our model, as illustrated in Figure 2.
As observed, both the SDR and HAAQI metrics exhibit incremental improvements
after applying our model and replacing cRM with DF.

| mean SDR (dB) | HAAQI
---|---|---
hdemucs (baseline) | 8.297 | 0.5697
hdemucs + DeepFilterNet [9] | 8.124 | –
hdemucs + Spec-UNet (cRM) |  8.315 $\pm$ 0.013 | –
hdemucs + Spec-UNet (DF) |  8.326 $\pm$ 0.009 | 0.5704
Table 1: Evaluation Results on the MUSDB18 Test Set. To ensure the statistical
significance of our results, we repeat the third and fourth experiments three
times each and report the average.

##  4 Conclusion

In conclusion, we have designed a system that builds upon hybrid Demucs to
further enhance its remixing performance. The results of SDR and HAAQI metrics
not only confirm the positive effect of our design on improving perception but
also indicate the challenges of generalizing beyond known listeners. We plan
to investigate this issue in future work.

## References

  * [1] Gerardo Roa Dabike, Michael A Akeroyd, Scott Bannister, Jon Barker, Trevor J Cox, Bruno Fazenda, Jennifer Firth, Simone Graetzer, Alinka Greasley, Rebecca R. Vos, and William M. Whitmer,  “Overview and Results of the ICASSP SP Cadenza Challenge: Music Demixing/Remixing for Hearing Aids,”  in 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2024.
  * [2] Jianwei Yu, Yi Luo, Hangting Chen, Rongzhi Gu, and Chao Weng,  “High fidelity speech enhancement with band-split rnn,”  Proc. Interspeech, 2023.
  * [3] Xiang Hao, Xiangdong Su, Radu Horaud, and Xiaofei Li,  “Fullsubnet: A full-band and sub-band fusion model for real-time single-channel speech enhancement,”  in 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2021, pp. 6633–6637.
  * [4] Alexandre Défossez,  “Hybrid spectrogram and waveform source separation,”  arXiv preprint arXiv:2111.03600, 2021.
  * [5] Zafar Rafii, Antoine Liutkus, Fabian-Robert Stöter, Stylianos Ioannis Mimilakis, and Rachel Bittner,  “Musdb18-hq - an uncompressed version of musdb18,” 2019.
  * [6] James M Kates and Kathryn H Arehart,  “The hearing-aid audio quality index (haaqi),”  IEEE/ACM transactions on audio, speech, and language processing, vol. 24, no. 2, pp. 354–365, 2015.
  * [7] Andreas Jansson, Eric J. Humphrey, Nicola Montecchio, Rachel M. Bittner, Aparna Kumar, and Tillman Weyde,  “Singing voice separation with deep u-net convolutional networks,”  in International Society for Music Information Retrieval Conference, 2017.
  * [8] Hendrik Schroter, Alberto N Escalante-B, Tobias Rosenkranz, and Andreas Maier,  “Deepfilternet: A low complexity speech enhancement framework for full-band audio based on deep filtering,”  in 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 7407–7411.
  * [9] Hendrik Schroter, Tobias Rosenkranz, Alberto-N. Escalante-B, and Andreas Maier,  “Deep multi-frame filtering for hearing aids,”  Proc. Interspeech, 2023.


# Categorical, Ratio, and Professorial Data:
The Case for Reciprocal Rank

###### Abstract

Search engine results pages are usually abstracted as binary relevance vectors
and hence are categorical data, meaning that only a limited set of operations
is permitted, most notably tabulation of occurrence frequencies, with
determination of medians and averages not possible. To compare retrieval
systems it is thus usual to make use of a categorical-to-numeric effectiveness
mapping. A previous paper has argued that any desired categorical-to-numeric
mapping may be used, provided only that there is an argued connection between
each category of SERP and the score that is assigned to that category by the
mapping. Further, once that plausible connection has been established, then
the mapped values can be treated as real-valued observations on a ratio scale,
allowing the computation of averages. This article is written in support of
that point of view, and to respond to ongoing claims that SERP scores may only
be averaged if very restrictive conditions are imposed on the effectiveness
mapping.

##  1 Introduction and Background

Researchers often work with datasets of categorical or ordinal observations,
on which only a restricted range of operations may be undertaken. In
particular, search engine result pages (SERPs), which are typically abstracted
as $k$-element binary vectors that tabulate the relevance or not of the
documents placed in positions $1$ through $k$ of the answer pages returned by
a search system, are categorical data. There is no total ordering of
$k$-element SERPs, and unless we get lucky and happen to have two SERPs that
have a relationship captured by a partial ordering, direct comparison of SERPs
is not possible. The primary operation that is permitted on categorical data
is tabulation of occurrence frequencies, to determine the mode (most
frequently occurring value), or to determine the fraction of the data
observations that consist of some particular value, where in the case of
$k$-element binary SERPs a “value” is one of the $2^{k}$ distinct bit-patterns
that can arise. Neither of those tabulations is helpful when seeking to
determine whether retrieval system $A$ is demonstrably better than retrieval
system $B$ when used for some specific type of search task.

Comparative evaluations of SERPs must thus be guided by pragmatic
requirements, and is usually carried out using a numeric _effectiveness
mapping_ from the categorical universe of possible $k$-element SERPs to the
real number line.111These mappings have been previously referred to as
effectiveness metrics, evaluation metrics, and effectiveness measures. To
avoid ambiguity with the metric spaces that play an important role in
mathematics, and which imply properties not satisfied by sets of SERP
effectiveness scores, we make use here of the phrase _effectiveness mapping_.
For example, the effectiveness mapping _reciprocal rank_ (RR) assigns the
score $1/d$ to all SERPs in which the first $1$-bit is in rank position $d$,
and assigns a score of zero to the SERP $[0]^{k}$ containing $k$ zeros.

A wide range of other effectiveness mappings have also been proposed over the
years, each of which makes different assumptions about how SERPs are valued
relative to each other, based on different models of user behavior. These
include precision (Prec), average precision (AP), normalized discounted
cumulative gain (NDCG), rank-biased precision (RBP), plus others that the
reader will be familiar with. Each of these is a numeric mapping that (in
“@$k$” evaluations) maps the categorical set of $2^{k}$ different $k$-item
binary vectors to a value on the real line, within the range (by convention)
from zero to one inclusive.

The tension between SERPs being categorical data and the desire for numeric
comparisons between SERPs then leads to a critical question:

> _Can any such categorical-to-numeric effectiveness mapping be used, provided
> it is argued (by the person planning to employ it) to be a plausible
> surrogate for SERP “benefit to user” in the context of the given search
> scenario and the assumed user demographic? And if not, what restrictions
> limit the choice of effectiveness mappings?_

That question has received substantial attention. In range of papers though
the last few years one group of authors has presented an opinion that such
mappings may only be used if the range of the mapping (that is, the set of
numeric values that emerges when the mapping is applied to every one of the
$2^{k}$ SERPs of $k$-items) generates a set of points that are equi-spaced on
the real line [Fuhr, 2017; Ferrante et al., 2019a, b, 2020, 2021]. Examples
detailing the implications of that belief appear in Section 2.

Sakai [2020] and the current author [Moffat, 2022] have argued the contrary –
that any categorical-to-numeric mapping may be used in SERP evaluation,
provided that it is motivated as reflecting the way in which users are
consuming those SERPs. We believe that mappings should be chosen so as to
plausibly capture some external assumption in regard to the measurement
scenario being pursued, but apart from that connection do not require any
intrinsic properties of their own. For example, one possible external
assumption might be that users place $d$ times more value on having the first
relevant document at rank $1$ than they do on having the first relevant
document at rank $d$; a relationship that then leads directly to RR.

Each other effectiveness mapping similarly corresponds to a model of user
behavior, meaning that the score associated with each SERP captures – or at
least, approximates – the extent to which it is of benefit to the user that
issued the query. From this point of view the user model can be thought of as
coming first, and then once the factors that govern the user model have been
settled, the desired categorical-to-numeric effectiveness mapping will emerge
[Moffat et al., 2022].

The debate between these two points of view has now percolated into the
guidelines provided to conference referees, creating new threads of
contention. For example, in November 2022 the SPC members for ECIR 2023 were
requested to vet their pools of papers against a set of guidelines that
included the (disputed) statement “ _RR is widely used but difficult to
interpret as interval scale, which would be a requirement for computing the
arithmetic averages_ ”, so that the PC Chairs could “ _instruct all authors of
every accepted paper to check and correct if it applies_ ”.222The email in
question is dated 25 November 2022 and is from EasyChair, over the names of
the three PC Chairs. The email refers to the guidance as coming from “ _our
‘official’ ECIR Methodology Chair_ ”, but no such role is listed at
https://ecir2023.org/organisers.html, and no author is listed in the document
either, meaning that the precise origin of the instruction is unknown. At time
of writing these “guidelines” remain available at
https://docs.google.com/document/d/1taETBOHpnWCAOps-rVDV-j_ecUJS0U7S.

Most recently, in December 2022, the first group of “RR should be avoided”
authors has again argued that measurement is sound only if the numeric mapping
involved employs a set of available target points that are uniformly spaced on
the real line [Ferrante et al., 2022].

In short, one group of authors in the IR community is strongly opposed to RR
(and several other commonly used effectiveness mappings; RR is simply an
exemplar that is used here in order to allow a focused discussion) as an
experimental tool, and has been seeking to enforce that view across the
community. In opposition, a second group of authors believes that there is no
basis for such a restrictive view, and that researchers are free to select
effectiveness mappings that suit their experimental context, provided only
that they are willing to argue the _why_ of their choice.

I’m firmly of that second opinion. My goal in writing this essay is to
reiterate my previous “you may use RR if you wish” position, and to provide
further information to colleagues who have been puzzled (or, in the case of
negative reviewing decisions, feel they have been adversely affected) by this
debate.

##  2 A Brief Quiz

The reader is invited to consider the following sequence of questions,
imagining perhaps that they are completing a “how data literate are you?” quiz
in the sealed center section of some research journal. In all cases the word
“average” refers to the arithmetic mean.

Question 1 What is the average of the following eight numbers?

$700,700,750,750,750,850,1000,1000$

(_a_) $812.5$; or (_b_) it is not appropriate to compute an average; or (_c_)
insufficient information has been provided in the question to allow the choice
between options “(_a_)” and “(_b_)” to be made.

Question 2 What is the average of the following eight monetary amounts?

$\$700,\$700,\$750,\$750,\$750,\$850,\$1000,\$1000$

(_a_) $$812.50$; or (_b_) it is not appropriate to compute an average; or
(_c_) insufficient information has been provided in the question to allow the
choice between options “(_a_)” and “(_b_)” to be made.

Question 3 What is the average of the eight amounts listed in Question 2 if it
is additionally known that they are the weekly salaries of eight employees?
(_a_) $$812.50$; or (_b_) it is not appropriate to compute an average; or
(_c_) insufficient information has been provided in the question to allow the
choice between options “(_a_)” and “(_b_)” to be made.

Question 4 What is the average of the eight weekly salaries if it is
additionally known that the eight employees are two junior professors, three
assistant professors, one associate professor, and two full professors?
(_a_) $$812.50$; or (_b_) it is not appropriate to compute an average; or
(_c_) insufficient information has been provided in the question to allow the
choice between options “(_a_)” and “(_b_)” to be made.

Question 5 What is the average of the eight professor’s salaries if it is
additionally known that all junior professors are paid $\$700$ per week, all
assistant professors are paid $\$750$ per week, all associate professors are
paid $\$850$ per week, and all full professors are paid $\$1000$ per week?
(_a_) $\$812.50$; or (_b_) it is not appropriate to compute an average; or
(_c_) insufficient information has been provided in the question to allow the
choice between options “(_a_)” and “(_b_)” to be made.

Question 6 What is the average of the eight professor’s salaries if it is
additionally known that there are no other professorial job titles, and no
other professorial salary levels?
(_a_) $\$812.50$; or (_b_) it is not appropriate to compute an average; or
(_c_) insufficient information has been provided in the question to allow the
choice between options “(_a_)” and “(_b_)” to be made.

I imagine that the majority of readers gave a clear “(_a_)” response to
Question 1, and then continued to respond “(_a_)” even as further information
was provided about what it was that was being averaged. Those six “(_a_)”s are
also the answers that I’m arguing for in this essay. Averaging numeric and
monetary amounts is a routine operation, and we would probably also feel
equally comfortable (for example) if were we to be asked to sum the same sets
of values.

That comfort arises because none of Questions 1 to 6 are asking about “average
professors”. The question sequence starts by asking about the average of a
given set of numbers without mentioning professors at all, and ends by asking
about the average salary of a group of professors, with the set of
_observations_ (the professors’ ranks) being mapped to a set of numeric
quantities (the salaries corresponding to those observations) in the form of
numbers that have “$” signs attached. If we were to be asked about the
“average professor”, we would shrug our shoulders, and quite properly respond
with “(_b_) it is not appropriate to compute an average”. We would recognize
that there is no basis for computing the average of “two junior professors,
three assistant professors, one associate professor, and two full professors”,
because that is a dataset of _ordinal_ observations.333However it _is_
permissible to enquire about the median of ordinal observations. For that same
set of eight observations, the median professor is an “assistant professor”;
and the “median professor’s salary” is the mapped salary value associated with
an assistant professor.

In opposition to that view of when averaging may be applied, and as part of a
direct response to my previous paper [Moffat, 2022] in which I introduced this
example of professorial salaries, Ferrante et al. [2022, page 12] write:

> “ _the examples by Moffat on professor salaries_ […] _are not interval
> scales, as a consequence means cannot be computed_ ”

Followup emails with Nicola Ferro444My thanks to Nicola for being willing to
engage in this conversation. in November 2023 have confirmed that it is his
belief that Question 6 should be answered “(_b_)”, because (in his words) “ _I
understand the numbers do not come from an interval scale_ ”. Nicola also
argues that Questions 1 to 5 must be answered “(_c_)”, because their
formulation doesn’t specify (or doesn’t specify in sufficient detail) what the
numbers represent.

To tease out the implications of Nicola’s assertions, suppose that ${\cal{U}}$
is the universe from which some given set of numeric observations is being
drawn. For example, in the case of the professors’ salaries,
${{\cal{U}}}_{\mbox{\scriptsize{prof}}}=\\{\$700,\$750,\$850,\$1000\\}$.
Nicola’s sequence of answers to Questions 1 to 6 can then be seen as being
equivalent to the belief that when $u=|{\cal{U}}|$ is finite and if a dataset
of observations drawn from ${\cal{U}}$ (with replacement) is to be averaged,
then ${\cal{U}}$ must have the form:

${{\cal{U}}}={\mbox{\emph{equi-interval}}}(m,c,u)=\\{m\cdot k+c\mid
k\in\mbox{\emph{range}}(u)\\}\,,$ (1)

in which $m,c\in{\cal{R}}$; in which
$\mbox{\emph{range}}(u)=\\{k\in{\cal{I}}\mid 0\leq k<u\\}$; and where
$\cal{R}$ and $\cal{I}$ are the sets of real values and integer values
respectively. Note the use of equality in Equation 1. Nicola’s assertion is
that not only must every value in ${\cal{U}}$ be expressible as a real-valued
linear transformation of a corresponding integer, but also that _every such
equi-interval mapped value over a dense range of integers $0\leq k<u$ must be
available in ${\cal{U}}$_. That is, Nicola believes that despite the
professors’ salaries all being multiples of $m=\$50$, a dataset of
observations over ${{\cal{U}}}_{\mbox{\scriptsize{prof}}}$ may _not_ be
averaged, because the equi-spaced salaries $\$800$, $\$900$, and $\$950$ are
not available; with
${{\cal{U}}}_{\mbox{\scriptsize{prof}}}\not=\mbox{\emph{equi-
interval}}(m,c,u)$ for every combination of $m$, $c$, and $u$.

In particular, in Question 5 there is still uncertainty as to what values
might be possible as salaries and the possibility that
${{\cal{U}}}_{\mbox{\scriptsize{prof}}}$ might yet satisfy Equation 1, which
is why Nicola’s answer to Question 5 is “(_c_)”; but that uncertainty is
dispelled by the additional information provided in Question 6, thus leading
to the switch to answer “(_b_)”. Pursuing the same argument one step further,
if the three extra options $\$800$, $\$900$, and $\$950$ are also “available”
as weekly salaries (for example, because a small number of year-long merit-
based bonus payments of either $\$50$ or $\$100$ per week are determined at
the start of each year) but happen to not be visible in this current set of
observations (because none of the current professors are being paid a bonus),
then the averaging in Question 6 _would_ (presumably) be permitted, because
the salary universe would now be equal to $\mbox{\emph{equi-
interval}}(\$50,\$750,9)$.

My main point in introducing the professors’ salaries was that similar
questions can be posed in connection with SERP effectiveness scores [Moffat,
2022]. Let’s continue the quiz.

Question 7 What is the average of the following eight numbers?

$0,\frac{1}{3},\frac{1}{3},\frac{1}{3},\frac{1}{2},\frac{1}{2},1,1$

(_a_) $0.5$; or (_b_) it is not appropriate to compute an average; or (_c_)
insufficient information has been provided in the question to allow the choice
between options “(_a_)” and “(_b_)” to be made.

Question 8 What is the average of the eight numbers in Question 7 if it is
additionally known that they are effectiveness scores associated with eight
SERPs of length $k=3$ via the use of a categorical-to-numeric effectiveness
mapping?
(_a_) $0.5$; or (_b_) it is not appropriate to compute an average; or (_c_)
insufficient information has been provided in the question to allow the choice
between options “(_a_)” and “(_b_)” to be made.

Question 9 What is the average of the eight mapped SERP scores in Question 8
if it is additionally known that there is one SERP with no relevant document
by depth $k=3$, three that have their first relevant document at depth three,
two that have their first relevant document at depth two, and two that have
their first relevant document at depth one?
(_a_) $0.5$; or (_b_) it is not appropriate to compute an average; or (_c_)
insufficient information has been provided in the question to allow the choice
between options “(_a_)” and “(_b_)” to be made.

Question 10 What is the average of the eight SERP scores in Question 9 if it
is additionally known that SERPs with no relevant document by depth $k=3$ are
always assigned a score of $0$ and that SERPs with the first relevant document
at depth $d$ are always assigned a scores of $1/d$; and thus that there are no
other SERP scores available when $k=3$?
(_a_) $0.5$; or (_b_) it is not appropriate to compute an average; or (_c_)
insufficient information has been provided in the question to allow the choice
between options “(_a_)” and “(_b_)” to be made.

In response to this second set of questions, I again argue that “(_a_)” is the
appropriate answer throughout. My basis for that statement remains that it is
the _numbers_ that are being averaged, and that just as Questions 1 to 6 were
not attempting to compute an average professor, _there is no suggestion at all
in Questions 7 to 10 that SERPs are being averaged_. Search engine result
pages are _categorical_ data, even when abstracted as $k$-vectors over
$\\{0,1\\}$, and hence may not be averaged, just as the professors’ ranks
(ordinal data) may not be averaged. Furthermore, it isn’t even possible to
compute the median SERP across a set of observations, since SERPs _do not
possess a total order_. As a simple example to support that claim, consider
the two $5$-element SERPs $S_{1}=[1,1,0,0,0]$ and $S_{2}=[0,0,1,1,1]$, in
which the document order is from left to right. There is no innate ordering
possible between these two SERPs (one has more early relevance, the other has
more total relevance) and different effectiveness mappings are free to assign
their scores in either order. In particular, RR prefers the first of those two
SERPs, with
${\mbox{{\small\sf{RR}}}}(S_{1})=1.0>0.3333={\mbox{{\small\sf{RR}}}}(S_{2})$;
whereas ${\mbox{{\small\sf{Prec}}}}@5$ prefers the second SERP, with
${\mbox{{\small\sf{Prec}}}}@5(S_{2})=0.6>0.4={\mbox{{\small\sf{Prec}}}}@5(S_{1})$.

If any of us were to be provided with the set of SERPs that was observed in an
IR experiment and asked to calculate either the average or the median of the
set, we would have no option but to shrug and quite properly answer “(_b_) it
is not appropriate”. It is the _mapped SERP scores_ that are being averaged in
Questions 8 to 10, not the SERPs themselves. Nor is there any suggestion at
all that the effectiveness mapping can or may be inverted once that average
score has been computed, to determine a SERP that corresponds to it. The
effectiveness mapping is many-to-one, and inversion is neither possible nor
sensible.

In opposition to my view of Questions 7 to 10, Nicola answers “(_c_)” to
Question 7, and then “(_b_)” to Questions 8 through 10. The explanation he
provides for this response is that from Question 8 onwards there is explicit
mention of a categorical-to-numeric mapping, implying (in his opinion) that
the average may not be computed. Nor does the universe of possible values
${{\cal{U}}}_{\mbox{\scriptsize{RR3}}}=\\{0,1/3,1/2,1\\}$ that is employed in
these questions fit the form required by Equation 1, providing (presumably)
another reason for Nicola to answer “(_b_)” to Question 10.

(a) Averaging eight professorial salaries.

(b) Averaging eight ${\mbox{{\small\sf{RR}}}}@3$ effectiveness scores.

(c) Averaging eight ${\mbox{{\small\sf{Prec}}}}@3$ effectiveness scores.

Figure 1: Averaging sets of values: (a) the four professorial salary levels,
and the distribution of the eight observations assumed in Questions 1 to 6;
(b) the four ${\mbox{{\small\sf{RR}}}}@3$ scores possible for SERPs of length
$k=3$, and the distribution of the eight observations assumed in Questions 7
to 10; and (c) the same eight SERPs, but now assessed using the effectiveness
mapping ${\mbox{{\small\sf{Prec}}}}@3$.

Figure 1 illustrates the situations embedded in the two sequences of
questions. Each part shows a _ruler_ , and a set of associated _tick-points_
with their corresponding _numeric labels_. As well, the mapped values
corresponding to the sets of eight data observations are shown. In Figure 1(a)
the salaries of the eight professors are depicted, along with their average.
The average salary of the eight professors has a clear and tangible physical
interpretation – it is the fulcrum point at which the data items become
symmetric in a weighted sense, to bring the “balance beam” into equilibrium.
That is, “averaging” has a compelling (and useful, if you like see-saws!)
analogy in the physical world, namely the identification of fulcrums for
balance beams. In this physical correspondence there is no requirement
whatsoever that the observations being averaged must be located at the ruler’s
equi-interval labeled tick-points, and the mapping being used can place the
items anywhere. Nor is there any requirement that every marked tick-point on
the ruler must be capable of having an item placed on it.

Figure 1(b) provides a similar picture in connection with Questions 7 to 10,
taking (for concreteness) the eight SERPs to be $[0,0,0]$, $[0,0,1]$,
$[0,0,1]$, $[0,0,1]$, $[0,1,0]$, $[0,1,1]$, $[1,0,0]$, and $[1,1,0]$.555A set
of SERPs can always be placed into lexicographic order. Lexicographic ordering
happens to agree with the ordering induced by RR, but, in general, mapped
score orderings are not preserved. As a simple example of that, note that
${\mbox{{\small\sf{Prec}}}}@3([0,1,1])\not\leq{\mbox{{\small\sf{Prec}}}}@3([1,0,0])$.
A balance point can always be identified, regardless of where on the balance
beam the data observations are placed, and can be identified even if some
tick-points are unavailable to host “weights”.

In Figure 1(a) the fact that the underlying observations are ordinal, and that
there are just four distinct observations possible, means that if a different
salary mapping is applied to this dataset the stacks might shift sideways to
new points on the number line, but cannot change their height, nor “swap over”
and shift past each other. On the other hand, in Figure 1(b) the underlying
observations are categorical, and there are $2^{3}=8$ different possibilities,
six of which appear in this dataset. A different $k=3$ effectiveness mapping
might thus give rise to as many as six different stacks for this set of
observations, and as many as eight stacks in general. Moreover, the stacks for
$[1,0,0]$ and $[0,1,1]$ might appear in either order along the number line,
whereas (making reasonable assumptions about the partial ordering on SERPs)
the other six stacks may “slide” and perhaps even “combine”, but may not “swap
over”. See Moffat and Mackenzie [2023] for discussion of the extent to which
this “freedom to swap” is available.

Figure 1(c) gives one such alternative mapping for the same set of SERPs, now
using ${\mbox{{\small\sf{Prec}}}}@3$. Because
$|{{\cal{U}}}_{\mbox{\scriptsize{Prec3}}}|=4$ too, there are still at most
four stacks, but they have different memberships, a consequence of the greater
freedom associated with categorical data. In this second configuration we have
${{\cal{U}}}_{\mbox{\scriptsize{Prec3}}}=\\{0,1/3,2/3,1\\}=\mbox{\emph{equi-
interval}}(1/3,0,4)$, a mapping that is compliant with Equation 1. However
Nicola hedges when asked about this specific situation, writing [the answer] “
_is (b) because you explicitly say it is a categorical scale; however, if I do
not use the information about the categorical scale, I would say (a) because …
the objects (runs) are equi-spaced_ ”. Needless to say, I believe that the
${\mbox{{\small\sf{Prec}}}}@3$ scores may be averaged in exactly the same way
that the ${\mbox{{\small\sf{RR}}}}@3$ scores may be. In both cases the average
continues to have a direct physical interpretation, and the fact that
${\mbox{{\small\sf{RR}}}}@3$ and ${\mbox{{\small\sf{Prec}}}}@3$ stack the
SERPs differently on the number line, assign different scores, and yield
different average scores is of no consequence, because _they employ different
effectiveness mappings, and are thus measuring different things_. And neither
of them is claimed to be “averaging SERPs”.

In terms of the extremes of the set of mapped values, $\min({\cal{U}})$ and
$\max({\cal{U}})$, most researchers would agree that a SERP with no relevance,
$[0]^{k}$, is a natural _bottom_ element to the SERP categories, and is thus
normally mapped to zero. Other SERPs might also be assigned a score of zero,
but $[0]^{k}$ _will_ have a score of zero. Similarly, a SERP that is all
relevant, $[1]^{k}$ is a natural _top_ element to the SERP categories, and is
thus normally mapped to the maximum score (typically $1.0$, or approaching
$1.0$ in the limit as $k$ increases). Other SERPs might also be assigned a
maximal score, but $[1]^{k}$ _will_ have a maximal score. That is, despite
SERPs not possessing a total order, there is a partial order, with _top_ and
_bottom_ elements that bookend the range of values generated by any plausible
categorical-to-numeric mapping that might be employed in IR evaluation.

To summarize the arguments presented in this section: SERPs are categorical
data, and hence the only directly permissible operation is tabulation of
frequencies, which is unhelpful in IR experimentation. We may not even ask
about the median of a set of SERPs. But each plausible model of the
interactions that take place as users consume SERPs gives rise to a
corresponding categorical-to-numeric effectiveness mapping. All such mappings
share a common natural zero point, and give rise to up to $2^{k}$ different
mapped scores in a many-to-one manner. The set of SERP scores derived from
each such mapping may then be legitimately averaged without any implication
that the SERPs themselves are being averaged, nor that the mean score may be
converted back to a SERP. And the mapping process is valid even if the numeric
values that the universe of $2^{k}$ different SERPs transform to are not
evenly-spaced on the number line.

Supporting that position is the clear fact that in the physical analogy
expressed via Figure 1 there is no requirement that the objects placed on the
balance beam be located solely at the equi-spaced labeled tick-points on the
ruler; and nor is there any constraint that renders the averaging process
improper if there are labeled tick-points at which none of the weights will
ever be placed. That is, there is a clear physical analogy that supports the
correctness of answer “(_a_)” to all ten questions in the data survey.

Of course, if the mapping changes and (in the context of Figure 1) the target
points of some or all of the red arrows shift, then the fulcrum point is
likely to also need to shift. But that is perfectly sensible – the average is
determined _after_ the mapping is applied to the set of observations, and
there is _no suggestion at all_ being made here that the average is (or even
should be) invariant to the categorical-to-numeric effectiveness mapping that
has been chosen. Such invariance is simply not possible in connection with
categorical (SERPs) or even ordinal (professorial ranks) data.

##  3 Interval Scales Versus Equi-Interval Scales

Presuming that the reader has absorbed the arguments that were presented in
Section 2, they may now be somewhat puzzled. If, as I have asserted, SERPs are
categorical data, but may be mapped to the real number line via some defended
choice of non-reversible categorical-to-numeric effectiveness mapping, and if
all plausible mappings on SERPs have a defined zero point, then what are
interval scales, and why do they need to be considered?

Let’s start with what an interval scale is. The best example of interval scale
data is the one I noted in my previous paper [Moffat, 2022], Unix timestamps.
The Unix operating system measures time as elapsed seconds, starting at
0:00:00am on 1 January 1970. The current time a few minutes ago was
$1694942015$, and sixty seconds prior it was $1694941955$. In an interval
scale measurement system the actual raw measurements may not be especially
informative (for example, the ten-digit numbers provided in the previous
sentence) unless the zero point is also of significance to the observer. But
_differences_ between observations provide information; furthermore, _ratios
of differences_ may be computed and also provide information. For example
(dropping six leading digits, to make the numbers manageable), $2015-1955=60$
and represents an interval (in seconds) that can be correctly stated as being
$1.5$ times longer than the interval between $2310$ and $2350$.

Unix times are normally thought of as being integers, but from a measurement
point of view there is no requirement that they be restricted to integral
values, and fractional values are also completely fine. Thus the interval from
$2380.5$ to $2396.4$ is $1.5$ times longer than the interval from $2320.5$ to
$2331.1$, because $(2396.4-2380.5)/(2331.1-2320.5)=1.5$. Moreover, if an
alternative reference time was chosen, for example, 2:22:22am on December 22,
2022, all of the raw measurements would change, but the differences, and
ratios, between them, would be invariant. And if the measurement units altered
from seconds to (say) tenths of seconds, both the raw measurements and their
differences would change, but the ratios between the differences would again
remain constant. This example highlights the essential property of interval
scale observations, namely that the zero point may be arbitrary, but that
ratios of differences between pairs of observations hold constant, regardless
of where the zero is placed and regardless of what the measurement unit is.
When measuring time at the granularity of minutes or seconds or hours or days
(but not years, because of the irregularity introduced by leap years) two
different measurement systems will have this property of being consistent in
terms of ratios of differences. We handle these scale and origin
transformations dozens of times every day, and (as one further example) know
that a meeting scheduled for 3:00pm–4:30pm tomorrow is $1.5$ times longer than
one that is scheduled for 10:15am–11:15am today.

In an interval scale each unit change in the measurement must have the same
interpretation, regardless of where in the scale it occurs. Those unit changes
can be denoted by equally-spaced tick marks on a ruler and then be labeled
with corresponding equally-spaced numeric measurements, as was shown in the
examples provided in Figure 1. But must a given set of observations
(professorial ranks mapped to salaries, or SERPs mapped to effectiveness
scores, or seconds since 0:00:00am on January 1, 1970) be required to _only_
fall on the equi-interval tick marks of some specific ruler in order for the
measurement (and more particularly, an average of measurements) to be possible
or valid? No! The observations can fall anywhere, and we can still measure
them, and then still average those measurements. Indeed, the constancy of
ratios of differences that is guaranteed by an interval scale means that we
may infer additional ticks on the ruler _anywhere that they might be helpful_.
If the salary ruler is labeled and ticked every $\$100$, we may infer tick
points at $\$50$ intervals as well, by taking ratios at intervals that are
$0.5$ long relative to the original tick intervals.666And that is exactly the
process that was used to place some of the elements of Figure 1. Or we can
infer tick marks every $\$10$, by taking ratios at intervals that are $0.1$
long relative to the original tick intervals. Indeed, we can infer a tick that
corresponds to the exact position on the ruler of _any object being measured_
, including a salary of $\$765.43$ per week, or an RR score of $1/7\approx
0.1429$, or a time that is $314.159$ seconds after 0:00:00am on January 1
1970.

We are not changing the scale when we add such “pro-rata” ticks to the ruler,
nor are we changing the previous equally-spaced unit labels. We are simply
employing the existing ruler, and our knowledge that it measures an interval
scale (and thus that ratios of differences have a valid interpretation) to
strategically add further ticks, so as to measure objects whose size is not
one of the original tick points on the ruler. I’ll say it again: _new tick-
points can be placed anywhere on the ruler, and once they have been labeled in
a proportionate manner, are first-class citizens, possessing the same rights,
privileges, and responsibilities as the original equi-spaced ticks._ An
interval scale does _not_ require that all valid measurements be at the places
on the ruler where the original evenly-spaced unit tick-marks arose. Nor does
it require that every original tick-point be attainable as an observation.

In contrast to this flexible interpretation of interval scale measurement,
Fuhr [2017] and Ferrante et al. [2021] espouse the more rigid requirement that
is documented in Equation 1. They require that the tick-marks and their labels
must only be at uniform intervals on the ruler; that every mapped observation
must exactly align with one of those equi-spaced tick-marks; and that every
tick-mark that appears on the ruler must be capable of being such a
measurement point.

In particular, Ferrante et al. [2022] assert that the professorial salaries
shown in Figure 1(a) are not “interval scale” because the red arrows denoting
the range of the mapping are not equi-spaced on the number line – see the
quotes already provided in Section 2. Ferrante et al. claim support for their
interpretation from the original work in this area, that of Stevens [1946],
who in connection with interval scales wrote “ _equal intervals of temperature
are scaled off by noting equal volumes of expansion_ ”, and “ _to devise
operations for equalizing the units of the scales_ ”. But in my opinion both
of those quotes refer to the starting configuration of the ruler, and the
overarching question that determines the scale of measurement: if the original
tick-points are equi-spaced on the ruler, then can equi-spaced numeric labels
be inferred, and if so, are those labels equi-spaced numerically? I do _not_
believe that Stevens’ comments imply a further mandate that every data
observation must fall at an equi-spaced tick-point on the ruler; and nor that
they imply that every equi-spaced label on the original ruler must be
available as an observation value.

In my opinion the three rulers shown in Figure 1 _do_ all have the property
underlying Stevens’ words: tick-points that are the same “distance” apart have
labels that are the same “value” apart. Some of the measurement values do not
align with the currently-drawn tick-points, that is correct. But as has
already been discussed, the first two rulers can be proportionately “sub-tick-
pointed” to _any required level of detail_ (twenty decimal places if
necessary, exceeding the resolution of even double arithmetic) so as to carry
out the required measurements. Provided that ratios of differences are
preserved, _we can put ticks and labels, and hence allow measurement, anywhere
we want on the ruler_. And, similarly, _we can remove from the ruler any tick
points and their corresponding labels that will not be needed when determining
the mapped measurements derived from a set of observations_.

To make this distinction concrete, I return to the professor’s salaries and
the restrictions implied by Equation 1. Ferrante et al. require that if an
ordinal-to-numeric salary mapping is to permit averaging, then either equi-
interval salaries of $\$700$, $\$800$, $\$900$, and $\$1000$ (or similar) must
be used; or several new grades of professor need to be introduced, to allow
salaries at all other multiples of $\$50$ between $\$700$ and $\$1000$; or
some combination between these two. That is, Ferrante et al. regard mapped
salary levels of $\$700$, $\$800$, $\$900$, and $\$1000$ as being “averagable”
given a set of “professorial job title” observations; and (would presumably)
also regard mapped salary levels of $\$700$, $\$801$, $\$902$, and $\$1003$ as
being “averagable”. But they regard averaging as being inappropriate for the
salary mapping that led to Question 6 (namely, $\$700$, $\$750$, $\$850$, and
$\$1000$), even though the ruler could equally have been ticked and labeled at
intervals of $\$50$; and for the same reason would, I presume, regard a salary
mapping to the values $\$701$, $\$800$, $\$900$, and $\$1000$ to also be non-
averagable, even though the ruler could equally have been ticked and labeled
at intervals of $\$1$.

In short, I’m bemused by the position that has been advocated by Ferrante et
al., and I firmly believe that the extraneous “interval scale” constraints
(captured via Equation 1, in line with Nicola Ferro’s answers to the quiz
questions) are neither required nor helpful. Certainly, in the balance beam
physical analogy (Figure 1) there is no requirement that the weights on the
beam be placed solely at a set of equi-spaced labeled tick-points on the ruler
in order for the fulcrum point to be determinable; and nor is the averaging
process rendered improper by the presence of labeled tick-points on the ruler
at which none of the weights can ever be placed.

All of which means that (in my opinion) effectiveness mappings like RR, AP,
NDCG, $\mbox{{\small\sf{RBP}}}_{0.8}$ and so on may be used with a clear
conscience – provided, of course, that the user model behind the selected
mapping can be argued as matching the task type and the behavior of the people
carrying out that task. Of course, the things being averaged must be numbers.
As I noted last year [Moffat, 2022], we can’t average international dialing
prefixes such as “+1”, “+61”, and “+40”. And nor can we average food additive
codes such as “414”, “950”, and “260”. In both of those examples the
observations are categorical, just like SERPs. We may average the numbers that
result from the application of a suitable (and argued for) categorical-to-
numeric mapping if we have reason to do so, but the observations themselves
cannot be averaged.

##  4 Ratio Scales

My next point suggests that there really is no need to be concerned about what
is and what is not an interval scale, and it is this: _effectiveness mappings
usually have a defined zero point_ , and hence _effectiveness scores are ratio
scale data_. In interval scale data the location of the zero point is
arbitrary, meaning that the computation of ratios of measurement differences
is permitted (because differences are origin independent), but that the
computation of ratios of actual measurements is not appropriate. But in ratio
scale data the fixing of the zero point means that ratios may be computed on
score _values_ as well as on score _differences_. For example, it is not
appropriate to state that $20^{\circ}$ is “twice as hot as” $10^{\circ}$ if we
are measuring temperature using Celsius or Fahrenheit, both of which interpret
“temperature” as being interval scale observations. In those two measurement
systems we are only permitted to say that $20^{\circ}$ is “$10^{\circ}$
hotter” than $10^{\circ}$, taking differences to do so. But we are permitted
to say that $20^{\circ}$ is “twice as hot as” $10^{\circ}$ if the temperatures
are being measured in degrees Kelvin, which has a fixed zero point, and is
thus a ratio scale. (Noting, of course, that $20^{\circ}$ Kelvin is a
different temperature to $20^{\circ}$ Fahrenheit, and that $20^{\circ}$
Celsius is different again).

Salaries also have a defined and unambiguous zero point. That means that in
the context of the professorial salaries supposed by Questions 1 to 6 (that
is, in the context established by one given ordinal-to-numeric mapping) it is
both defensible and informative to compute ratios and make statements such as
“at that university an assistant professor earns three-quarters the salary of
a full professor”. It is equally valid to compute a difference and write “at
that university an associate professor earns $\$100$ more per week than does
an assistant professor”. Neither of those statements is invariant to the
mapping, and if the salary scales change, or a different university is
considered, the relativities may well change too. But _in the context of the
specified mapping_ , such statements are legitimate.

With those examples in hand, let’s now turn back to categorical-to-numeric
effectiveness mappings. It was noted in Section 2 that the binary $k$-vector
$[0]^{k}$ is almost always mapped to a score of zero. And zero does indeed
have a fixed and identified meaning when evaluating SERP effectiveness – it
denotes the absence of usefulness. Hence, if the effectiveness mapping being
applied to a set of SERPs is one that maps $[0]^{k}$ to zero, then the set of
mapped effectiveness scores constitutes _ratio scale_ data in the context
provided by that mapping. And that means, for example, that we are permitted
to state (for that specified mapping) that an effectiveness score of $0.6$ is
“$1.5$ times larger” than an effectiveness score of $0.4$, as well as being
permitted to state that it is “$0.2$ bigger”. But it is also important to note
that such statements _do not and must not_ be used to make claims such as “if
$M_{1}(S_{1})$ is $1.5$ times bigger than $M_{1}(S_{2})$, then $M_{2}(S_{1})$
will also be $1.5$ times bigger than $M_{2}(S_{2})$”, where $S_{1}$ and
$S_{2}$ are two SERPs, and $M_{1}(\cdot)$ and $M_{2}(\cdot)$ are two
effectiveness mappings. As with the professor’s salaries, conclusions are not,
and never can be, invariant to the mapping.

To summarize this argument: if the effectiveness mapping in use assigns a
score of zero to useless SERPs (that is, those that are $[0]^{k}$), then
discussion of interval measures is irrelevant, because – strictly within the
context established by that effectiveness mapping – the (broader) power of
ratio scales may be applied to the set of SERP scores. Nor are the
categorical-to-numeric mappings ${\mbox{{\small\sf{Prec}}}}c@k$ and
$\mbox{{\small\sf{RBP}}}_{0.5}$ any different in this regard. Yes, they do
happen to have the property that (when there are $k$ or more relevant
documents in the collection) the mapped scores are a dense equi-spaced set of
points on the number line, satisfying Equation 1. But it is _still_ not
possible to make claims such as “if ${\mbox{{\small\sf{Prec}}}}c@k(S_{1})$ is
$1.5$ times bigger than ${\mbox{{\small\sf{Prec}}}}c@k(S_{2})$, then
$\mbox{{\small\sf{RBP}}}_{0.5}@k(S_{1})$ will also be $1.5$ times bigger than
$\mbox{{\small\sf{RBP}}}_{0.5}@k(S_{2})$”. As was noted in Section 2, SERPs
themselves are _categorical_ data, and it is simply impossible for there to be
any order-based, interval-based, or ratio-based invariance. The bottom line
here is that _each different effectiveness mapping is measuring a different
thing_.

Finally, there is one more issue to address. Ferrante et al. [2021] proceed to
describe the _intervalization_ process that is summarized in Table 1. A
conventional IR experiment can be thought of as following steps 0 to 3 of that
experimental sketch, leading to sets of scores that, in line with my arguments
here, may then be averaged or compared in other ways. But having claimed that
in general such sets of real-valued effectiveness scores may _not_ be
averaged, Ferrante et al. propose the “amelioration” shown at steps 4 and 5.
Their suggestion is that the scores be converted to ordinal integers, by
assigning a rank between $1$ (for the lowest possible score) and $2^{k}$ (or
less, depending on the effectiveness mapping, for the highest possible score)
to each possible numeric value that can arise from the chosen categorical-to-
numeric mapping when applied to the universe of $2^{k}$ binary $k$-element
SERP abstractions. And that those score ranks, which as is noted in the table,
are now _ordinal_ data, be converted back to effectiveness scores by dividing
each rank by the number of distinct scores arising, that is, by dividing by
$u=|{\cal{U}}|$.

Step | Activity | Observations and measurement scale
---|---|---
0 |  Design experiment; identify documents, topics, and queries; select and justify retrieval effectiveness mapping.
|  —

1 |  Carry out experiment, collect SERPs created by system(s) in response to queries.
|  Document $k$-vectors, one per system per query; _categorical_ data.

2 |  Collect relevance judgments, then apply them to the document $k$-vectors.
|  Binary $k$-vectors, one per system per query; _categorical_ data.

3 |  Apply effectiveness mapping chosen at step 0 (categorical-to-numeric) to the binary $k$-vectors, with $[0]^{k}\rightarrow 0$.
|  Numeric scores, one per system per query; _ratio_ data.

4 |  Transform each mapped score to its rank position in the sorted list of all possible scores possible for that mapping when applied to binary $k$-vectors.
|  Score ranks relative to ${\cal{U}}$, the universe of possible scores for
the mapping chosen at step 0; _ordinal_ data.

5 |  Transform SERP ranks to the range $[0,1]$ using equi-interval ordinal-to-numeric mapping.
|  Mapped score ranks in the range $[0,1]$;
_ratio_ data again.

Table 1: Experimental structure suggested by Ferrante et al. [2021]. Steps 0
to 3 are the standard IR experimental methodology; steps 4 and 5 are then
added by Ferrante et al. The commentary at the right is my analysis of the
data that is created by each step.

In this proposal if $M(\cdot)$ is the effectiveness mapping selected in step 0
and applied in step 3, first the universe of all possible mapped values
${\cal{U}}=\\{M(S)\mid S\in[\\{0,1\\}]^{k}\\}$ is computed; then the
cardinality of that set $u=|{\cal{U}}|$ is determined; and then each SERP $S$
observed in the experiment is represented by a “new” score $M^{\prime}(\cdot)$
computed as

$M^{\prime}(S)=\frac{\mbox{\emph{rank}}(M(S),\,{\cal{U}})-1}{u-1}\,,$

where $\mbox{\emph{rank}}(s,{\cal{U}})$ returns the integer value $|\\{x\mid
x\in{\cal{U}}\wedge x\leq s\\}|$; and where the ostensible goal is to convert
non-equi-interval observations to equi-interval observations via a non-linear
transformation from the real number line to the real number line.777Which is
not a permissible operation for either interval scale data or ratio scale
data. For example, in the case of the ${\mbox{{\small\sf{RR}}}}@3$ mapping
that was depicted in Figure 1(b), we have
${{\cal{U}}}_{\mbox{\scriptsize{RR3}}}=\\{0,1/3,1/2,1\\}$, and thus $u=4$;
which in turn means that the adjusted mapping $M^{\prime}(\cdot)$ assigns each
SERP $S$ one of $\mbox{\emph{equi-interval}}(1/3,0,4)=\\{0,1/3,2/3,1\\}$,
“sliding” the stack of observations at $1/2$ in Figure 1(b) rightward, to sit
at $2/3$. This new effectiveness mapping could be used in an IR experiment,
yes, _provided it is argued for based on a plausible model of user behavior_.
But it no longer reflects the user model associated with
${\mbox{{\small\sf{RR}}}}@3$; it measures SERP usefulness via a different user
model.

The annotations in the right column in Table 1 make clear what, in my opinion,
is happening: step 3 creates ratio scale values that, by virtue of and in the
context of the user model associated the chosen effectiveness mapping, are
believed to represent user satisfaction with observed SERPs. Those ratio scale
values are then being deliberately stripped of their ratio-based relativities
(step 4). Instead, the scores are distorted into a new arrangement (step 5)
that possesses the same ordering, yes, but associates different scores; thus
undermining the interpretability of the experimental results.

Ferrante et al. [2021] further note that in certain cases
$M^{\prime}(S)=c\cdot M(S)$ for a multiplicative constant $c>0$ and all binary
$k$-vectors $S$. For example, provided there are at least $k$ relevant
documents, both ${\mbox{{\small\sf{Prec}}}}c@k$ and
$\mbox{{\small\sf{RBP}}}_{0.5}@k$ (the RBP mapping, with parameter $\phi=0.5$
and calculation limited to a SERP length of $k$) have this property. On the
other hand, RR does not, as is evident in the previous example. But even when
this property holds for two mappings $M_{1}(\cdot)$ and $M_{2}(\cdot)$, they
will _still_ not be ratio-of-difference invariant or ratio-invariant with
respect to each other. _Every categorical-to-numeric effectiveness mapping,
whether equi-interval or not, is measuring something different._

As I argued previously [Moffat, 2022], I doubt the need for the
intervalization process. Yes, it is another way of developing new
effectiveness mappings that might be argued for and then used; but no, those
new mappings do not possess any innate properties that make them, absent of
any discussion of users and their behavior, preferable to the mappings they
are derived from. Intervalization most certainly does _not_ result in order-
based, ratio-of-difference-based, or ratio-based score invariance across
different effectiveness mappings. That simply isn’t possible, because the
SERPs themselves are _categorical data_.

##  5 The Case For Reciprocal Rank

I’ll admit to not being a great fan of RR as a way of assessing SERP
usefulness. It is a relatively blunt instrument, and for strongly top-weighted
evaluations I’d probably use $\mbox{{\small\sf{RBP}}}_{0.5}$ as a measurement
tool, so that score ties are broken by taking into account the position of any
subsequent relevant documents after the first. Diaz [2023] has also recently
considered how to add sensitivity to RR while retaining a strong focus on the
head of the SERP.

On the other hand, I’m also willing to accept that for some applications RR
might be exactly the right way to assess user perceptions of SERP usefulness.
For example, mention was made in Section 1 of search tasks in which “users
place $d$ times more value on having the first relevant document at rank $1$
than they do on having the first relevant document at rank $d$” – which, by
the way, is a statement about relativities that directly corresponds with the
use of ratio scales.

Most importantly of all, I’d like my colleagues in the IR community to be able
to use RR (or AP, or NDCG, or $\mbox{{\small\sf{RBP}}}_{0.8}$, and so on) in
their evaluations if they wish to. To satisfy me they need to provide a
justification as to _why_ they have chosen some particular effectiveness
mapping, preferably grounded in a discussion about user behavior; and in my
opinion that rationale is an important component of the experimental design
(step 0 in Table 1). But once they have made that argument (and it has been
accepted as plausible) they should be able to report their results and present
their papers at conferences without being concerned that they may have erred
in some nebulous way that they don’t understand, and without attracting
“methodology” criticisms that they don’t deserve.

Also worth noting is that recent work with Joel Mackenzie [Moffat and
Mackenzie, 2023] has led to a proposal for comparing sets of SERPs (in other
words, comparing systems over a set of topics) via any _innate pairwise SERP
orderings_ that can be identified. As was discussed in Section 2, there is no
total ordering of binary $k$-vectors, with the example $S_{1}=[1,1,0,0,0]$
versus $S_{2}=[0,0,1,1,1]$ used to defend the claim that SERPs are categorical
data. But there are also many sub-chains and partial-orderings of SERPs. For
example, the two $5$-vectors $S_{3}=[1,0,1,1,0]$ and $S_{4}=[0,1,0,0,1]$
probably _can_ be ordered – at least for all plausible effectiveness mappings
$M(\cdot)$ – as being $M(S_{3})\not<M(S_{4})$, with $S_{3}$ possessing both
more total relevance, and also unambiguously earlier relevance. That is, we
cannot imagine any user-based model for SERP consumption under which a user
would find $S_{4}$ more useful than $S_{3}$. This proposal is based on the
“Rule 1” and “Rule 2” relationships I have previously written about [Moffat,
2022], and leads to a technique we dub IPSO for carrying out mapping-agnostic
system comparisons in some circumstances. Diaz and Ferraro [2022] have also
considered “metric free” comparisons.

Finally, at a more abstract level of experimental commentary, note that Zobel
[2022] has recently pondered the degree to which batch evaluations of the type
considered in detail here should be relied on as a way of assessing search
performance; and discusses the extent to which any numeric measurement can be
employed as a reliable surrogate for what is, fundamentally, a qualitative
human-oriented goal.

### Acknowledgement

This work was supported under the Australian Research Council’s Discovery
Projects funding scheme (project DP190101113). A number of colleagues have
contributed their thoughts in response to drafts, including Joel Mackenzie,
Justin Zobel, Paul Thomas, and Tetsuya Sakai.

## References

  * Diaz [2023] F. Diaz.  Best-case retrieval evaluation: Improving the sensitivity of reciprocal rank with lexicographic precision, 2023.  arXiv:2306.07908v1.
  * Diaz and Ferraro [2022] F. Diaz and A. Ferraro.  Offline retrieval evaluation without evaluation metrics.  In _Proc. ACM Int. Conf. on Research and Development in Information Retrieval (SIGIR)_ , pages 599–609, 2022.
  * Ferrante et al. [2019a] M. Ferrante, N. Ferro, E. Losiouk, and S. Pontarollo.  Scales of evaluation measures: From theory to experimentation.  In _Proc. Italian Inf. Retr. Wrkshp._ , pages 10–11, 2019a.
  * Ferrante et al. [2019b] M. Ferrante, N. Ferro, and S. Pontarollo.  A general theory of IR evaluation measures.  _IEEE Transactions on Knowledge and Data Engineering_ , 31(3):409–422, 2019b.
  * Ferrante et al. [2020] M. Ferrante, N. Ferro, and E. Losiouk.  How do interval scales help us with better understanding IR evaluation measures?  _Information Retrieval_ , 23(3):289–317, 2020\.
  * Ferrante et al. [2021] M. Ferrante, N. Ferro, and N. Fuhr.  Towards meaningful statements in IR evaluation: Mapping evaluation measures to interval scales.  _IEEE Access_ , 9:136182–136216, 2021.
  * Ferrante et al. [2022] M. Ferrante, N. Ferro, and N. Fuhr.  Response to Moffat’s comment on “Towards meaningful statements in IR evaluation: Mapping evaluation measures to interval scales”.  arXiv2212.11735, 2022.
  * Fuhr [2017] N. Fuhr.  Some common mistakes in IR evaluation, and how they can be avoided.  _SIGIR Forum_ , 51(3):32–41, 2017.
  * Moffat [2022] A. Moffat.  Batch evaluation metrics in information retrieval: Measures, scales, and meaning.  _IEEE Access_ , 10:105564–105577, 2022.
  * Moffat and Mackenzie [2023] A. Moffat and J. Mackenzie.  How much freedom does an effectiveness metric really have?  arXiv:2309.09477, 2023.
  * Moffat et al. [2022] A. Moffat, J. Mackenzie, P. Thomas, and L. Azzopardi.  A flexible framework for offline effectiveness metrics.  In _Proc. ACM Int. Conf. on Research and Development in Information Retrieval (SIGIR)_ , pages 578–587, 2022.
  * Sakai [2020] T. Sakai.  On Fuhr’s guideline for IR evaluation.  _SIGIR Forum_ , 54(1):12:1–12:8, 2020.
  * Stevens [1946] S. S. Stevens.  On the theory of scales of measurement.  _Science_ , 103(2684):677–680, 1946.
  * Zobel [2022] J. Zobel.  When measurement misleads: The limits of batch assessment of retrieval systems.  _SIGIR Forum_ , 56(1):1–20, 2022.


# Ideals with approximate unit in semicrossed products

Charalampos Magiatis

###### Abstract

We characterize the ideals of the semicrossed product
$C_{0}(X)\times_{\phi}\mathbb{Z}_{+}$ with left (resp. right) approximate
unit.

##  1 Introduction and Notation

The semicrossed product is a non-selfadjoint operator algebra which is
constructed from a dynamical system. We recall the construction of the
semicrossed product we will consider in this work. Let $X$ be a locally
compact Hausdorff space and $\phi:X\rightarrow X$ be a continuous and proper
surjection (recall that a map $\phi$ is _proper_ if the inverse image
$\phi^{-1}(K)$ is compact for every compact $K\subseteq X$). The pair
$(X,\phi)$ is called a _dynamical system_. An action of
$\mathbb{Z}_{+}:=\mathbb{N}\cup\\{0\\}$ on $C_{0}(X)$ by isometric
$*$-automorphisms $\alpha_{n}$, $n\in\mathbb{Z}_{+}$, is obtained by defining
$\alpha_{n}(f)=f\circ\phi^{n}$. We write the elements of the Banach space
$\ell^{1}(\mathbb{Z}_{+},C_{0}(X))$ as formal series
$A=\sum_{n\in\mathbb{Z}_{+}}U^{n}f_{n}$ with the norm given by
$\|A\|_{1}=\sum_{n\in\mathbb{Z}_{+}}\|f_{n}\|_{C_{0}(X)}$. Multiplication on
$\ell^{1}(\mathbb{Z}_{+},C_{0}(X))$ is defined by setting

$(U^{n}f)(U^{m}g)=U^{n+m}(\alpha^{m}(f)g)\ ,$

and extending by linearity and continuity. With this multiplication
$\ell^{1}(\mathbb{Z}_{+},C_{0}(X))$ is a Banach algebra.

The Banach algebra $\ell^{1}(\mathbb{Z}_{+},C_{0}(X))$ can be faithfully
represented as a (concrete) operator algebra on a Hilbert space. This is
achieved by assuming a faithful action of $C_{0}(X)$ on a Hilbert space
$\mathcal{H}_{0}$. Then we can define a faithful contractive representation
$\pi$ of $\ell_{1}(\mathbb{Z}_{+},C_{0}(X))$ on the Hilbert space
$\mathcal{H}=\mathcal{H}_{0}\otimes\ell^{2}(\mathbb{Z}_{+})$ by defining
$\pi(U^{n}f)$ as

$\pi(U^{n}f)(\xi\otimes e_{k})=\alpha^{k}(f)\xi\otimes e_{k+n}\ .$

The _semicrossed product_ $C_{0}(X)\times_{\phi}\mathbb{Z}_{+}$ is the closure
of the image of $\ell^{1}(\mathbb{Z}_{+},C_{0}(X))$ in $\mathcal{B(H)}$ in the
representation just defined. We will denote an element $\pi(U^{n}f)$ of
$C_{0}(X)\times_{\phi}\mathbb{Z}_{+}$ by $U^{n}f$ to simplify the notation.

For
$A=\sum_{n\in\mathbb{Z}_{+}}U^{n}f_{n}\in\ell^{1}(\mathbb{Z}_{+},C_{0}(X))$ we
call $f_{n}\equiv E_{n}(A)$ the _$n$ th Fourier coefficient_ of $A$. The maps
$E_{n}:\ell^{1}(\mathbb{Z}_{+},C_{0}(X))\rightarrow C_{0}(X)$ are contractive
in the (operator) norm of $C_{0}(X)\times_{\phi}\mathbb{Z}_{+}$, and therefore
they extend to contractions
$E_{n}:C_{0}(X)\times_{\phi}\mathbb{Z}_{+}\rightarrow C_{0}(X)$. An element
$A$ of the semicrossed product $C_{0}(X)\times_{\phi}\mathbb{Z}_{+}$ is $0$ if
and only if $E_{n}(A)=0$, for all $n\in\mathbb{Z}_{+}$, and thus $A$ is
completely determined by its Fourier coefficients. We will denote $A$ by the
formal series $A=\sum_{n\in\mathbb{Z}_{+}}U^{n}f_{n}$, where $f_{n}=E_{n}(A)$.
Note however that the series $\sum_{n\in\mathbb{Z}_{+}}U^{n}f_{n}$ does not in
general converge to $A$ [6, II.9 p.512]. The _$k$ th arithmetic mean_ of $A$
is defined to be $\bar{A}_{k}=\frac{1}{k+1}\sum_{l=0}^{k}S_{l}(A)$, where
$S_{l}(A)=\sum_{n=0}^{l}U^{n}f_{n}$. Then, the sequence
$\\{\bar{A}_{k}\\}_{k\in\mathbb{Z}_{+}}$ is norm convergent to $A$ [6, Remark
p.524]. We refer to [6, 4, 3] for more information about the semicrossed
product.

Let $\\{X_{n}\\}_{n=0}^{\infty}$ be a sequence of closed subsets of $X$
satisfying

$\displaystyle X_{n+1}\cup\phi(X_{n+1})\subseteq X_{n}\ ,$ ($*$)

for all $n\in\mathbb{N}$. Peters proved in [7] that there is a one-to-one
correspondence between closed two-sided ideals $\mathcal{I}\subseteq
C_{0}(X)\times_{\phi}\mathbb{Z}_{+}$ and sequences
$\\{X_{n}\\}_{n=0}^{\infty}$ of closed subsets of $X$ satisfying ($*$ ‣ 1),
under the additional assumptions that $X$ is metrizable and the dynamical
system $(X,\phi)$ contains no periodic points. In fact, the ideal
$\mathcal{I}$ associated with the sequence $\\{X_{n}\\}_{n=0}^{\infty}$ is
$\mathcal{I}=\\{A\in
C_{0}(X)\times_{\phi}\mathbb{Z}_{+}:E_{n}(A)(X_{n})=\\{0\\}\\}$. We will write
this as $\mathcal{I}\sim\\{X_{n}\\}_{n=0}^{\infty}$. Moreover, under the above
assumptions, he characterizes the maximal and prime ideals of the semicrossed
product $C_{0}(X)\times_{\phi}\mathbb{Z}_{+}$.

Donsig, Katavolos and Manousos obtained in [4] a characterization of the
Jacobson radical for the semicrossed product
$C_{0}(X)\times_{\phi}\mathbb{Z}_{+}$, where $X$ is a locally compact
metrisable space and $\phi:X\rightarrow X$ is a continuous and proper
surjection. Andreolas, Anoussis and the author characterized in [2] the ideal
generated by the compact elements and in [1] the hypocompact and the scattered
radical of the semicrossed product $C_{0}(X)\times_{\phi}\mathbb{Z}_{+}$,
where $X$ is a locally compact Hausdorff space and $\phi:X\rightarrow X$ is a
homeomorphism. All these ideals are of the form
$\mathcal{I}\sim\\{X_{n}\\}_{n=0}^{\infty}$ for suitable families of closed
subsets $\\{X_{n}\\}_{n=0}^{\infty}$.

In the present paper we characterize the closed two-sided ideals
$\mathcal{I}\sim\\{X_{n}\\}_{n=0}^{\infty}$ of
$C_{0}(X)\times_{\phi}\mathbb{Z}_{+}$ with left (resp. right) approximate
unit. As a consequence, we obtain a complete characterization of ideals with
left (resp. right) approximate unit under the additional assumptions that $X$
is metrizable and the dynamical system $(X,\phi)$ contains no periodic points.

Recall that a _left_ (resp. _right_) _approximate unit_ of a Banach algebra
$\mathcal{A}$ is a net $\\{u_{\lambda}\\}_{\lambda\in\Lambda}$ of elements of
$\mathcal{A}$ such that:

  1. 1.

for some positive number $r$, $\|u_{\lambda}\|\leq r$ for all
$\lambda\in\Lambda$,

  2. 2.

$\lim u_{\lambda}a=a$ (resp. $\lim au_{\lambda}=a$), for all
$a\in\mathcal{A}$, in the norm topology of $\mathcal{A}$.

A net which is both a left and a right approximate unit of $\mathcal{A}$ is
called an _approximate unit_ of $\mathcal{A}$. A left (resp. right)
approximate unit $\\{u_{\lambda}\\}_{\lambda\in\Lambda}$ that satisfies
$\|u_{\lambda}\|\leq 1$ for all $\lambda\in\Lambda$ is called a _contractive
left_ (resp. _right_) _approximate unit_.

We will say that an ideal $\mathcal{I}$ of a Banach algebra $\mathcal{A}$ has
a left (resp. right) approximate unit if it has a left (resp. right)
approximate unit as an algebra.

##  2 Ideals with approximate unit

In the following theorem the ideals
$\mathcal{I}\sim\\{X_{n}\\}_{n=0}^{\infty}$ with right approximate unit are
characterized.

###### Theorem 2.1

Let $\mathcal{I}\sim\\{X_{n}\\}_{n=0}^{\infty}$ be a non-zero ideal of
$C_{0}(X)\times_{\phi}\mathbb{Z}_{+}$. The following are equivalent:

  1. 1.

$\mathcal{I}$ has a right approximate unit.

  2. 2.

$X_{n}=X_{n+1}$, for all $n\in\mathbb{Z}_{+}$.

Proof. We start by proving that (1) $\Rightarrow$ (2). Let
$\mathcal{I}\sim\\{X_{n}\\}_{n=0}^{\infty}$ be an ideal with right approximate
unit $\\{V_{\lambda}\\}_{\lambda\in\Lambda}$. We suppose that there exists
$n\in\mathbb{Z}_{+}$ such that $X_{n+1}\subsetneq X_{n}$. Let

$n_{0}=\min\\{n\in\mathbb{Z}_{+}:X_{n+1}\subsetneq X_{n}\\}\ ,$

$x_{0}\in X_{n_{0}}\setminus X_{n_{0}+1}$ and $f\in C_{0}(X)$ such that
$f(x_{0})=1$, $f(X_{n_{0}+1})=\\{0\\}$ and $\|f\|=1$. Then, for
$A=U^{n_{0}+1}f$, we have

$\|AV_{\lambda}-A\|\geq\|E_{n_{0}+1}(AV_{\lambda}-A)\|=\|fE_{0}(V_{\lambda})-f\|\geq|(fE_{0}(V_{\lambda})-f)(x_{0})|=1\
,$

for all $\lambda\in\Lambda$, since $x_{0}\in X_{n_{0}}$ and
$E_{0}(V_{\lambda})(X_{n_{0}})=0$, which is a contradiction. Therefore
$X_{n}=X_{n+1}$ for all $n\in\mathbb{Z}_{+}$.

For (2) $\Rightarrow$ (1), assume that $X_{n}=X_{n+1}$ for all
$n\in\mathbb{Z}_{+}$. By ($*$ ‣ 1), we get that $\phi(X_{0})\subseteq X_{0}$.
We will show that if $\\{u_{\lambda}\\}_{\lambda\in\Lambda}$ is a contractive
approximate unit of the ideal $C_{0}(X\setminus X_{0})$ of $C_{0}(X)$, then
$\\{U^{0}u_{\lambda}\\}_{\lambda\in\Lambda}$ is a right approximate unit of
$\mathcal{I}$. Since $\|u_{\lambda}\|\leq 1$, we have
$\|U^{0}u_{\lambda}\|\leq 1$.

Let $A\in\mathcal{I}$ and $\varepsilon>0$. Then there exists
$k\in\mathbb{Z}_{+}$ such that

$\|A-\bar{A}_{k}\|<\frac{\varepsilon}{4}\ ,$

where $\bar{A}_{k}$ is the $k$th arithmetic mean of $A$. Since $X_{n}=X_{0}$,
$E_{n}(\bar{A}_{k})\in C_{0}(X\setminus X_{0})$ and
$\\{u_{\lambda}\\}_{\lambda\in\Lambda}$ is an approximate unit of
$C_{0}(X\setminus X_{0})$, there exists $\lambda_{0}\in\Lambda$ such that

$\|E_{l}(\bar{A}_{k})u_{\lambda}-E_{l}(\bar{A}_{k})\|<\frac{\varepsilon}{2(k+1)}\
,$

for all $l\leq k$ and $\lambda>\lambda_{0}$. So, for $\lambda>\lambda_{0}$ we
get that

$\displaystyle\|AU^{0}u_{\lambda}-A\|$ $\displaystyle<$
$\displaystyle\|AU^{0}u_{\lambda}-\bar{A}_{k}U^{0}u_{\lambda}+\bar{A}_{k}U^{0}u_{\lambda}-\bar{A}_{k}+\bar{A}_{k}-A\|$
$\displaystyle<$
$\displaystyle\|AU^{0}u_{\lambda}-\bar{A}_{k}U^{0}u_{\lambda}\|+\|\bar{A}_{k}U^{0}u_{\lambda}-\bar{A}_{k}\|+\|A-\bar{A}_{k}\|$
$\displaystyle<$
$\displaystyle\|\bar{A}_{k}U^{0}u_{\lambda}-\bar{A}_{k}\|+\frac{\varepsilon}{2}$
$\displaystyle\leq$
$\displaystyle\sum_{l=0}^{k}\|E_{l}(\bar{A}_{k})u_{\lambda}-E_{l}(\bar{A}_{k})\|+\frac{\varepsilon}{2}$
$\displaystyle<$ $\displaystyle\varepsilon\ ,$

which concludes the proof. $\Box$

In the following theorem the ideals
$\mathcal{I}\sim\\{X_{n}\\}_{n=0}^{\infty}$ with left approximate unit are
characterized.

###### Theorem 2.2

Let $\mathcal{I}\sim\\{X_{n}\\}_{n=0}^{\infty}$ be a non-zero ideal of
$C_{0}(X)\times_{\phi}\mathbb{Z}_{+}$. The following are equivalent:

  1. 1.

$\mathcal{I}$ has a left approximate unit.

  2. 2.

$X_{0}\subsetneq X$ and $\phi^{n}(X\setminus X_{n})=X\setminus X_{0}$, for all
$n\in\mathbb{Z}_{+}$.

  3. 3.

$\phi(X\setminus X_{1})=X\setminus X_{0}$ and $\phi(X_{n+1}\setminus
X_{n+2})=X_{n}\setminus X_{n+1}$, for all $n\in\mathbb{Z}_{+}$.

Proof. We start by proving that (1) $\Rightarrow$ (2). Let
$\mathcal{I}\sim\\{X_{n}\\}_{n=0}^{\infty}$ be an ideal with left approximate
unit $\\{V_{\lambda}\\}_{\lambda\in\Lambda}$.

First we prove that $X_{0}\subsetneq X$. We suppose that $X_{0}=X$. Then
$E_{0}(V_{\lambda})=0$, for all $\lambda\in\Lambda$, and hence for every
$U^{n}f\in\mathcal{I}$ we have

$\|V_{\lambda}U^{n}f-U^{n}f\|\geq\|E_{n}(V_{\lambda}U^{n}f-U^{n}f)\|=\|E_{0}(V_{\lambda})f-f\|=\|f\|\
,$

for all $\lambda\in\Lambda$, which is a contradiction. Therefore
$X_{0}\subsetneq X$.

Now we prove that $\phi^{n}(X\setminus X_{n})=X\setminus X_{0}$, for all
$n\in\mathbb{Z}_{+}$. We suppose that there exists $n\in\mathbb{Z}_{+}$ such
that $\phi^{n}(X\setminus X_{n})\not\subseteq X\setminus X_{0}$ and let

$n_{0}=\min\\{n\in\mathbb{Z}_{+}:\phi^{n}(X\setminus X_{n})\not\subseteq
X\setminus X_{0}\\}\ .$

Then, there exist $x_{0}\in X\setminus X_{n_{0}}$ such that
$\phi^{n_{0}}(x_{0})\in X_{0}$ and a function $f\in C_{0}(X)$ such that
$f(x_{0})=1$, $f(X_{n_{0}})=\\{0\\}$ and $\|f\|=1$. If $A=U^{n_{0}}f$, we have
that $A\in\mathcal{I}$, $\|A\|=1$ and

$\displaystyle\|V_{\lambda}A-A\|$ $\displaystyle\geq$
$\displaystyle\|E_{n_{0}}(V_{\lambda}A-A)\|$ $\displaystyle=$
$\displaystyle\|E_{0}(V_{\lambda})\circ\phi^{n_{0}}f-f\|$ $\displaystyle\geq$
$\displaystyle|(E_{0}(V_{\lambda})\circ\phi^{n_{0}}f-f)(x_{0})|$
$\displaystyle=$ $\displaystyle 1\ ,$

for all $\lambda\in\Lambda$, since $\phi^{n_{0}}(x_{0})\in X_{0}$ and
$E_{0}(V_{\lambda})(X_{0})=\\{0\\}$, which is a contradiction. Therefore
$\phi^{n}(X\setminus X_{n})\subseteq X\setminus X_{0}$. Furthermore, by ($*$ ‣
1) we get that $\phi^{n}(X_{n})\subseteq X_{0}$, for all $n\in\mathbb{Z}_{+}$,
and hence

$X=\phi^{n}(X)=\phi^{n}(X_{n}\cup(X\setminus
X_{n}))=\phi^{n}(X_{n})\cup\phi^{n}(X\setminus X_{n})\subseteq
X_{0}\cup\phi^{n}(X\setminus X_{n})\ .$

Since $\phi^{n}(X\setminus X_{n})\subseteq X\setminus X_{0}$ and $\phi$ is
surjective, $\phi^{n}(X\setminus X_{n})=X\setminus X_{0}$, for all
$n\in\mathbb{Z}_{+}$.

For (2) $\Rightarrow$ (1), assume that $X_{0}\subsetneq X$ and
$\phi^{n}(X\setminus X_{n})=X\setminus X_{0}$, for all $n\in\mathbb{Z}_{+}$.
We will show that if $\\{u_{\lambda}\\}_{\lambda\in\Lambda}$ is a contractive
approximate unit of the ideal $C_{0}(X\setminus X_{0})$ of $C_{0}(X)$, then
$\\{U^{0}u_{\lambda}\\}_{\lambda\in\Lambda}$ is a left approximate unit of
$\mathcal{I}$. Since $\|u_{\lambda}\|\leq 1$, we have
$\|U^{0}u_{\lambda}\|\leq 1$.

Let $A$ be a norm-one element of $\mathcal{I}$ and $\varepsilon>0$. Then there
exists $k\in\mathbb{Z}_{+}$ such that

$\|A-\bar{A}_{k}\|<\frac{\varepsilon}{4}\ ,$

where $\bar{A}_{k}$ is the $k$th arithmetic mean of $A$. For $l\leq k$, let

$D_{\varepsilon}(E_{l}(\bar{A}_{k}))=\left\\{x\in
X:|E_{l}(\bar{A}_{k})(x)|\geq\frac{\varepsilon}{4(k+1)}\right\\}\ .$

Since $A\in\mathcal{I}$, we have $E_{l}(\bar{A}_{k})(X_{l})=\\{0\\}$ and hence
$D_{\varepsilon}(E_{l}(\bar{A}_{k}))\subseteq X\setminus X_{l}$. Furthermore,
since $\phi^{n}(X\setminus X_{n})=X\setminus X_{0}$, for all
$n\in\mathbb{Z}_{+}$, we have that
$\phi^{l}(D_{\varepsilon}(E_{l}(\bar{A}_{k})))\subseteq X\setminus X_{0}$.
Moreover, the set $D_{\varepsilon}(E_{l}(\bar{A}_{k}))$ is compact and hence
the set $\phi^{l}(D_{\varepsilon}(E_{l}(\bar{A}_{k})))$ is also compact. By
Urysohn’s lemma, there is a norm-one function $v_{l}\in C_{0}(X)$ such that

$v_{l}(x)=\left\\{\begin{tabular}[]{ll}$1$,&$x\in\phi^{l}(D_{\varepsilon}(E_{l}(\bar{A}_{k})))$\\\
$0$,&$x\in X_{0}$\end{tabular}\right..$

Then, there exists $\lambda_{0}\in\Lambda$ such that

$\|u_{\lambda}v_{l}-v_{l}\|<\frac{\varepsilon}{2(k+1)}\ ,$

for all $l\leq k$ and $\lambda>\lambda_{0}$, and hence

$|u_{\lambda}(x)-1|<\frac{\varepsilon}{2(k+1)}\ ,$

for all $x\in\cup_{l=0}^{k}\phi^{l}(D_{\varepsilon}(E_{l}(\bar{A}_{k})))$ and
$\lambda>\lambda_{0}$. Therefore, if
$x\in\cup_{l=0}^{k}(D_{\varepsilon}(E_{l}(\bar{A}_{k})))$ then
$\phi^{l}(x)\in\cup_{l=0}^{k}\phi^{l}(D_{\varepsilon}(E_{l}(\bar{A}_{k})))$
and hence

$\|((u_{\lambda}\circ\phi^{l})E_{l}(\bar{A}_{k})-E_{l}(\bar{A}_{k}))(x)\|<\frac{\varepsilon}{2(k+1)}\
,$

for all $l\leq k$ and $\lambda>\lambda_{0}$. On the other hand, if
$x\not\in\cup_{l=0}^{k}(D_{\varepsilon}(E_{l}(\bar{A}_{k})))$, then

$|E_{l}(\bar{A}_{k})(x)|<\frac{\varepsilon}{4(k+1)}\ ,$

for all $l\leq k$, and hence

$\|((u_{\lambda}\circ\phi^{l})E_{l}(\bar{A}_{k})-E_{l}(\bar{A}_{k}))(x)\|<\frac{\varepsilon}{2(k+1)}\
.$

From what we said so far we get that

$\displaystyle\|U^{0}u_{\lambda}A-A\|$ $\displaystyle<$
$\displaystyle\|U^{0}u_{\lambda}\bar{A}_{k}-\bar{A}_{k}\|+\frac{\varepsilon}{2}$
$\displaystyle\leq$
$\displaystyle\sum_{l=0}^{k}\|(u_{\lambda}\circ\phi^{l})E_{l}(\bar{A}_{k})-E_{l}(\bar{A}_{k})\|+\frac{\varepsilon}{2}$
$\displaystyle<$ $\displaystyle\varepsilon\ ,$

for all $\lambda>\lambda_{0}$.

Now we show that (2) $\Rightarrow$ (3). We assume that $\phi^{n}(X\setminus
X_{n})=X\setminus X_{0}$, for all $n\in\mathbb{Z}_{+}$. Then $\phi(X\setminus
X_{n+2})\subseteq X\setminus X_{n+1}$. Indeed, if $x\in X\setminus X_{n+2}$
and $\phi(x)\in X_{n+1}$ then $\phi^{n+2}(x)\in X_{0}$, by ($*$ ‣ 1), which is
a contradiction. Furthermore, by ($*$ ‣ 1), we know that
$\phi(X_{n+1})\subseteq X_{n}$ and hence $\phi(X_{n+1}\setminus
X_{n+2})\subseteq X_{n}\setminus X_{n+1}$ for all $n\in\mathbb{Z}_{+}$.

To prove that $\phi(X_{n+1}\setminus X_{n+2})=X_{n}\setminus X_{n+1}$ for all
$n\in\mathbb{Z}_{+}$, we suppose that there exists $n\in\mathbb{Z}_{+}$ such
that $\phi(X_{n+1}\setminus X_{n+2})\subsetneq X_{n}\setminus X_{n+1}$. If

$n_{0}=\min\\{n\in\mathbb{Z}_{+}:\phi(X_{n+1}\setminus X_{n+2})\subsetneq
X_{n}\setminus X_{n+1}\\}\ ,$

then

$\displaystyle\phi(X)$ $\displaystyle=$
$\displaystyle\phi(X_{n_{0}+1}\cup(X\setminus X_{n_{0}+1}))$ $\displaystyle=$
$\displaystyle\phi(X_{n_{0}+1})\cup\phi(X\setminus X_{n_{0}+1})$
$\displaystyle\subseteq$ $\displaystyle\phi(X_{n_{0}+1})\cup(X\setminus
X_{n_{0}})$ $\displaystyle\subsetneq$ $\displaystyle X\ ,$

which is a contradiction, since $\phi$ is surjective. Therefore,
$\phi(X_{n+1}\setminus X_{n+2})=X_{n}\setminus X_{n+1}$ for all
$n\in\mathbb{Z}_{+}$.

Finally we show that (3) $\Rightarrow$ (2). We assume that $\phi(X\setminus
X_{1})=X\setminus X_{0}$ and $\phi(X_{n+1}\setminus X_{n+2})=X_{n}\setminus
X_{n+1}$, for all $n\in\mathbb{Z}_{+}$. Then, $X_{0}\subsetneq X$. Indeed, if
$X_{0}=X$, then $\mathcal{I}\equiv\\{0\\}$ which is a contradiction. If $n>1$,
we have that

$\displaystyle\phi(X\setminus X_{n})$ $\displaystyle=$
$\displaystyle\phi\left[(X\setminus X_{1})\cup(X_{1}\setminus
X_{2})\cup\dots\cup(X_{n-1}\setminus X_{n})\right]$ $\displaystyle=$
$\displaystyle\phi(X\setminus X_{1})\cup\phi(X_{1}\setminus
X_{2})\cup\dots\cup\phi(X_{n-1}\setminus X_{n})$ $\displaystyle=$
$\displaystyle(X\setminus X_{0})\cup(X_{0}\setminus
X_{1})\cup\dots\cup(X_{n-2}\setminus X_{n-1})$ $\displaystyle=$ $\displaystyle
X\setminus X_{n-1}\ ,$

and hence, $\phi^{n}(X\setminus X_{n})=X\setminus X_{0}$, for all
$n\in\mathbb{Z}_{+}$. $\Box$

By Theorem 2.2, if $\mathcal{I}\sim\\{X_{n}\\}_{n=0}^{\infty}$ is an ideal of
$C_{0}(X)\times_{\phi}\mathbb{Z}_{+}$ with left approximate unit, then
$X_{n+1}=X_{n}$ or $X_{n+1}\subsetneq X_{n}$ for all $n\in\mathbb{Z}_{+}$. If
$\mathcal{I}\sim\\{X_{n}\\}_{n=0}^{\infty}$ and $X_{n+1}=X_{n}$, for all
$n\in\mathbb{Z}_{+}$, we will write $\mathcal{I}\sim\\{X_{0}\\}$. We obtain
the following characterization.

###### Corollary 2.3

Let $\mathcal{I}\sim\\{X_{0}\\}$ be a non-zero ideal of
$C_{0}(X)\times_{\phi}\mathbb{Z}_{+}$. The following are equivalent:

  1. 1.

$\mathcal{I}$ has a left approximate unit.

  2. 2.

$\phi(X_{0})=X_{0}$ and $\phi(X\setminus X_{0})=X\setminus X_{0}$.

Proof. By Theorem 2.2 we have $\phi(X\setminus X_{0})=X\setminus X_{0}$. By
($*$ ‣ 1) we have $\phi(X_{0})\subseteq X_{0}$ and since $\phi$ is surjective
we get $\phi(X_{0})=X_{0}$. $\Box$

In the following proposition the ideals
$\mathcal{I}\sim\\{X_{n}\\}_{n=1}^{\infty}$ of
$C_{0}(X)\times_{\phi}\mathbb{Z}_{+}$ with left approximate unit are
characterized, when $\phi$ is a homeomorphism.

###### Proposition 2.4

Let $\mathcal{I}\sim\\{X_{n}\\}_{n=1}^{\infty}$ be a non-zero ideal of
$C_{0}(X)\times_{\phi}\mathbb{Z}_{+}$, where $\phi$ is a homeomorphism. The
following are equivalent:

  1. 1.

$\mathcal{I}$ has a left approximate unit.

  2. 2.

There exist $S,W\subsetneq X$ such that $S$ is closed and $\phi(S)=S$, the
sets $\phi^{-1}(W),\phi^{-2}(W),\dots$ are pairwise disjoint and
$\phi^{k}(W)\cap S=\emptyset$, for all $k\in\mathbb{Z}$, and

$X_{n}=S\cup(\cup_{k=n}^{\infty}\phi^{-k}(W))\ ,$

for all $n\in\mathbb{Z}_{+}$.

Proof. The second condition implies the second condition of Theorem 2.2 and
hence the implication (2) $\Rightarrow$ (1) is immediate. We will prove the
implication (1) $\Rightarrow$ (2).

We set $S=\cap_{n=0}^{\infty}X_{n}$. Clearly the set $S$ is closed and, by
($*$ ‣ 1), we have $\phi(S)\subseteq S$. We will prove that $\phi(S)=S$. We
suppose $\phi(S)\subsetneq S$. Since $\phi$ is surjective, there exists $x\in
X\setminus S$ such that $\phi(x)\in S$. Moreover, $\phi^{n}(x)\in S$ for all
$n\geq 1$. However, since $x\notin S$ there exists $n_{0}$ such that $x\notin
X_{n_{0}}$ and hence $\phi^{n_{0}}(x)\in X\setminus X_{0}$, by Theorem 2.2,
which is a contradiction since $S\cap(X\setminus X_{0})=\emptyset$.

By Theorem 2.2, $\phi(X_{n+1}\setminus X_{n+2})=X_{n}\setminus X_{n+1}$ for
all $n\in\mathbb{Z}_{+}$ and hence $\phi^{n}(X_{n}\setminus
X_{n+1})=X_{0}\setminus X_{1}$ or equivalently $X_{n}\setminus
X_{n+1}=\phi^{-n}(X_{0}\setminus X_{1})$ since $\phi$ is a homeomorphism.
Furthermore, the sets $\phi^{-1}(X_{0}\setminus
X_{1}),\phi^{-2}(X_{0}\setminus X_{1}),\dots$ are pairwise disjoint.

We set $W=X_{0}\setminus X_{1}$. Clearly, $\phi^{k}(W)\cap S=\emptyset$ for
all $k\in\mathbb{Z}$, since $\phi(S)=S$ and $\phi(W)\subseteq X\setminus
X_{0}$. Also, $X_{0}=S\cup(X_{0}\setminus X_{1})\cup(X_{1}\setminus
X_{2})\cup\dots$ and hence

$X_{0}=S\cup(\cup_{k=0}^{\infty}\phi^{-k}(W))\ .$

Finally, for all $n\in\mathbb{Z}_{+}$ we have that

$X_{0}=X_{n}\cup(\cup_{k=1}^{n}X_{k-1}\setminus
X_{k})=X_{n}\cup(\cup_{k=1}^{n}\phi^{-k+1}(W))=X_{n}\cup(\cup_{k=0}^{n-1}\phi^{-k}(W))\
,$

and so

$X_{n}=X_{0}\setminus(\cup_{k=0}^{n-1}\phi^{-k}(W))=S\cup(\cup_{k=n}^{\infty}\phi^{-k}(W))\
.$

$\Box$

In the following corollary the ideals with an approximate unit are
characterized.

###### Corollary 2.5

Let $\mathcal{I}\sim\\{X_{n}\\}_{n=0}^{\infty}$ be a non-zero ideal of
$C_{0}(X)\times_{\phi}\mathbb{Z}_{+}$. The following are equivalent:

  1. 1.

$\mathcal{I}$ has an approximate unit.

  2. 2.

$X_{n}=X_{n+1}$, for all $n\in\mathbb{Z}_{+}$, and $\phi(X\setminus
X_{0})=X\setminus X_{0}$.

Proof. (1) $\Rightarrow$ (2) is immediate from Theorem 2.1 and Corollary 2.3.

We show (2) $\Rightarrow$ (1). If $X_{n}=X_{n+1}$, by ($*$ ‣ 1), we have
$\phi(X_{0})\subseteq X_{0}$. Since $\phi(X\setminus X_{0})=X\setminus X_{0}$
and $\phi$ surjective we have $\phi(X_{0})=X_{0}$. Theorem 2.1 and Corollary
2.3 conclude the proof. $\Box$

###### Remark 2.6

If $\mathcal{I}\sim\\{X_{n}\\}_{n=0}^{\infty}$ is an ideal of
$C_{0}(X)\times_{\phi}\mathbb{Z}_{+}$ with a left (resp. right) approximate
unit, then it has a contractive left (resp. right) approximate unit. Moreover,
the semicrossed product $C_{0}(X)\times_{\phi}\mathbb{Z}_{+}$ has a
contractive approximate unit.

Let $B$ be a Banach space and $C$ be a subspace of $B$. The set of linear
functionals that vanish on a subspace $C$ of $B$ is called the _annihilator_
of $C$. A subspace $C$ of a Banach space $B$ is an _$M$ -ideal_ in $B$ if its
annihilator is the kernel of a projection $P$ on $B^{*}$ such that
$\|y\|=\|P(y)\|+\|y-P(y)\|$, for all $y$, where $B^{*}$ is the dual space of
$B$.

Effros and Ruan proved that the $M$-ideals in a unital operator algebra are
the closed two-sided ideals with an approximate unit, [5, Theorem 2.2].
Therefore, we obtain the following corollary about the $M$-ideals of a
semicrossed product.

###### Corollary 2.7

Let $\mathcal{I}\sim\\{X_{n}\\}_{n=0}^{\infty}$ be a non-zero ideal of
$C_{0}(X)\times_{\phi}\mathbb{Z}_{+}$, where $X$ is compact. The following are
equivalent:

  1. 1.

$\mathcal{I}$ is $M$-ideal.

  2. 2.

$\mathcal{I}$ has an approximate unit.

  3. 3.

$X_{n}=X_{n+1}$, for all $n\in\mathbb{Z}_{+}$, and $\phi(X\setminus
X_{0})=X\setminus X_{0}$.

## References

  * [1] G. Andreolas, M. Anoussis and C. Magiatis, _Topological radicals of semicrossed products_ , Serdica Math. J. 47 (2021), 81–92.
  * [2] G. Andreolas, M. Anoussis and C. Magiatis, _Compact multiplication operators on semicrossed products_ , Studia Math, to appear.
  * [3] K. R. Davidson, A. H. Fuller and E. T. A. Kakariadis, _Semicrossed products of operator algebras: a survey_ , New York J. Math. 24A (2018), 56–86.
  * [4] A. Donsig, A. Katavolos and A. Manoussos, _The Jacobson radical for analytic crossed products_ , J. Funct. Anal. 187 (2001), 129–145.
  * [5] E. G. Effros and Z-J. Ruan, _On non self-adjoint operator algebras_ , Proc. Amer. Math. Soc. 110 (1990), 915–-922.
  * [6] J. Peters, Semicrossed products of $C^{*}$-algebras, J. Funct. Anal. 59 (1984), 498–534.
  * [7] J. Peters, _The ideal structure of certain nonselfadjoint operator algebras_ , Trans. Amer. Math. Soc. 305 (1988), 333–352.