Gindraft=false ]}

# Mitigation Procedures to Rank Experts
through Information Retrieval Measures

Matthieu Vergne  Center for Information and Communication Technology, FBK-ICT
Via Sommarive, 18 I-38123 Povo, Trento, Italy
<EMAIL_ADDRESS>Doctoral School in Information and Communication Technology
Via Sommarive, 5 I-38123 Povo, Trento, Italy
<EMAIL_ADDRESS>
(2016-03-15)

###### Abstract

In order to find experts, different approaches build rankings of people,
assuming that they are ranked by level of expertise, and use typical
Information Retrieval (IR) measures to evaluate their effectiveness. However,
we figured out that expert rankings (i) tend to be partially ordered, (ii)
incomplete, and (iii) consequently provide more an order rather than absolute
ranks, which is not what usual IR measures exploit. To improve this state of
the art, we propose to revise the formalism used in IR to design proper
measures for comparing expert rankings. In this report, we investigate a first
step by providing mitigation procedures for the three issues, and we analyse
IR measures with the help of these procedures to identify interesting
revisions and remaining limitations. From this analysis, we see that most of
the measures can be exploited for this more generic context because of our
mitigation procedures. Moreover, measures based on precision and recall,
usually unable to consider the order of the ranked items, are of first
interest if we represent a ranking as a set of ordered pairs. Cumulative
measures, on the other hand, are specifically designed for considering the
order but suffer from a higher complexity, motivating the use of
precision/recall measures with the right representation.

Keywords— Expert Recommendation, Information Retrieval, Design Evaluation  To
the extent possible under law, Matthieu Vergne has waived all copyright and
related or neighboring rights to this technical report. For a more detailed
description of this waiving, visit:
https://creativecommons.org/publicdomain/zero/1.0/

##  1 Introduction

When dealing with complex situations, it is sometimes recommended to rely on
experts in related domains to obtain the best feedback for achieving our
goals. We work in Requirements Engineering (RE), in which people are asked to
identify and maintain the specifications of a system, which often requires to
have a broad knowledge about the system and its environment, thus motivating
the involvement of domain experts. To find experts, many systems have been
developed with various techniques based on past activities [Mockus and
Herbsleb, 2002], question-answer patterns [Zhang et al., 2007], language
models [Balog, 2008, Serdyukov and Hiemstra, 2008], as well as other and more
comprehensive approaches [McDonald and Ackerman, 2000, Tang et al., 2008].
Basically, they aim at building a ranking of people ordered by decreasing
level of expertise, allowing to recommend the top people as the most expert.
To evaluate these approaches, mainly inspired from document retrieval
techniques, it is common to use Information Retrieval (IR) measures, which
compare the rankings produced to some reference rankings.

While we designed our own approach [Vergne and Susi, 2014], we noticed that
building a reference ranking does not result naturally to a complete and
totally ordered ranking [Vergne, 2016]. Yet, these assumptions are common in
IR, leading these measures to be hardly applicable to expert rankings unless
we enforce them to be complete and totally ordered. This concern becomes
particularly relevant when aiming for Open Source communities, with people
freely joining and leaving the community, making infeasible for a human to
know exactly who is expert in what in a community of hundreds of members. This
observation leads us to consider that these reference rankings tend to be
_incomplete_ (all people are not ranked) and _partially ordered_ (several
people can be at the same rank). Both these assumptions make the _ranks_ of
the people a poor indicator to consider, because this rank can change by
introducing new people or by introducing additional orders (making it closer
to a total order). At the opposite, the _order_ before such modifications is
preserved once they are applied, making it a more reliable property to
consider.

In this report, we first propose to investigate procedures in Section 3 to
mitigate these new properties, not considered by usual IR measures. Then, in
Section 4, we analyse usual IR measures described in the literature by (i)
identifying how to make them applicable to our general context by applying the
previous procedures and (ii) what are their advantages and limitations.
Finally, in Section 5, we draw a summary of this analysis highlighting the
most interesting measures and the remaining issues for potential future works.

##  2 Problem: Comparing Expert Rankings

#### Context

We work on a recommender system aiming for recommending the most expert people
on some topics, which is achieved by building rankings of experts. To evaluate
our system, we need to compare the rankings produced between each other to
assess its stability, or compare them to some references to assess its
correctness. Consequently, we are looking for similarity (or distance)
measures to apply for comparing two rankings of experts A and B. Usually, IR
measures are used by relying on the similarity between expert retrieval and
document retrieval, but expert rankings have specific properties which need to
be considered to design proper comparison measures:

  * Incompleteness

Each ranking can be limited in size, especially human-made rankings, leading
to have only few people ranked compared to the whole set of available people.
Moreover, we can face situations where we should compare a ranking on a subset
of people to another ranking on a different (but usually overlapping) subset,
which means that we should be able to deal with rankings providing different
information.

  * Partial order

Some people might be on the same rank, which means that they have the same
expertise or, more generally, that we are not able to tell which one is more
expert than the other. This can happen even if the human who made the ranking
is _more_ knowledgeable on the topic than other people, while the latter would
provide a more complete order [Vergne, 2016]. So we cannot simply assume that
a reference ranking must be totally ordered.

  * Order consistency

Both these observations lead to a third one, which is that the _rank_ is not a
reliable property to count on. In case of incompleteness, this rank can vary
among rankings on different people, yet the order would be the same (e.g.
$a>b>c$ is compatible with $b>c>d$, although $b$ and $c$ have different
ranks). Also with partial order, the rank assigned to equal elements implies
to consider it like a conflict (e.g. although $a>b>c>d$ is compatible with
$a>(b,c)>d$, the equal rank of $b$ and $c$ leads to at least one being in
conflict with the former ranking). Moreover, expertise is highly domain-
specific [Ericsson, 2006], making the identification of the exact level of
expertise achieved by a person particularly hard. Most of the time, the people
ranked are just ordered based on who appears to be _more or less_ expert,
independently of their actual level, which again supports that the key
property to consider is not the _rank_ of a person, but the relative _order_
compared to other people.

#### Problem/RQ

We could design our own measure, but we want first to know if existing
measures could be used, which is the purpose of this report. Thus, we are
asking: is there existing similarity/distance measures that can be used to
compare expert rankings? Today, the answer is assumed to be affirmative
because expert finding systems are inspired from document retrieval systems
[Balog, 2012], so we use well known IR measures to validate them. The problem
is that, by analysing these measures and how they are used, we can see that
they do not fit the three requirements of incompleteness, partial order, and
order consistency.

##  3 Mitigation Procedures

In order to investigate properly existing measures, it is important to
consider the specific requirements of our expert rankings: the fact that they
represent an _order_ only, which can be _partial_ and _incomplete_. One way to
do is by finding measures which already consider such properties, which would
allow us to build entirely on existing measures, but as we will see in later
sections no measure properly deal with all 3 properties. Another way to
increase the chances of finding interesting measures is to consider additional
procedures which deal with these properties, so existing measures can be
enriched in order to properly satisfy each requirement. In this section, we
present a procedure for each of these 3 properties.

###  3.1 Order: Ranking vs. Set of Ordered Pairs

When comparing rankings, it is important to consider the order in which the
elements are ranked. Effort has been made to design measures which look at the
order of the ranked items, like CGk and derived measures (Section 4.9 and
after). If the measure is already designed for considering orders, then we can
use it as-is and deal with expert rankings as an ordered list of people, e.g.
$(a,b,c)$. So the _items_ considered by the measure are _people_ and the
measure directly manage the orders in its own way.

Yet, other measures not considering orders might be of interest, for instance
because they are simpler or because the measures considering the orders are
based on specific strategies. If we face such a situation, it is interesting
to investigate a way to use it such that we can consider the order. In our
case, we replace a ranking by its equivalent set of ordered pairs, e.g.
$a>b>c=\\{a>b,a>c,b>c\\}$, so an _item_ is an _ordered pair of people_ and the
measure does not need to consider any order, because they are directly part of
the items. In particular, two rankings based on the same people $r_{1}=a>b>c$
and $r_{2}=c>b>a$ lead to compare two sets $r_{1}=\\{a>b,a>c,b>c\\}$ and
$r_{2}=\\{c>b,c>a,b>a\\}$. Looking at the union of these two sets, like
precision (Section 4.1) which does not consider the order, results in an empty
set interpreted as a complete difference although they rank the same people,
which is what we expect when we look at how the two rankings are reversed.

However, this representation as a set of ordered pairs still has a potential
limitation: whether a pair is reversed or absent, the interpretation can be
the same. For instance, if we have :

  * •

$r_{1}=a>b>c=\\{a>b,a>c,b>c\\}$,

  * •

$r_{2}=c>b>a=\\{c>b,c>a,b>a\\}$,

  * •

$r_{3}=c>b=\\{c>b\\}$,

then we see that comparing $r_{1}$ to $r_{2}$ by looking at the shared pairs
($\emptyset$) gives the same result than comparing $r_{1}$ to $r_{3}$,
although $r_{2}$ is closer to $r_{1}$ because it has the same set of elements.
This problem can be solved by differentiating an absent item from an
“opposite” item, but none of the measures analysed in this report provide such
a feature.

###  3.2 Partial Order: Maximal Similarity

A ranking of experts can be partially ordered simply because one does not have
enough information to differentiate the expertise of two people. In such a
case, it is important that the measure used to compare two rankings does not
enforce arbitrary assumptions, which is what happen when it requires to have a
totally ordered ranking. For instance, a measure based on the rank of the item
might be tricked by a misalignment due to a partial order: $a>b>c>d$ is not
conflictual with $a>(b,c)>d$ while it is conflictual with $a>c>b>d$. Yet, both
cases are treated similarly: because $b$ and $c$ have the same rank (2, 2.5 or
3 depending on the method used) then at least one of them is “wrong”, which is
clearly not expected if the reference is the partial one. Of course, having
more than two elements at the same rank makes it even worse, potentially
making such a lack of information appearing as more conflictual than a single,
but explicit, conflict.

If we face a measure which considers the order of the ranking, we assume that
two rankings should be considered as different based on their _explicit_
misalignments, so when two items are ordered in a reversed way. To do so, we
consider Algorithm 1 which, from two partially ordered rankings, compares all
the possible totally ordered rankings and take the maximal similarity. In the
case of a distance, the initial value ($-\infty$) should be reversed
($\infty$) and we should take the minimal value instead of the maximal one.

0: $r_{1},r_{2}$: partially ordered rankings

0: $s$: similarity function for totally ordered rankings

0: $sim$: similarity value between $a$ and $b$

1: $R_{1}=buildPossibleTotalRankingsFor(r_{1})$

2: $R_{2}=buildPossibleTotalRankingsFor(r_{2})$

3: $sim=-\infty$

4: for each $(r^{\prime}_{1},r^{\prime}_{2})\in R_{1}\times R_{2}$ do

5: $sim=max(sim,s(r^{\prime}_{1},r^{\prime}_{2}))$

6: end for

Algorithm 1 Similarity computation for partial rankings.

The method $buildPossibleTotalRankingsFor(r)$ provides the set of totally
ordered rankings (1 person per rank) which are compatible with the partially
ordered ranking $r$. For instance, for a partial ranking $r=a>(b,c)>d$, the
function returns the two possible total rankings $\\{a>b>c>d,a>c>b>d\\}$. If
the ranking is already totally ordered, then only one ranking (the same) is
returned, and if both rankings are totally ordered, only the similarity
measure $s(r_{1},r_{2})$ is computed, so we go back to the original measure.
This is how we see that this algorithm properly generalizes measures assuming
total orders.

###  3.3 Incompleteness: Homogenization

Expert rankings are often limited in size: we can restrict them because we are
only interested in the top people, or we might face practical limitations,
like human-made rankings which cannot rank hundreds of people in a reliable
way (e.g. because of fatigue or simply lacks of information). Of course,
human-made rankings can be combined to complete each other, but conflicts can
occur [Vergne, 2016] and need to be mitigated. This is usually achieved by
assuming that a broad agreement is correct: if a majority of rankings say the
same, then it should be true. Assuming such equivalence is arguable, because
to be able to evaluate the expertise of someone, one needs himself to know
what to evaluate and how, what is correct, etc. which means to have enough
expertise too [Ericsson, 2006]. So if, among all the human-made rankings, only
few of them are made by actual experts, an agreement-based ranking might loose
correct information in favour of social agreements.

Consequently, if the available rankings do not allow to obtain a complete
ranking, we need to use a measure able to deal with it. With this in mind, we
might still be interested in measures which assume that the rankings are
complete, or at least that rank the same set of people. In such a case, we
need to adapt them to deal with incomplete rankings, which does not mean only
having one ranking subset of the other, because each ranking could be
incomplete, meaning that each ranks people that the other does not. For us, if
a person is missing for a ranking A, then no constraint is given, so any rank
is fine, in particular the rank given by the other ranking B. In other word,
independently of the rank given by B, this person does not intervene in the
comparison between the two rankings A and B. Consequently, we propose to
homogenize the two rankings by removing the people which are not in both
rankings.

This is consistent with the fact that we consider order rankings (Section
3.1): by reducing a ranking, we change the ranks of the lowest people, so
measures relying on the rank are impacted, but the order is preserved, so
measures considering the order only are not impacted. In particular, Algorithm
1, which deals with the partial order issue, provides the same result whether
or not this procedure is applied. An important point is that this procedure
does not generalizes to more than 2 rankings: it works only because we are
making a 1-to-1 comparison. In the case where we compare 3 rankings A, B, and
C, removing a person from A and B because C does not have it means that we
ignore any potential misalignment between A and B regarding this person.
Finally, some measures could assume that a missing/extra element should hurt
the similarity, in which case no homogenization should be considered to not
corrupt the results of the measure.

##  4 Analysis of IR Similarity Measures

[Balog, 2012] presents usual IR measures, established by the TREC community,
for evaluating expert finding methods, which are evaluated in exactly the same
way as document retrieval systems. From their point of view, this is a
reasonable choice, since “ _the quality of rankings can be estimated
independently of what we rank if quality measures for individual items are
alike_ ”. We confirmed from one of the authors that this sentence essentially
means it does not matter whether we rank documents or experts (or other
objects), we can use the same measures. Although we might agree on the
feasibility of applying the same measures, we don’t see clear evidences that
the measures cited are such well-fitted measures for a generic purpose.

In this section, we investigate a broad set of IR measures designed to
evaluate how well an IR system performs upon a query $q$ (or a set of queries
$Q$). The measures investigated are based on [Balog, 2012] and [Manning et
al., 2008], but also other resources taken from the Web about
IR111Introduction to IR: http://nlp.stanford.edu/IR-
book/html/htmledition/evaluation-in-information-retrieval-1.html222IR on
Wikipedia:
https://en.wikipedia.org/w/index.php?title=Information_retrieval&oldid=679995615333Cumulative
Gain and derived measures on Wikipedia:
https://en.wikipedia.org/w/index.php?title=Discounted_cumulative_gain&oldid=674619295.
For each measure, we provide the definition of the measure and analyse its use
for comparing expert rankings, considering also the procedures described in
Section 3 to complete them when necessary.

theory

#### Theory

application

#### Application to expert rankings A and B

###  4.1 Precision (P)

$\displaystyle P(q)=\frac{|\\{\text{relevant items for
$q$}\\}\cap\\{\text{retrieved items for $q$}\\}|}{|\\{\text{retrieved items
for $q$}\\}|}$

With this measure, we have two types of items: the retrieved ones and the
relevant ones. We can use the ranking A for the retrieved items and B for the
relevant ones, such that the intersection of the numerator retrieves the
common items between the two, making it a proper similarity measure. The issue
here is that this measure does not consider any order, what can be fixed by
using ordered pairs, as described in Section 3.1. However, because there is no
difference between absent and reversed pairs, we might be interested in
reducing as much as possible the differences caused by absent pairs,
especially the ones which _cannot_ be shared, which happens for people who are
not ranked by both rankings This can be reduced by restricting the rankings to
their shared people by homogenization, as described in Section 3.3.

For the remaining pairs, the partial ordering can still lead to absent pairs,
which are considered like reversed pairs. If this behaviour is not wanted, we
can reverse the logics by computing the maximal similarity with the procedure
described in Section 3.2. This is a matter of interpretation: we might be
interested in using an optimistic measure or a pessimistic one, leading to use
the procedure or not. A better measure might be one which differentiates
between explicit agreement (same pair), explicit disagreement (reversed pair),
and uncertain agreement (absent pair).

Finally, we can use this measure to compare a ranking to a reference or to
compare two rankings in a symmetric way. In the first case, the reference can
be naturally used as relevant items while the other ranking, the one
evaluated, can be naturally used as retrieved items. The difficulty comes with
the symmetric comparison: because the numerator considers only one ranking, it
implies to have a different normalization factor depending on which ranking is
used for what, making it potentially asymmetric. To make the measure properly
symmetric ($P(A,B)=P(B,A)$), we can ensure that both rankings provide the same
number of items, which can be achieved by using the three mitigation
procedures presented. If some of them are not used, we can still ensure the
symmetry by combining both uses in a symmetric computation, for instance with
$\frac{P(A,B)+P(B,A)}{2}$.

###  4.2 Recall (R)

$\displaystyle R(q)=\frac{|\\{\text{relevant items for
$q$}\\}\cap\\{\text{retrieved items for $q$}\\}|}{|\\{\text{relevant items for
$q$}\\}|}$

Recall is in many ways similar to precision, and we can apply the same
reasoning to obtain the same conclusions. The only difference is on the
normalization factor, which is the number of relevant items instead of the
number of retrieved items, but we can see that by reversing the two we obtain
the same formula than for precision ($P(A,B)=R(B,A)$). Rather than an
additional measure, we see that it offers a complementary measure that we
already used to make the precision measure symmetric. Indeed, by computing
$\frac{P(A,B)+P(B,A)}{2}$ we actually compute $\frac{P(A,B)+R(A,B)}{2}$ (and
also $\frac{R(A,B)+R(B,A)}{2}$). If precision or recall appear as good
similarity measures, one might consider this formula as a good way to combine
the strengths of both, especially if we don’t use all the mitigation
procedures.

###  4.3 Fall-out

$\displaystyle\text{fall-out}(q)=\frac{|\\{\text{non-relevant items for
$q$}\\}\cap\\{\text{retrieved items for $q$}\\}|}{|\\{\text{non-relevant items
for $q$}\\}|}$

Like recall offers a complementary perspective to precision by changing the
normalization factor, fall-out offers yet another complementary perspective by
considering the complement of the relevant items. In fact, we can use it in
the very same way than recall, but to compute a distance measure (as opposed
to similarity) because we look at the complement items. This is possible as
long as the items are ordered pairs: if the complement corresponds to the
_reversed pairs_ (not reversed _and_ absent pairs), then we deal with the very
same information. If the items were people only, then the complement would be
the non-ranked people (reversing the ranks would have no effect), which is
arguably justifiable because the ranked people are usually the _most relevant_
people, not the _relevant_ ones, so the complement would not be the non-
relevant people only. Anyway, this strategy is not interesting because it does
not allow to consider the order of the people, justifying that we use the
ordered pairs instead.

Actually, it might be interesting to investigate to which extent recall and
fall-out complement each other: although they are strictly opposed for
explicit pairs, the absent pairs (present only in the other ranking) are still
part of the computation. In particular, it could be interesting to see if we
can find an equivalence between the fall-out and a complemented recall, with
or without the maximal similarity procedure described in Section 3.2. We
consider such a deep analysis out of the scope of this report and limit our
conclusion to the following: due to their similarity, the fall-out measure
shows as much interest as precision and recall as a candidate measure to
compare rankings of experts.

###  4.4 F-Score

$\displaystyle
F_{\beta}(q)=\frac{(1+\beta^{2})(P(q).R(q))}{\beta^{2}P(q)+R(q)}$

$F_{0.5}$, $F_{1}$ and $F_{2}$ seems particularly used to, respectively, give
priority to precision, have a balanced measure, or give priority to recall.

It combines precision and recall and allow to have a symmetric measure with
$F_{1}$, which is interesting when no one is assumed to be a reference. If
another value is used for $\beta$, we might, like mentioned before, compute
both $F_{\beta}(A,B)$ and $F_{\beta}(B,A)$ and combine them in a balanced way
to have a symmetric measure. Consequently, this measure acts as another
solution to combine both precision and recall values, but with an additional
complexity which makes it harder to interpret.

###  4.5 Precision at $k$ (P@$k$)

$\displaystyle P@k(q)=\frac{|\\{\text{relevant items for
$q$}\\}\cap\\{\text{$k$ first retrieved items for $q$}\\}|}{k}$

At the opposite of the usual precision computation, P@$k$ aims at evaluating a
reduced set of retrieved items, not all of them. If, like precision, we use a
set of ordered pairs, we need to choose the right pairs to consider, which is
not straightforward. Indeed, the rank of a person is described, with this
representation, by several pairs, so we need to make a trade-off between the
number of pairs to consider for each person and the number of persons
represented by these pairs. Additionally, assuming that a good trade-off would
be identified for a given $k$, choosing $k$ might also suffer some
arbitrariness.

It seems to us more interesting to use this measure with people as items
(rather than pairs) in order to evaluate the correctness of the top of the
ranking. Still, no order is considered by this measure, but we can reduce this
issue if the ranking used to represent the _relevant_ items is also reduced to
the $k$ first items. In such a case, we actually use the same formula than for
precision, but with an additional reduction to $k$ items of both the compared
rankings, and evaluating it with an increasing $k$ allows to analyse the full
set, as shown for instance with AveP in Section 4.7. Nevertheless, the order
is only superficially considered and requires to analyse several values. It
also adds to the fact that this measure analyses the rank differences, not
only the orders, so we need to homogenize the rankings to avoid the
misalignments due to having different people ranked.

###  4.6 R-Precision

We compute P@$k$ with $k=|\text{relevant items for $q$}|$, which can be
different for each query $q$.

This is equivalent to the P@$k$ measure, but applied to a single $k$ value,
and with the necessity of knowing the exact number of relevant items, which is
different depending on the representation used (ranking or set of pairs). By
using a ranking, the number of relevant items is the size of the ranking
providing the relevant items, and with the homogenization, both rankings have
the same size and items leading to have always 1, which is useless. By using a
set of ordered pairs, the number of relevant items is the number of pairs, and
because of the homogenization, it is finally equivalent to compute the
precision as described in Section 4.1.

###  4.7 Average Precision (AveP)

$\displaystyle
AveP(q)=\frac{\sum_{k=1}^{n}\delta_{k}(q)P@k(q)}{|\text{relevant items for
$q$}|}$

with $n$ the number of retrieved items for the query $q$ and $\delta_{k}(q)$
the relevance of the $k$-th item retrieved (1 if it is relevant, 0 otherwise).
In the case where we rank the whole set of items, so all the relevant items
are for sure retrieved, the formula can be rewritten more like a typical
weighted average, thus justifying its naming:

$\displaystyle
AveP(q)=\frac{\sum_{k=1}^{n}\delta_{k}(q)P@k(q)}{\sum_{k=1}^{n}\delta_{k}(q)}$

As we see from the second formula, we compute an average of P@$k$ over the
consecutive $k$, which means that the order directly influences the final
result. This is an example of use of P@$k$ as described in Section 4.5, where
several values are computed for an analysis able to consider the order.
Indeed, although we consider the same number of P@$k$, by having relevant
items sooner in the ranking we consider higher values of P@$k$, leading to a
higher value of AveP.

###  4.8 Mean Average Precision (MAP)

$\displaystyle MAP(Q)=\frac{\sum_{q\in Q}AveP(q)}{|Q|}$

With this measure, we change the level of evaluation: while the previous
measures allowed us to compare two rankings designed for a specific query $q$,
this measure aims at aggregating the values over the whole set of queries $Q$
to evaluate the overall performance of the recommender system. This strategy
can be applied to any query-specific measure described before, and MAP simply
applies it to AveP. In other words, this is not an interesting measure for
identifying a comparison function between two rankings, but the general
strategy is interesting as an overall measure building on them, so it is not
completely out of scope and deserves to be mentioned.

###  4.9 Cumulative Gain (CGk)

$\displaystyle CG_{k}(q)=\sum_{i=1}^{k}rel_{i}(q)$

with $k$ the number of top ranks to evaluate and $rel_{i}(q)$ the relevance of
the $i$-th retrieved item for $q$. Like P@$k$, the aim is to evaluate a
reduced amount of items, but the value sums with lower ranks instead of being
computed rather independently.

From this measure, we consider a different kind of evaluation, where the
relevance of an item is provided by an external function $rel$ providing
higher values for more relevant items. With this flexibility, it is possible
to consider the orders even with a ranking representation by using a function
giving a higher relevance to items which should be closer to the top. For
instance, assuming that one ranking is used as a reference, we can assign a
relevance of $1$ to the last item of this ranking and $k$ to the top item, or
a more complex distribution to give more weight to some ranks. Then, CGk is
computed for the other ranking (the retrieved items) based on this reference.
If we need a symmetric value, we can imagine to compute an average of two
values: one computing CGk for the ranking A based on the ranking B, and
another for the ranking B based on A.

Although it might seem an interesting measure because the order can be
considered through the design of $rel$, the order still has a little impact,
because values are summed up on the sole criteria of the presence of the item.
It means that, for the same ranking but re-ordered, the same value is
computed, showing that it gives few information individually. The power of
this measure occurs when we compute it for different $k$ values, so we can
analyse the evolution of the value in a graph, showing for instance that we
gain more relevance when looking at long rankings, so the most relevant items
are actually ranked quite low. Analysing a graph is more complex than
analysing a single value, so using this measure requires a greater effort to
draw conclusions, which is one of the critics we could made. Additionally,
this measure lacks the required normalization to reliably compare values for
one ranking to values for another.

Because this measure relies on the $k$ first items, it means that we should be
able to identify what comes “first”, so using the representation of a set of
ordered pairs, which is not ordered, does not seem adapted. However, if we are
computing only the value for $k$ (i.e. the whole set) then all pairs should be
considered and no problem of identifying the “first” pairs occur. In this
case, we can for example use this representation with a binary function: 1 if
the pair is in the right order, 0 otherwise. Actually, it can be better by
assigning a high relevance to pairs in the right order, a low relevance to
reversed pairs, and a middle relevance to absent pairs, making the measure
able to make the difference between absent and reversed pairs. This is an
advantage of this measure compared to the previous ones, based on precision
and recall, which cannot make this difference. However, we still face the
limitation of the normalization, which hurts our ability to compare the values
computed for different rankings.

###  4.10 Discounted CG (DCGk)

$\displaystyle
DCG_{k}(q)=rel_{1}(q)+\sum_{i=2}^{k}\frac{rel_{i}(q)}{log_{2}(i)}$

Like CGk, we sum up the relevance values, but this time by applying a
logarithmic weight to decrease the global relevance if highly relevant items
appear too late in the ranking. We might take the freedom of slightly changing
the logarithmic value in order to rewrite the formula in a simpler way:

$\displaystyle
DCG^{\prime}_{k}(q)=\sum_{i=1}^{k}\frac{rel_{i}(q)}{log_{2}(i+1)}$

With such a shape, we see better how it simply enriches the terms of the sum
with a weighting factor compared to CGk. Additionally, it also allows to make
the link with another formula used for DCGk:

$\displaystyle DCG_{k}(q)=\sum_{i=1}^{k}\frac{2^{rel_{i}(q)}-1}{log_{2}(i+1)}$

which prioritizes the relevance value upon the weighting factor.

At the opposite of CGk, DCGk directly involves the index of the item in the
computation, so we need to map each item to an index. This addition makes it
inapplicable to the representation using sets of ordered pairs because these
sets are not ordered (unless we assign the same index to all the pairs, in
which case it is equivalent to CGk). Consequently, we can only use the ranking
representation, and the $rel$ function should be designed carefully based on a
reference ranking (A or B) before to compute the DCGk value of the other
ranking (resp. B or A). In this context, DCGk only revises the weighting
strategy, so it does not provide anything more than what CGk already provides
($rel$ can be directly designed to compute the same values). In other words,
we can summarize DCGk (and its variants) as a specialization of CGk applied to
rankings with a specific class of $rel$ functions. In particular, DCGk does
not introduce any normalization allowing to compare values computed for
different rankings.

###  4.11 Normalized DCG (NDCGk)

$\displaystyle NDCG_{k}(q)=\frac{DCG_{k}(q)}{IDCG_{k}(q)}$

where IDCGk is the ideal DCGk, meaning the value of DCGk when the items are
re-ordered to maximize it (i.e. sorted by decreasing relevance).

This measure solves the main issue of the previous ones: the normalization
applied allows to compare values computed for different rankings to see which
one is better. As such, what is interesting with this measure is, like for
MAP, the generic strategy it involves: divide the value of the “sub-measure”
(here DCGk) by the best value achievable with this “sub-measure”. This
integral dependency to the “sub-measure” allows to use it even if we use a
different one. In particular, we might be interested in using CGk on the sets
of ordered pairs, leading to have a normalized version following the same
strategy:

$\displaystyle NCG_{k}(q)=\frac{CG_{k}(q)}{ICG_{k}(q)}$

where ICGk is the ideal CGk, meaning the value of CGk when the pairs retrieved
are the same than the reference, or said another way the value of CGk when we
evaluate the ranking A while taking A also as the reference (or evaluating B
based on B).

##  5 Discussion

As we saw, many usual IR measures described in Section 4 are interesting for
comparing two rankings of experts, whether we are looking for symmetric or
reference-based comparisons. However, due to the specific properties of our
rankings, the usefulness of these measures cannot be highlighted without using
the procedures proposed in Section 3. In particular, two main types of
measures show up: the measures based on precision and recall, and the
cumulative measures.

Precision and recall (sections 4.1 and 4.2), and more extensively with fall-
out (Section 4.3), show an interesting complementarity worth to exploit. The
fact that they do not consider any order makes them particularly suited for a
representation using sets of ordered pairs, described in Section 3.1. Measures
derived from them show additional interests: the F-score provides an
interesting combination strategy (Section 4.4), and if P@$k$ has many issues
(Section 4.5), particular uses appear to be interesting, like r-precision for
ordered pairs (Section 4.6) and AveP for rankings (Section 4.7).

On the other hand, Cumulative measures like CGk (Section 4.9) use an external
relevance function $rel$ which makes them flexible enough to adapt to both
representations, rankings as well as sets of ordered pairs. While one of the
main issues of CGk is its lack of normalization, Section 4.11 shows how to fix
it, and Section 4.10 shows interesting functions to use, although they are
more adapted to rankings than sets of ordered pairs. However, another main
issue of these measures is the need to compute a set of them (for different
$k$ values) rather than a single one, at the opposite of precision/recall
measures.

Additionally, while all these measures are worth investigating for our
purpose, i.e. to compare two rankings of experts, we might argue that it is
not enough to evaluate a complete approach, which is the ultimate goal of an
evaluation. Indeed, each ranking corresponds to a specific query $q$, while
evaluating an approach implies to analyse it over a full set of queries $Q$.
Nevertheless, usual IR measures already consider such facilities, like for MAP
(Section 4.8) which uses a generic method applicable to any of the described
measures (simple average over the set of queries).

Of course, this report remains short in its analysis, its aim being to
identify measures worth to investigate rather than making a deep investigation
of each of them. With the procedures proposed in Section 3, we were able to
highlight potential uses of usual measures for a more general context than
what they were designed for. In particular, by showing how they appear to be
equivalent or complementary, and which issues might be faced when using them,
this report provides a comprehensive basis for future works.

##  6 Conclusion

In this report, we investigated usual IR measures to evaluate their
applicability to a more generic context than what they were designed for. In
particular, we highlighted the need to consider rankings which are
_incomplete_ and _partially ordered_ , leading to focus on the relative
_order_ of the ranked items rather than their absolute ranks. We proposed
procedures to adapt IR measures, in order to consider these properties, and
shown that most IR measures remain applicable with their help. In particular,
by representing a ranking as a set of ordered pairs, measures based on
precision and recall appear as the most interesting measures. Although
cumulative measures (i.e. CGk and derived) have been designed precisely for
advanced analysis, the simplicity of computation and interpretation of the
precision/recall measures enforce this regain of interest.

This report, although it considers a comprehensive list of IR measures,
remains superficial in its analysis by focusing on specific adaptations and
potential uses. An interesting future work would be a deeper investigation on
the consequences of the procedures proposed to adapt these measures,
identifying the differences and equivalences, with some highlights on the
computation performances. In particular, it would be interesting to know how
usual measures can be rewritten to consider sets of ordered pairs naturally,
rather than using additional procedures like homogenization and maximal
similarity. For instance, by replacing the _rank_ (or index) of an item in
current formalizations by the _number of ordered pairs_ which rank this item
lower than others, thus relying only on the relative order rather than an
absolute rank, it might be possible to generalize these measures and make them
able to deal with partial orders and incompleteness more naturally.

## References

  * [Balog, 2008] Balog, K. (2008).  People search in the enterprise.  PhD thesis, University of Amsterdam, http://krisztianbalog.com/publications/phd-thesis/.
  * [Balog, 2012] Balog, K. (2012).  Expertise Retrieval.  Foundations and Trends® in Information Retrieval, 6(2-3):127–256, ISSN: 1554-0669, 1554-0677, DOI: 10.1561/1500000024, http://www.nowpublishers.com/article/Details/INR-024.
  * [Ericsson, 2006] Ericsson, K. A., editor (2006).  The Cambridge Handbook of Expertise and Expert Performance.  Cambridge University Press, Cambridge ; New York, ISBN: 978-0-521-84097-2 978-0-521-60081-1, http://www.cambridge.org/us/academic/subjects/psychology/cognition/cambridge-handbook-expertise-and-expert-performance.
  * [Manning et al., 2008] Manning, C. D., Raghavan, P., and Schütze, H. (2008).  Introduction to information retrieval.  Cambridge University Press, New York, ISBN: 978-0-521-86571-5.
  * [McDonald and Ackerman, 2000] McDonald, D. W. and Ackerman, M. S. (2000).  Expertise recommender: a flexible recommendation system and architecture.  In Proceedings of the 2000 ACM conference on Computer supported cooperative work, pages 231--240. ACM Press, ISBN: 978-1-58113-222-9, DOI: 10.1145/358916.358994, http://portal.acm.org/citation.cfm?doid=358916.358994.
  * [Mockus and Herbsleb, 2002] Mockus, A. and Herbsleb, J. D. (2002).  Expertise browser: a quantitative approach to identifying expertise.  In Proceedings of the 24th International Conference on Software Engineering, ICSE ’02, pages 503--512, New York, NY, USA. ACM, ISBN: 1-58113-472-X, DOI: 10.1145/581339.581401, http://doi.acm.org/10.1145/581339.581401.
  * [Serdyukov and Hiemstra, 2008] Serdyukov, P. and Hiemstra, D. (2008).  Modeling Documents As Mixtures of Persons for Expert Finding.  In Proceedings of the IR Research, 30th European Conference on Advances in Information Retrieval, ECIR’08, pages 309--320, Berlin, Heidelberg. Springer-Verlag, ISBN: 3-540-78645-7 978-3-540-78645-0, http://dl.acm.org/citation.cfm?id=1793274.1793313.
  * [Tang et al., 2008] Tang, J., Zhang, J., Yao, L., Li, J., Zhang, L., and Su, Z. (2008).  ArnetMiner: Extraction and Mining of Academic Social Networks.  In Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’08, pages 990--998, New York, NY, USA. ACM, ISBN: 978-1-60558-193-4, DOI: 10.1145/1401890.1402008, http://doi.acm.org/10.1145/1401890.1402008.
  * [Vergne, 2016] Vergne, M. (2016).  Gold Standard for Expert Ranking: A Survey on the XWiki Dataset.  Technical Report arXiv:1603.03809 [cs.SE], http://arxiv.org/abs/1603.03809.
  * [Vergne and Susi, 2014] Vergne, M. and Susi, A. (2014).  Expert Finding Using Markov Networks in Open Source Communities.  In Jarke, M., Mylopoulos, J., Quix, C., Rolland, C., Manolopoulos, Y., Mouratidis, H., and Horkoff, J., editors, Advanced Information Systems Engineering, number 8484 in Lecture Notes in Computer Science, pages 196--210. Springer International Publishing, ISBN: 978-3-319-07880-9 978-3-319-07881-6, http://link.springer.com/chapter/10.1007/978-3-319-07881-6_14.
  * [Zhang et al., 2007] Zhang, J., Ackerman, M. S., and Adamic, L. (2007).  Expertise networks in online communities: structure and algorithms.  In Proceedings of the 16th international conference on World Wide Web, WWW ’07, pages 221--230, New York, NY, USA. ACM, ISBN: 978-1-59593-654-7, DOI: 10.1145/1242572.1242603, http://doi.acm.org/10.1145/1242572.1242603.


††thanks: Contributed equally to this article.
Corresponding author<EMAIL_ADDRESS>Contributed equally to this
article.
Corresponding author<EMAIL_ADDRESS>
# Realisation of versatile and effective quantum metrology using a single
bosonic mode

Xiaozhou Pan  Tanjung Krisnanda  Andrea Duina  Centre for Quantum
Technologies, National University of Singapore, Singapore 117543, Singapore
Kimin Park  Department of Optics, Palacky University, 77146 Olomouc, Czech
Republic  Pengtao Song  Centre for Quantum Technologies, National University
of Singapore, Singapore 117543, Singapore  Clara Yun Fontaine  Centre for
Quantum Technologies, National University of Singapore, Singapore 117543,
Singapore  Adrian Copetudo  Centre for Quantum Technologies, National
University of Singapore, Singapore 117543, Singapore  Radim Filip  Department
of Optics, Palacky University, 77146 Olomouc, Czech Republic  Yvonne Y. Gao
<EMAIL_ADDRESS>Centre for Quantum Technologies, National University of
Singapore, Singapore 117543, Singapore  Department of Physics, National
University of Singapore, Singapore 117542, Singapore

###### Abstract

Quantum metrology offers superior measurement precision compared to classical
counterparts. Traditionally, achieving this enhancement involves employing
multi-particle entangled quantum states, necessitating state preparation and
detection schemes that are practically challenging. In this article, we
present a versatile and on-demand protocol for deterministic parameter
estimation that leverages two state-transfer operations on a single bosonic
mode. Specifically, we demonstrate this protocol in the context of phase and
amplitude estimation using the superposition of coherent states in the bosonic
circuit quantum electrodynamics (cQED) platform. With low photon numbers of up
to 1.76, we achieve quantum-enhanced precision approaching the Heisenberg
scaling, reaching a metrological gain of $7.5(6)$ dB and $9.3(5)$ dB
respectively for phase and amplitude estimation. We show that the gain or
sensitivity range can be further enhanced on the fly by tailoring the input
states based on specific system constraints. Our protocol can be adapted to
further optimize the desired figures of merit using tailored quantum states or
operations, not only for bosonic cQED hardware but also readily extensible to
other continuous-variable platforms. The realisation of this versatile and
effective scheme affords a promising path towards practical quantum-enhanced
sensing.

Quantum metrology utilizes optimally chosen sets of quantum states and
measurement schemes to attain measurement precision beyond what is possible
with classical states [1]. This quantum-enhanced precision appeals to many
disciplines. For example, it has already applications for sensing magnetic
fields [2, 3], electric fields [4, 5], and displacements [5, 6], as well as
improving clock stability [7, 8] and quantum imaging [9, 10]. It is also
expected to enhance gravitational wave detection in LIGO [11, 12] and the
search for dark matter [13].

One of the earliest and most canonical examples is quantum-enhanced phase
estimation in a two-mode interferometer [14, 15]. The use of coherent states
(CS) defines the standard quantum limit (SQL) with precision
$\Delta\theta=1/\sqrt{N}$, where $\Delta\theta$ is the standard deviation of
the phase and $N$ is the number of particles or excitations. Using
entanglement as the resource state in this configuration, the precision scales
as $\Delta\theta=1/N$ (known as the Heisenberg limit), which affords a
$\sqrt{N}$ improvement over the SQL. This enhanced precision has been
demonstrated in many experiments using entangled multipartite systems [16, 17,
18, 19, 20]. Practically, however, both the preparation of these non-local
states and the implementation of correlated or conditional measurements can be
challenging. Furthermore, preparing such quantum states often demands post-
selection or heralding, which undermines the quantum enhancement [21, 22] over
deterministically created coherent states.

Figure 1:  Universal quantum metrology with a single bosonic mode. The
approach consists of three fundamental steps: preparation of input state
$|\psi\rangle$; distortion from the to-be-estimated process, potentially
induced by stray fields, which effectively encode the parameter of interest
$\beta$ into the state $|\psi(\beta)\rangle$; and information retrieval by
projection onto the initial state, depicted by the shaded region.

In light of these challenges, there has been growing interest in quantum
metrology using non-entangled states [23]. For instance, Fock states are used
to perform phase-insensitive measurements with enhanced precision due to their
nonclassical features [24, 25]. Quantum-enhanced phase estimation has also
been shown using displaced Fock states or their superposition [26, 27, 28].
This handily bypasses the demanding non-local operations, which generally
suppress the quantum advantage under even slight decoherence. However, the
existing single-mode schemes are system-specific, as they rely on highly-
specialised input states or quantum gates.

Figure 2: Parameter estimation procedure with bosonic cQED. (a) Implementation
of the protocol in cQED. A cavity stores a bosonic mode, coupled to a transmon
qubit. The readout resonator provides single-shot measurements of the qubit
state. The protocol includes a state-transfer operation $\hat{O}_{1}$, a
process $\hat{U}(\beta)$, and a projection operation involving a state-
transfer operation $\hat{O}_{2}$, qubit measurement $p_{g}$, and cavity
measurement $p_{0|g}$. (b) Illustration of phase estimation using SCS. The
shaded area visualises the overlap between the distorted and initial state
$|\langle\psi|\psi(\theta)\rangle|^{2}$. (c) The FI for SCS (solid blue) and
CS (solid purple), calculated using Eq. (1), where $p$ is from the polynomial
fit (dotted curves) of the experimental $p_{g0}(\theta)$ (markers) in the
inset. The shaded area with the corresponding color indicates the standard
deviation error from bootstrapping. Bandwidth is defined as the range on which
the FI of SCS is above the maximum FI ($\rm{FI}_{max}$) of CS. The grey area
represents the region where we have a quantum advantage compared to CS and
$\tilde{\rm{FI}}$ is its corresponding average FI. Dashed curves show
simulation. (d) The $\rm{FI}_{max}$ for SCS (blue) and CS (purple) are plotted
as markers for states with different photon numbers $\overline{n}$. The errors
are standard deviation from bootstrapping. Dashed curves are simulation and
dotted curves are fitting: $3.92(9)\overline{n}^{1.71(1)}+0.15(4)$ for SCS and
$2.62(6)\overline{n}^{0.99(1)}+0.00(4)$ for CS. The solid pink curve is the
ideal QFI of SCS.

Here, we demonstrate a versatile and on-demand protocol for deterministic
parameter estimation using a single bosonic mode. With standard bosonic cQED
hardware, we show quantum-enhanced performance for both phase and amplitude
estimation, achieving state-of-the-art metrological gain in the low photon
number regime. We present a comprehensive analysis of key metrological figures
of merit, including the sensitivity bandwidth, average sensitivity within the
bandwidth, and adjustment of the position of maximum sensitivity. Importantly,
we show that our implementation allows these key figures to be optimally
traded on the fly to best suit the intended metrological goal by simply
varying a few programmable pulse parameters in the protocol. Our strategy is
universal across different physical systems and can be applied to other
physical implementations of bosonic modes, such as optical resonators [29],
trapped ions [30], neutral atoms [31], and mechanical resonators [32].

Figure 3: Comprehensive analysis for phase estimation and quantum-enhanced
precision. (a) The average FI ($\tilde{\rm{FI}}$) of SCS within the bandwidth
and $\theta$ offset at which FI is maximum vs $\overline{n}$. The dotted curve
is a fit: $2.7(3)\overline{n}^{2.0(1)}+0.3(1)$. (b) The same protocol as in
Fig. 2(c) with initial states prepared with offset in $\theta$. As the
protocol is symmetrical with respect to $\theta$, maximum FI at $\theta=0$ can
be obtained by a positive/negative offset of about $0.2-0.25$ rad. (c) The
phase precision vs $\overline{n}$ for both SCS (blue circles) and CS (purple
diamonds). The solid lines are different theoretical Heisenberg limits and
SQL. The dotted lines are linear fit:
$\log(\Delta\theta)=-0.915(2)\log(\overline{n})-0.987(1)$ for SCS and
$\log(\Delta\theta)=-0.464(1)\log(\overline{n})-0.508(1)$ for CS. The highest
metrological gain for SCS compared to CS with the same photon number is
$7.5(6)$ dB.

Conceptually, our methodology is akin to shining lights on an object under the
influence of some external perturbations and comparing the resulting shadow
with the original unperturbed object, as illustrated in Fig. 1(a). The shadow
represents the projection of the distorted state
$|\psi(\beta)\rangle=\hat{U}(\beta)|\psi\rangle$ onto the initial state
$|\psi\rangle$, written as $p(\beta)=|\langle\psi|\psi(\beta)\rangle|^{2}$
where $\beta$ is the to-be-estimated parameter,
$\hat{U}(\beta)=\exp(i\hat{B}\beta)$, and we shall refer to $\hat{B}$ as the
_process generator_. The measurement outcome $p(\beta)$ provides an optimal
precision for estimating $\beta$ [33], as indicated by the Fisher information
(FI)

$F(\beta)=\frac{1}{p(\beta)(1-p(\beta))}\left(\frac{\partial
p(\beta)}{\partial\beta}\right)^{2},$ (1)

which provides a lower asymptotic bound on the achievable precision, i.e.,
$\Delta\beta\geq 1/\sqrt{F}$, known as the Cramér-Rao bound [34]. The
optimality transpires as the FI saturates its maximum value given by the
quantum Fisher information (QFI) $F_{q}=4(\Delta\hat{B})^{2}$, where
$(\Delta\hat{B})^{2}=\langle\psi|\hat{B}^{2}|\psi\rangle-\langle\psi|\hat{B}|\psi\rangle^{2}$.
In addition to having an optimal measurement scheme, it is also important to
choose the input states strategically for optimal metrological performance,
i.e., those with high variance $(\Delta\hat{B})^{2}$.

We implement this generalised metrology scheme on standard cQED hardware as
depicted in Fig. 2(a). Initially, a desired quantum state is prepared with a
state-transfer unitary $\hat{O}_{1}$, i.e.,
$|g\psi\rangle=\hat{O}_{1}|g0\rangle$. After the system undergoes a process
$\hat{U}(\beta)$, a projection measurement is performed with a unitary
$\hat{O}_{2}$ and qubit-cavity measurements. The operation $\hat{O}_{2}$ can
be done with _any_ state-transfer unitary such that
$|g0\rangle=\hat{O}_{2}|g\psi\rangle$. Finally, standard measurements are
performed to obtain the probabilities of the qubit in the ground state $p_{g}$
and cavity in vacuum $p_{0|g}$, which is achieved by driving the qubit with a
vacuum-selective $\pi$-pulse and measuring the qubit in the excited state. The
projection yields the probability
$p_{g0}=|\langle\psi|\hat{U}(\beta)|\psi\rangle|^{2}$ [33]. An important
advantage of our protocol is that any two operations $\hat{O}_{1}$ and
$\hat{O}_{2}$ realising the required state transfers are sufficient. This
minimalistic approach makes it highly adaptable to a variety of hardware
parameters and readily extensible to different physical systems with their
preferred native gates. For instance, in cQED platforms featuring small qubit-
oscillator dispersive coupling, $\hat{O}_{1,2}$ can be realised using echoed
conditional displacement (ECD) gates [35, 36]; whereas devices with more
significant dispersive coupling can leverage on gradient ascent pulse
engineering (GRAPE) [37, 38, 39] or selective number-dependent arbitrary phase
gates [40]. Beyond cQED, the state transfer operations can be achieved through
coherent coupling of the internal-motional states of trapped ions/atoms [41,
27], ECD gates in an atom-light system in cavity-QED [42], or swap gates in
qubit-mechanical resonator system in quantum acoustics [43].

We first demonstrate this universal protocol in the context of phase
estimation in cQED. The target process is enacted via the dispersive qubit-
cavity coupling with Hamiltonian
$H_{\text{int}}/\hbar=(\chi/2)\hat{\sigma}_{z}\hat{a}^{\dagger}\hat{a}$, where
$\chi$ is the coupling strength, $\hat{\sigma}_{z}$ the $z$-Pauli matrix, and
$\hat{a}$($\hat{a}^{\dagger}$) is the annihilation (creation) operator of the
bosonic cavity. For a qubit in the ground state $|g\rangle$, the dynamics
realises a unitary for the cavity
$\hat{U}(\theta)=\exp(i\hat{a}^{\dagger}\hat{a}\theta)$, with $\theta=\chi
t/2$. As a result, the process generator is given by
$\hat{n}=\hat{a}^{\dagger}\hat{a}$, which means useful states here are those
with high variance in photon number $(\Delta\hat{n})^{2}$.

Theoretically, we investigate states with high photon number variances such as
(listed from lowest to highest) superposition of Fock states, superposition of
coherent states (SCS), and squeezed vacuum. Experimentally, we find SCS to be
the most robust against device imperfections such as self-Kerr and offer the
highest sensitivity in our system [33]. Therefore, we shall primarily focus on
SCS, given by $\mathcal{N}_{\alpha}(|0\rangle+|\alpha\rangle)$ (in short
SCS($\alpha$)), with the normalisation factor $\mathcal{N}_{\alpha}$, to
demonstrate the efficacy of our strategy. The intuition behind the performance
of SCS for phase estimation is understood by visualizing the projection
$p_{g0}=|\langle\psi|\hat{U}(\theta)|\psi\rangle|^{2}$ in phase space, see
Fig. 2(b). With this illustration in mind, we note two important observations.
Firstly, as $\theta$ increases, the probability $p_{g0}(\theta)$ changes from
$1$ to a constant value, where the fringes do not overlap and the probability
is insensitive to a further increase in $\theta$. Secondly, the use of higher
$\alpha$ (further right position of the right blob in Fig. 2(b)) will reduce
the overlap more sharply at lower $\theta$, making the detection more
sensitive to $\theta$ at the cost of a smaller range of sensitivity.

Experimentally, we chose to implement the state transfer operations
$\hat{O}_{1,2}$ using GRAPE, which is a convenient and widely used tool in
cQED [37, 38, 39, 28]. However, this choice is not unique, e.g., in the weak
dispersive regime, one may opt to use ECD gates to create SCS [44]. By
executing the gate sequence in Fig. 2(a), we directly obtain the probability
$p_{g0}(\theta)$. An exemplary plot comparing the use of SCS($\alpha=2$) and
CS with the same average photon number
$\overline{n}=\langle\hat{a}^{\dagger}\hat{a}\rangle$, is presented in Fig.
2(c) inset. Each single-shot measurement is one bit of information and we
perform $1000$ repetitions for each point to obtain the statistics. The
experimental data shows good agreement with simulations (corresponding dashed
curves), which take into account decoherences and measurement imperfections
[33]. It is apparent from the probability that the SCS is more sensitive to
the change in $\theta$ than the CS.

To quantitatively examine the performance of our implementation, we compute
the FI in Eq. (1), where the corresponding $p$ is a smooth function obtained
from the polynomial fit of $p_{g0}(\theta)$. The resulting FI for SCS and CS
are plotted in Fig. 2(c) as solid blue and purple curves, respectively. As
expected, the maximum FI of SCS beats that of the CS at the cost of the
sensitivity range. Furthermore, figure 2(d) shows the maximum FI for SCS (blue
circles) and CS (purple diamonds) for states with different photon numbers. We
note that decoherence and experimental imperfections hinder our results from
reaching the ideal QFI of SCS (pink solid curve), as by simulations
(corresponding dashed curves) with real device parameters. However, despite
that, second-order polynomial fits (corresponding dotted curves) show that the
$\text{FI}_{\text{max}}$ of SCS is nearly quadratic, whereas that of the CS is
linear, which indicates a clear $\sqrt{\overline{n}}$ quantum enhancement in
the estimation precision.

To holistically evaluate the performance of a quantum metrology scheme,
several other important figures of merit must also be considered in
conjunction with the maximum FI. Specifically, these include the parameter
range in which we achieve quantum-enhanced precision, the corresponding
average sensitivity, and the capability to shift the maximum sensitivity point
to origin $\theta=0$ which is crucial for sensing applications. We define the
bandwidth as the range of $\theta$ in which the FI of SCS is above the maximum
FI of the CS, $\tilde{\text{FI}}$ as the average FI within the bandwidth, and
$\theta$ offset where the FI is maximum. The bandwidth for SCS up to
$\overline{n}=1.76$ is nearly constant at approximately 0.5 rad, which sets
the limit on the allowed phase fluctuations in the system. The
$\tilde{\text{FI}}$ against photon number $\overline{n}$ in Fig. 3(a) shows a
quadratic trend, suggesting that working within the bandwidth still affords
favourable precision scaling. The offset in $\theta$ is close to the position
of the maximum slope $|\partial p/\partial\theta|$ in the measured probability
[33] and decreases for SCS with larger $\alpha$, as shown in Fig. 3(a). With
our versatile protocol, this offset is a fully programmable parameter and can
be adjusted by simply preparing input states with an initial phase offset
(Fig. 3(b)) to ensure that we can operate around the maximum sensitivity
point.

Figure 4: On-demand optimization and amplitude estimation. (a) FI of SCS with
different weights and fixed photon number $\overline{n}=1$. The $w$ in
SCS${}_{w}(\alpha)$ denotes the weight in the $|\alpha\rangle$ component,
i.e., $\mathcal{N}^{\prime}(\sqrt{1-w}|0\rangle+\sqrt{w}|\alpha\rangle)$,
$\mathcal{N}^{\prime}$ represents the normalization coefficient. The
corresponding shaded area indicates the standard deviation from bootstrapping.
(b) Similar to (a) but for SCS with different weights and fixed $\alpha=1.6$.
(c) FI for amplitude estimation using CS (solid purple), SCS-v for amplitude
in the vertical direction (solid blue), and SCS-h for amplitude in the
horizontal direction (solid grey). The corresponding dashed curves indicate
simulation. (d) The maximum FI for SCS-v (blue) and CS (purple) vs photon
number $\overline{n}$. Dashed curves are simulation and dotted curves are
fitting: $4.9(2)n^{1.06(2)}+2.2(1)$ for SCS-v and $0.411(1)n^{0.00(3)}$ \+
2.165(1) for CS. The solid pink curve is the ideal QFI of SCS-v.

In addition to the FI, which only sets the lower bound to the precision
according to the Cramér-Rao bound, we now present a benchmark for direct
calculation of the phase precision, $\Delta\theta$. It is computed as
$\Delta\theta=\Delta p/|\partial p/\partial\theta|$, where $\partial
p/\partial\theta$ is obtained from the polynomial fit as in Fig. 2(c) inset
and $\Delta p$ is the standard deviation of the single shot measurement with
$1000$ repetitions. The standard deviation for SCS (blue circles) and CS
(purple diamonds) are plotted in Fig. 3(c), which respectively beat the
precision limit $\Delta\theta=1/\overline{n}$ and the SQL
($\Delta\theta=1/\sqrt{\overline{n}}$) commonly used in optical [14, 15] or
atomic [45] interferometers. We also note that the experimentally obtained
precision for SCS, $\Delta\theta_{\mathrm{SCS}}$, surpasses the ideal limit
used for the superposition of Fock states ($\Delta\theta=1/2\overline{n}$)
[27, 28]. This is because the QFI of SCS and CS in our optimised strategy
gives $\Delta\theta=1/\sqrt{4\overline{n}+4\overline{n}^{2}}$ and
$\Delta\theta=1/\sqrt{4\overline{n}}$, respectively [33] to afford superior
performance by design. Furthermore, the linear fit of the SCS shows that
$\Delta\theta_{\text{SCS}}$ approaches the Heisenberg limit scaling. To
benchmark the quantum enhancement of our implementation, we compute the
metrological gain of
$20\log_{10}(\Delta\theta_{\text{CS}}/\Delta\theta_{\text{SCS}})=7.5(6)$ dB
with a low photon number of 1.76. Here, the uncertainty is one standard
deviation obtained from bootstrapping and stabilises at $\sim 600$ measurement
repetitions [33].

Moreover, the gain or bandwidth can be enhanced on demand by optimizing the
initial states with respect to system constraints. In typical hardware
implementations, a device may allow access to a limited maximum photon number
$\overline{n}$ or SCS amplitude $\alpha$, which may be due to photon loss,
self-Kerr effect, etc. Depending on the specific hardware limitations and the
figures of merit, we can optimise the metrological performance by simply
varying the superposition weights of the SCS. Physically, this is achieved by
adjusting the qubit and cavity drives for $\hat{O}_{1,2}$, without any changes
to the device or system dynamics. For the case where the constraint is the
photon number, e.g., $\overline{n}=1$, we show that higher maximum FI can be
achieved by putting more weight on the $|0\rangle$ component of the SCS, at
the expense of sensitivity range, as shown in Fig. 4(a). On the other hand, if
the constraint is given by the maximum amplitude, e.g., $\alpha=1.6$, we show
states with a higher weight on $|\alpha\rangle$ give a higher maximum FI,
without compromising the sensitivity range, see Fig. 4(b). Essentially, in
both examples, states with higher photon number variance result in higher
maximum FI. We can also make use of the illustration given in Fig. 2(b). If
$\alpha$ is fixed, the Wigner function for states with more weight $w$ on the
$|\alpha\rangle$ component will have a stronger blob at $\alpha$. This way the
change of probability $p_{g0}$ is from 1 to a number which gets closer to zero
with increasing $w$, resulting in higher sensitivity. However, if
$\overline{n}$ is fixed, increasing the weight on the $|\alpha\rangle$
component comes at the cost of using a smaller $\alpha$, which is a more
dominant factor. As a result, the sensitivity is reduced.

To illustrate the versatility of our protocol, we demonstrate quantum-enhanced
precision for amplitude estimation by simply changing the process unitary in
Fig. 2(a) to a displacement in the imaginary direction
$\hat{U}(\gamma)=\exp(i\gamma(\hat{a}^{\dagger}+\hat{a}))$, where $\gamma$ is
the displacement amplitude. The process generator is now
$\hat{x}=(\hat{a}^{\dagger}+\hat{a})$, making states with high variance
$(\Delta\hat{x})^{2}$ useful for amplitude estimation. Here, SCS offer high
variance $\propto\overline{n}$ compared to CS, which have a constant variance
[33]. The illustration is similar to that of phase estimation in Fig. 2(b),
except the SCS are now displaced in the imaginary direction instead of
rotated. SCS with higher $\alpha$ have more and sharper fringes in between the
blobs, such that the overlap with the initial state reduces more sharply as
$\gamma$ increases.

Experimentally, we show that SCS displaced vertically in the imaginary
direction (SCS-v) offer the highest FI, compared to both CS and SCS displaced
horizontally in the real direction (SCS-h), as shown in Fig. 4(c). This can be
explained by noting that SCS-h have a different process unitary
$\hat{U}(\gamma)=\exp(i\gamma i(\hat{a}-\hat{a}^{\dagger}))$, where the
process generator is now $\hat{y}=i(\hat{a}-\hat{a}^{\dagger})$, such that the
variance of SCS is low, even lower than that of the CS.

Furthermore, by operating at the maximum FI, we show that the SCS-v (CS)
exhibits a linear (constant) trend against $\overline{n}$, as shown in Fig.
4(d). This suggests a $\sqrt{\overline{n}}$ quantum-enhanced precision of the
SCS-v over the CS. We refer to Ref. [33] for a comprehensive analysis of the
FI, which importantly includes the shift of the max FI to the initial position
$\gamma=0$. The $\sqrt{\overline{n}}$ enhancement also manifests in the direct
calculation of the amplitude precision, which reveals that
$\Delta\gamma\propto 1/\sqrt{\overline{n}}$ for SCS-v and
$\Delta\gamma\approx$ constant for CS [33]. Finally, directly comparing SCS-v
to CS reveals a high metrological gain of $9.3(5)$ dB.

In summary, we have introduced a versatile and on-demand protocol for
parameter estimation using a single bosonic mode. With this scheme, we
demonstrated both phase and amplitude estimation using the superposition of
coherent states and achieved quantum-enhanced precision
$\sim\sqrt{\overline{n}}$ in both cases, compared to classically behaving
coherent states. Using states with photon numbers of up to only 1.76, the high
variance of SCS results in state-of-the-art metrological gain, $7.5(6)$ dB for
phase and $9.3(5)$ dB for amplitude estimation. We also demonstrated the on-
demand adaption of the scheme to optimise for certain metrological performance
metrics with respect to hardware constraints.

The simplicity and effectiveness of our protocol stem from the use of a single
bosonic mode with two general state-transfer operations that can be realised
by any standard gates. As such, our implementation is accessible to systems
with different device parameters. Furthermore, the strategy is universal and
can be readily performed with many other bosonic systems beyond cQED using
their preferred native operations. Our results demonstrate a simple yet
powerful strategy for real metrological applications. For instance, quantities
such as stray magnetic, electric field, etc. that may affect the qubit or
cavity frequency will result in the change of dispersive coupling strength
$\chi$ [46], which in turn leads to phase variations on the bosonic state.
Hence, the phase precision $\Delta\theta$ obtained in our implementation is
directly related to the sensing precision of the relevant physical parameters
under investigation. Therefore, our work offers a promising first step towards
practical and optimal quantum metrology using a single bosonic system.

Acknowledgement. This project is supported by the National Research
Foundation, Singapore, grant number NRFF12-2020-0063 and NRF2020-NRF-
ISF004-3540. RF and KP acknowledge support of 21-13265X of the Czech Science
Foundation and EU H2020-WIDESPREAD-2020-5 project NONGAUSS (951737) under the
CSA - Coordination and Support Action.

## References

  * Giovannetti  _et al._ [2004] V. Giovannetti, S. Lloyd, and L. Maccone, Quantum-enhanced measurements: beating the standard quantum limit, Science 306, 1330 (2004).
  * Jones  _et al._ [2009] J. A. Jones, S. D. Karlen, J. Fitzsimons, A. Ardavan, S. C. Benjamin, G. A. D. Briggs, and J. J. Morton, Magnetic field sensing beyond the standard quantum limit using 10-spin noon states, science 324, 1166 (2009).
  * Napolitano  _et al._ [2011] M. Napolitano, M. Koschorreck, B. Dubost, N. Behbood, R. Sewell, and M. W. Mitchell, Interaction-based quantum metrology showing scaling beyond the heisenberg limit, Nature 471, 486 (2011).
  * Facon  _et al._ [2016] A. Facon, E.-K. Dietsche, D. Grosso, S. Haroche, J.-M. Raimond, M. Brune, and S. Gleyzes, A sensitive electrometer based on a rydberg atom in a schrödinger-cat state, Nature 535, 262 (2016).
  * Gilmore  _et al._ [2021] K. A. Gilmore, M. Affolter, R. J. Lewis-Swan, D. Barberena, E. Jordan, A. M. Rey, and J. J. Bollinger, Quantum-enhanced sensing of displacements and electric fields with two-dimensional trapped-ion crystals, Science 373, 673 (2021).
  * Burd  _et al._ [2019] S. Burd, R. Srinivas, J. Bollinger, A. Wilson, D. Wineland, D. Leibfried, D. Slichter, and D. Allcock, Quantum amplification of mechanical oscillator motion, Science 364, 1163 (2019).
  * Ludlow  _et al._ [2015] A. D. Ludlow, M. M. Boyd, J. Ye, E. Peik, and P. O. Schmidt, Optical atomic clocks, Reviews of Modern Physics 87, 637 (2015).
  * Riedel  _et al._ [2010] M. F. Riedel, P. Böhi, Y. Li, T. W. Hänsch, A. Sinatra, and P. Treutlein, Atom-chip-based generation of entanglement for quantum metrology, Nature 464, 1170 (2010).
  * Rouvière  _et al._ [2024] C. Rouvière, D. Barral, A. Grateau, I. Karuseichyk, G. Sorelli, M. Walschaers, and N. Treps, Ultra-sensitive separation estimation of optical sources, Optica 11, 166 (2024).
  * Tsang  _et al._ [2016] M. Tsang, R. Nair, and X.-M. Lu, Quantum theory of superresolution for two incoherent optical point sources, Physical Review X 6, 031033 (2016).
  * Tse  _et al._ [2019] M. Tse, H. Yu, N. Kijbunchoo, A. Fernandez-Galiana, P. Dupej, L. Barsotti, C. Blair, D. Brown, S. Dwyer, A. Effler, _et al._ , Quantum-enhanced advanced ligo detectors in the era of gravitational-wave astronomy, Physical Review Letters 123, 231107 (2019).
  * Yu  _et al._ [2020] H. Yu, L. McCuller, M. Tse, N. Kijbunchoo, L. Barsotti, and N. Mavalvala, Quantum correlations between light and the kilogram-mass mirrors of ligo, Nature 583, 43 (2020).
  * Backes  _et al._ [2021] K. M. Backes, D. A. Palken, S. A. Kenany, B. M. Brubaker, S. Cahn, A. Droster, G. C. Hilton, S. Ghosh, H. Jackson, S. K. Lamoreaux, _et al._ , A quantum enhanced search for dark matter axions, Nature 590, 238 (2021).
  * Nagata  _et al._ [2007] T. Nagata, R. Okamoto, J. L. O’brien, K. Sasaki, and S. Takeuchi, Beating the standard quantum limit with four-entangled photons, Science 316, 726 (2007).
  * Slussarenko  _et al._ [2017] S. Slussarenko, M. M. Weston, H. M. Chrzanowski, L. K. Shalm, V. B. Verma, S. W. Nam, and G. J. Pryde, Unconditional violation of the shot-noise limit in photonic quantum metrology, Nature Photonics 11, 700 (2017).
  * Guo  _et al._ [2020] X. Guo, C. R. Breum, J. Borregaard, S. Izumi, M. V. Larsen, T. Gehring, M. Christandl, J. S. Neergaard-Nielsen, and U. L. Andersen, Distributed quantum sensing in a continuous-variable entangled network, Nature Physics 16, 281 (2020).
  * Li  _et al._ [2023] Z. Li, S. Colombo, C. Shu, G. Velez, S. Pilatowsky-Cameo, R. Schmied, S. Choi, M. Lukin, E. Pedrozo-Peñafiel, and V. Vuletić, Improving metrology with quantum scrambling, Science 380, 1381 (2023).
  * Marciniak  _et al._ [2022] C. D. Marciniak, T. Feldker, I. Pogorelov, R. Kaubruegger, D. V. Vasilyev, R. van Bijnen, P. Schindler, P. Zoller, R. Blatt, and T. Monz, Optimal metrology with programmable quantum sensors, Nature 603, 604 (2022).
  * Mao  _et al._ [2023] T.-W. Mao, Q. Liu, X.-W. Li, J.-H. Cao, F. Chen, W.-X. Xu, M. K. Tey, Y.-X. Huang, and L. You, Quantum-enhanced sensing by echoing spin-nematic squeezing in atomic bose–einstein condensate, Nature Physics , 1 (2023).
  * Franke  _et al._ [2023] J. Franke, S. R. Muleady, R. Kaubruegger, F. Kranzl, R. Blatt, A. M. Rey, M. K. Joshi, and C. F. Roos, Quantum-enhanced sensing on optical transitions through finite-range interactions, Nature 621, 740 (2023).
  * Resch  _et al._ [2007] K. J. Resch, K. L. Pregnell, R. Prevedel, A. Gilchrist, G. J. Pryde, J. L. O’Brien, and A. G. White, Time-reversal and super-resolving phase measurements, Physical review letters 98, 223601 (2007).
  * Giovannetti  _et al._ [2011] V. Giovannetti, S. Lloyd, and L. Maccone, Advances in quantum metrology, Nature photonics 5, 222 (2011).
  * Braun  _et al._ [2018] D. Braun, G. Adesso, F. Benatti, R. Floreanini, U. Marzolino, M. W. Mitchell, and S. Pirandola, Quantum-enhanced measurements without entanglement, Reviews of Modern Physics 90, 035006 (2018).
  * Wolf  _et al._ [2019] F. Wolf, C. Shi, J. C. Heip, M. Gessner, L. Pezzè, A. Smerzi, M. Schulte, K. Hammerer, and P. O. Schmidt, Motional fock states for quantum-enhanced amplitude and phase measurements with trapped ions, Nature communications 10, 2929 (2019).
  * Podhora  _et al._ [2022] L. Podhora, L. Lachman, T. Pham, A. Lešundák, O. Číp, L. Slodička, and R. Filip, Quantum non-gaussianity of multiphonon states of a single atom, Physical Review Letters 129, 013602 (2022).
  * Deng  _et al._ [2023] X. Deng, S. Li, Z.-J. Chen, Z. Ni, Y. Cai, J. Mai, L. Zhang, P. Zheng, H. Yu, C.-L. Zou, _et al._ , Heisenberg-limited quantum metrology using 100-photon fock states, arXiv preprint arXiv:2306.16919 (2023).
  * McCormick  _et al._ [2019] K. C. McCormick, J. Keller, S. C. Burd, D. J. Wineland, A. C. Wilson, and D. Leibfried, Quantum-enhanced sensing of a single-ion mechanical oscillator, Nature 572, 86 (2019).
  * Wang  _et al._ [2019] W. Wang, Y. Wu, Y. Ma, W. Cai, L. Hu, X. Mu, Y. Xu, Z.-J. Chen, H. Wang, Y. Song, _et al._ , Heisenberg-limited single-mode quantum metrology in a superconducting circuit, Nature communications 10, 4382 (2019).
  * Haroche and Raimond [2006] S. Haroche and J.-M. Raimond,  _Exploring the quantum: atoms, cavities, and photons_ (Oxford university press, 2006).
  * Leibfried  _et al._ [2003] D. Leibfried, R. Blatt, C. Monroe, and D. Wineland, Quantum dynamics of single trapped ions, Reviews of Modern Physics 75, 281 (2003).
  * Grimm  _et al._ [2000] R. Grimm, M. Weidemüller, and Y. B. Ovchinnikov, Optical dipole traps for neutral atoms, in  _Advances in atomic, molecular, and optical physics_ , Vol. 42 (Elsevier, 2000) pp. 95–170.
  * Aspelmeyer  _et al._ [2014] M. Aspelmeyer, T. J. Kippenberg, and F. Marquardt, Cavity optomechanics, Reviews of Modern Physics 86, 1391 (2014).
  * [33] See details in supplementary material, .
  * Braunstein and Caves [1994] S. L. Braunstein and C. M. Caves, Statistical distance and the geometry of quantum states, Physical Review Letters 72, 3439 (1994).
  * Campagne-Ibarcq  _et al._ [2020] P. Campagne-Ibarcq, A. Eickbusch, S. Touzard, E. Zalys-Geller, N. E. Frattini, V. V. Sivak, P. Reinhold, S. Puri, S. Shankar, R. J. Schoelkopf, _et al._ , Quantum error correction of a qubit encoded in grid states of an oscillator, Nature 584, 368 (2020).
  * Eickbusch  _et al._ [2022] A. Eickbusch, V. Sivak, A. Z. Ding, S. S. Elder, S. R. Jha, J. Venkatraman, B. Royer, S. M. Girvin, R. J. Schoelkopf, and M. H. Devoret, Fast universal control of an oscillator with weak dispersive coupling to a qubit, Nature Physics 18, 1464 (2022).
  * Ofek  _et al._ [2016] N. Ofek, A. Petrenko, R. Heeres, P. Reinhold, Z. Leghtas, B. Vlastakis, Y. Liu, L. Frunzio, S. Girvin, L. Jiang, _et al._ , Extending the lifetime of a quantum bit with error correction in superconducting circuits, Nature 536, 441 (2016).
  * Heeres  _et al._ [2017] R. W. Heeres, P. Reinhold, N. Ofek, L. Frunzio, L. Jiang, M. H. Devoret, and R. J. Schoelkopf, Implementing a universal gate set on a logical qubit encoded in an oscillator, Nature communications 8, 94 (2017).
  * Hu  _et al._ [2019] L. Hu, Y. Ma, W. Cai, X. Mu, Y. Xu, W. Wang, Y. Wu, H. Wang, Y. Song, C.-L. Zou, _et al._ , Quantum error correction and universal gate set operation on a binomial bosonic logical qubit, Nature Physics 15, 503 (2019).
  * Heeres  _et al._ [2015] R. W. Heeres, B. Vlastakis, E. Holland, S. Krastanov, V. V. Albert, L. Frunzio, L. Jiang, and R. J. Schoelkopf, Cavity state manipulation using photon-number selective phase gates, Physical review letters 115, 137002 (2015).
  * Meekhof  _et al._ [1996] D. Meekhof, C. Monroe, B. King, W. M. Itano, and D. J. Wineland, Generation of nonclassical motional states of a trapped atom, Physical review letters 76, 1796 (1996).
  * Hacker  _et al._ [2019] B. Hacker, S. Welte, S. Daiss, A. Shaukat, S. Ritter, L. Li, and G. Rempe, Deterministic creation of entangled atom–light schrödinger-cat states, Nature Photonics 13, 110 (2019).
  * Wollack  _et al._ [2022] E. A. Wollack, A. Y. Cleland, R. G. Gruenke, Z. Wang, P. Arrangoiz-Arriola, and A. H. Safavi-Naeini, Quantum state preparation and tomography of entangled mechanical resonators, Nature 604, 463 (2022).
  * Pan  _et al._ [2023] X. Pan, J. Schwinger, N.-N. Huang, P. Song, W. Chua, F. Hanamura, A. Joshi, F. Valadares, R. Filip, and Y. Y. Gao, Protecting the quantum interference of cat states by phase-space compression, Physical Review X 13, 021004 (2023).
  * Colombo  _et al._ [2022] S. Colombo, E. Pedrozo-Peñafiel, A. F. Adiyatullin, Z. Li, E. Mendez, C. Shu, and V. Vuletić, Time-reversal-based quantum metrology with many-body entangled states, Nature Physics 18, 925 (2022).
  * Blais  _et al._ [2021] A. Blais, A. L. Grimsmo, S. M. Girvin, and A. Wallraff, Circuit quantum electrodynamics, Reviews of Modern Physics 93, 025005 (2021).

Supplementary Information

##  I Optimality of the protocol

The protocol consists of three parts: state preparation, estimated process and
projection measurement as shown in the main text, in which state-transfer
unitaries $\hat{O}_{1,2}$ can be done with any unitary for which the following
holds:

$\displaystyle|g\psi\rangle$ $\displaystyle=$
$\displaystyle\hat{O}_{1}|g0\rangle,$ $\displaystyle|g0\rangle$
$\displaystyle=$ $\displaystyle\hat{O}_{2}|g\psi\rangle.$ (2)

Note that the second line is equivalent to
$\hat{O}_{2}^{\dagger}|g0\rangle=|g\psi\rangle$ and $\langle
g0|\hat{O}_{2}=\langle g\psi|$. Let us denote the final state before the
qubit-cavity measurements as
$|\psi_{\text{final}}\rangle=\hat{O}_{2}\hat{U}(\beta)\hat{O}_{1}|g0\rangle$
and the state with encoded parameter as
$|g\psi(\beta)\rangle=\hat{U}(\beta)\hat{O}_{1}|g0\rangle$. The probability
after the qubit-cavity measurements, i.e., projection into $|g0\rangle$ can be
expressed as

$\displaystyle p_{g0}$ $\displaystyle=$ $\displaystyle\langle
g0|\psi_{\text{final}}\rangle\langle\psi_{\text{final}}|g0\rangle$ (3)
$\displaystyle=$ $\displaystyle\langle
g0|\hat{O}_{2}\hat{U}(\beta)\hat{O}_{1}|g0\rangle\langle
g0|\hat{O}_{1}^{\dagger}\hat{U}^{\dagger}(\beta)\hat{O}^{\dagger}_{2}|g0\rangle$
$\displaystyle=$ $\displaystyle\langle
g0|\hat{O}_{2}|g\psi(\beta)\rangle\langle
g\psi(\beta)|\hat{O}^{\dagger}_{2}|g0\rangle$ $\displaystyle=$
$\displaystyle\langle g\psi|g\psi(\beta)\rangle\langle
g\psi(\beta)|g\psi\rangle$ $\displaystyle=$
$\displaystyle|\langle\psi|\psi(\beta)\rangle|^{2}$

This overlap probability contains information regarding the parameter $\beta$
and is achieved with two state-transfer unitaries and standard qubit-cavity
measurements.

To assess the sensitivity of the probability for parameter estimation, we
compute the FI [34] for binary outcome:

${\rm FI}=\frac{1}{p_{g0}(1-p_{g0})}\left(\frac{\partial
p_{g0}}{\partial\beta}\right)^{2}.$ (4)

The FI depends on particular measurement setting. An optimal measurement
setting allows the FI to reach the QFI $4(\Delta\hat{B})^{2}$, which now only
depends on the state and encoding operator of the corresponding parameter of
interest. We now show that our measurement setting is optimal, i.e., the FI
saturates the QFI when $\beta\rightarrow 0$. In this case, we write
$\hat{U}(\beta)=e^{i\hat{B}\beta}\approx\openone+i\hat{B}\beta-\hat{B}^{2}\beta^{2}/2-i\hat{B}^{3}\beta^{3}/6+\hat{B}^{4}\beta^{4}/24$
such that the probability reads

$\displaystyle p_{g0}$ $\displaystyle=$
$\displaystyle|\langle\psi|\hat{U}(\beta)|\psi\rangle|^{2}$ (5)
$\displaystyle\approx$ $\displaystyle 1-\beta^{2}(\Delta\hat{B})^{2}$
$\displaystyle+\beta^{4}\left(\frac{1}{12}\langle\hat{B}^{4}\rangle-\frac{1}{3}\langle\hat{B}\rangle\langle\hat{B}^{3}\rangle+\frac{1}{4}\langle\hat{B}^{2}\rangle^{2}\right)$

Referring to Eq. (4), the FI can be written as

$\displaystyle\rm{FI}$ $\displaystyle=$
$\displaystyle\frac{1}{p_{g0}(1-p_{g0})}\left(\frac{\partial
p_{g0}}{\partial\beta}\right)^{2}$ $\displaystyle\approx$ $\displaystyle
4(\Delta\hat{B})^{2}$
$\displaystyle-\beta^{2}\left(\langle\hat{B}^{4}\rangle-4\langle\hat{B}\rangle\langle\hat{B}^{3}\rangle+3\langle\hat{B}^{2}\rangle^{2}-4(\Delta\hat{B})^{2}\right)$

where the absolute term indepedent of $\beta$ represents QFI.

##  II device and system parameters

Our device consists of a standard three-dimensional (3D) cavity, an ancillary
transmon and a planar readout resonator, see Fig. 5. The cavity is made of
high-purity (4N) aluminum and is a $\lambda/4$ cylinder-shaped transmission
line surrounded by a wall. The readout resonator is deposited together with
the transmon on the sapphire using double-angle evaporation technique. The
sapphire is inserted in the tunnel, with the pad slightly extending to the
cavity. The cavity serves as a high-Q quantum memory to store a bosonic mode
and is designed to dispersively couple with the ancillary transmon qubit,
which provides the necessary nonlinearity. The low-Q readout resonator is also
dispersively coupled with the transom qubit, enabling fast single-shot
measurement.

Figure 5: Schematic representation of 3D cQED hardware. The device comprises
three essential components: the cavity functions as the bosonic mode, the
transmon introduces nonlinearity, and the resonator serves for signal readout.
Additionally, one pin is coupled with the cavity for driving, while the
transmon and resonator share the same channel for operation.

The Hamiltonian of the system can be written as:

$\hat{H}=\hat{H_{0}}+\hat{H_{d}},$ (7)

where $\hat{H}_{0}$ $(\hat{H}_{d})$ incorporates the bare (drive) terms

$H_{0}=\Delta\hat{a}^{\dagger}\hat{a}-\chi\hat{a}^{\dagger}\hat{a}\hat{q}^{\dagger}\hat{q}-\frac{\alpha}{2}\hat{q}^{\dagger}\hat{q}^{\dagger}\hat{q}\hat{q}-\frac{K}{2}\hat{a}^{\dagger}\hat{a}^{\dagger}\hat{a}\hat{a}-\frac{\chi^{\prime}}{2}\hat{q}^{\dagger}\hat{q}\hat{a}^{\dagger}\hat{a}^{\dagger}\hat{a}\hat{a}$
$\hat{H}_{d}=i\zeta(t)\hat{a}^{\dagger}-i\zeta(t)^{*}\hat{a}+i\epsilon(t)\hat{q}^{\dagger}-i\epsilon(t)^{*}\hat{q}.$

Here $\hat{a}$ ($\hat{q}$) denotes the annihilation operator of the cavity
(transmon), $\Delta$ the frequency detunning between cavity and its drive,
$\chi$ the dispersive coupling, $\chi^{\prime}$ the second order dispersive
coupling, $\alpha$ the transmon anharmonicity, $K$ cavity anharmonicity or
Kerr, $\zeta$ and $\epsilon$ respectively the drive strength of the cavity and
transmon.

The system parameters are summarized in Table 1.

Cavity frequency (GHz) |  $\omega_{c}/2\pi$ |  4.587
---|---|---
Qubit frequency (GHz) |  $\omega_{q}/2\pi$ |  5.277
Resonator frequency (GHz) |  $\omega_{r}/2\pi$ |  7.617
Cavity single-photon lifetime (ms) |  $T_{c,1}$ |  1
Transmon energy lifetime $(\mu s)$ |  $T_{q,1}$ |  96
Transmon coherence lifetime $(\mu s)$ |  $T_{q,2}$ |  $15.47$
Cavity anharmonicity (kHz) |  $K/2\pi$ |  $6$
Transmon anharmonicity (kHz) |  $\alpha/2\pi$ |  $175.3$
1st cavity-transmon dispersive shift (MHz) |  $\chi/2\pi$ |  1.423
2nd cavity-transmon dispersive shift (MHz) |  $\chi^{\prime}/2\pi$ |  15.76
Transmon-resonator diespersive shift (MHz) |  $\chi_{qt}/2\pi$ |  0.64
Table 1: System parameters. Summary of the key system parameters and
lifetimes.

Note that for sufficiently high transmon anharmonicity, it can be effectively
treated as a qubit (two-level system with $|g\rangle$ ($|e\rangle$) as the
ground (excited) level). The first two terms in the bare Hamiltonian then
becomes
$\Delta\hat{a}^{\dagger}\hat{a}-\chi\hat{a}^{\dagger}\hat{a}|e\rangle\langle
e|$, which reduces to $(\chi/2)\hat{\sigma}_{z}\hat{a}^{\dagger}\hat{a}$ for
$\Delta=\chi/2$ that is chosen.

##  III Preliminary investigations using different states

Useful states for achieving low estimation error $\Delta\beta$ encoded with an
operator $\hat{B}$ via $\hat{U}(\beta)=e^{i\hat{B}\beta}$ are those with high
variance $(\Delta\hat{B})^{2}$.

In the context of phase estimation, we investigate several states such as CS
$|\alpha\rangle$, superposition of Fock states
$(|0\rangle+|N\rangle)/\sqrt{2}$, squeezed vacuum
$e^{\frac{-r}{2}(\hat{a}\hat{a}-\hat{a}^{\dagger}\hat{a}^{\dagger})}|0\rangle$,
and SCS $\mathcal{N}(|0\rangle+|\alpha\rangle)$. Here, the normalisation
factor $\mathcal{N}_{\alpha}=1/\sqrt{2(1+e^{-|\alpha|^{2}/2})}$. The average
photon number $\overline{n}=\langle\hat{a}^{\dagger}\hat{a}\rangle$ of these
states are respectively $|\alpha|^{2}$, $N/2$, $\sinh^{2}{r}$, and
$\mathcal{N}_{\alpha}^{2}|\alpha|^{2}$. The QFI $4(\Delta\hat{n})^{2}$ of
these states in terms of $n$ are respectively $4\overline{n}$,
$4\overline{n}^{2}$, $8\overline{n}^{2}+8\overline{n}$, and
$4((1-\mathcal{N}_{\alpha}^{2})/\mathcal{N}_{\alpha}^{2})\overline{n}^{2}+4\overline{n}$.
The QFI of SCS reduces to $4\overline{n}^{2}+4\overline{n}$ for large
$\overline{n}$ as $\mathcal{N}_{\alpha}\approx 1/\sqrt{2}$.

Figure 6: Phase and amplitude estimation utilizing various states. The FI of
phase estimation (a) and amplitude estimation (b) utilizing CS, superposition
of Fock states, squeezed vacuum (Sqvac) and SCS with fixed average photon
numbe $n=0.5$.

Experimentally, we realise these states and compute the corresponding FI with
the same method as described in the main text, see Fig. 6(a). The comparison
is made for states with the same photon number $\overline{n}=0.5$. It is
apparent that the SCS offers the highest FI. Although squeezed vacuum is
supposed to give a higher theoretical QFI, the state is harder to generate as
it requires high squeezing. Highly squeezed states exhibit increased spreading
in phase space, making them susceptible to the Kerr effect, which, in turn,
hinders quantum-enhanced sensitivity. A viable strategy to mitigate the impact
of the Kerr effect involves minimizing the coupling between the cavity and
transmon. By reducing this coupling, the undesirable consequences of the Kerr
effect can be alleviated, contributing to enhanced quantum sensitivity.
Therefore, we discern that with our device, the optimal states are SCS.

Similarly, in the context of amplitude estimation the QFI
$4(\Delta\hat{x})^{2}$ for CS, squeezed vacuum, and SCS are respectively given
by $4$, $8\sqrt{\overline{n}(\overline{n}+1)}+8\overline{n}+4$, and
$8\overline{n}+4$. The QFI for superposition of Fock states is $4$,
$12+4\sqrt{2}$, and $8\overline{n}+4$ for $\overline{n}=0.5$,
$\overline{n}=1$, and $\overline{n}>1$ respectively.

We experimentally perform the protocol and compute the FI, see Fig. 6(b).
Here, unlike the phase estimation, the highest FI is achieved by squeezed
vacuum followed by SCS. However, squeezed vacuum with higher $n$ is less
robust to Kerr and harder to realise. Therefore, we proceed with SCS for more
detailed analysis.

The phase and amplitude estimation underscore the importance of selecting
appropriate quantum states to achieve the best precision. Accompanied with
practical considerations, this presents an optimal quantum metrology.

##  IV Imperfection of the system

Figure 7: Position of maximum sensitivity. (a) Theoretical FI for SCS vs
$\theta$ for states with different average photon number $\bar{n}$. The pink
and red lines are FI at $\beta\rightarrow 0$ and maximum probability slope
$|\partial p/\partial\theta|$ respectively. Experimental data (blue dot) and
simulation (blue dash line) from Fig. 3(a) in the main text are replotted here
for comparison. (b) The corresponding different FI are plotted for comparison.
Figure 8: Performance of quantum-enhanced amplitude precision. (a) The average
FI of SCS for which it is above the max FI of CS and $\gamma$ offset at which
FI is maximum vs $n$. The dotted line is lineaer fit of
$3.2(2)\overline{n}^{1.0(3)}+2.2(1)$. (b) The same protocol as in Fig. 4(c) in
main text with initial states prepared with an offset. (c) The amplitude
precision vs $n$ for both SCS (blue circles) and CS (purple diamonds). The
dotted lines are linear fit of $\log(\Delta\gamma)=a\log(\Delta n)+b$ with $a$
and $b$ as free parameters. The fitted parameters are
$a,b=-0.460(3),-1.311(2)$ for SCS and $a,b=-0.0105(2),-0.480(1)$ for CS.

In the experiment, we use gradient ascent pulse engineering (GRAPE) pulse to
realize the two state transfer of the protocol. The GRAPE pulses ($\zeta(t)$
and $\epsilon(t)$) are optimized in an ideal system without imperfections. In
practice, each element of the system is susceptible to environmental noise,
making it crucial to account for the coherence and energy decay rate of both
the cavity and transmon qubit. Table 1 presents the corresponding parameters.
To account for decoherence, the simulations in the main text (all the
sequence) are performed with the following master equation:

Figure 9: Demonstration of amplitude estimation by state optimization. The FI
of SCS of different weights with (a) fixed photon number $n=1$ and (b) fixed
$\alpha=1.6$. The corresponding shaded area indicates standard deviation from
bootstrap.

$\frac{d\rho}{dt}=-\frac{i}{\hbar}[\hat{H},\rho]+\sum_{k}\hat{L}_{k}\rho\hat{L}_{k}^{\dagger}-\frac{1}{2}(\hat{L}_{k}^{\dagger}\hat{L}_{k}\rho+\rho\hat{L}_{k}^{\dagger}\hat{L}_{k}),$
(8)

where $\hat{L}_{k}$ denotes the jump operator for the corresponding
decoherence. We consider five types of decoherence: qubit relaxation
$\hat{L}_{1}=\hat{q}(1+E_{q})/\sqrt{T_{q,1}}$, qubit thermal excitation
$\hat{L}_{2}=\hat{q}^{\dagger}E_{q}/\sqrt{T_{q,1}}$, qubit dephasing
$\hat{L}_{3}=\hat{q}^{\dagger}\hat{q}/\sqrt{T_{q,2}}$, cavity relaxation
$\hat{L}_{4}=\hat{a}(1+E_{c})/\sqrt{T_{c,1}}$, cavity thermal excitation
$\hat{L}_{5}=\hat{a}^{\dagger}E_{c}/\sqrt{T_{c,1}}$, where $E_{c},E_{q}$ are
the spontaneous excitation rate of the cavity and qubit population
respectively.

##  V Comprehensive analysis for parameter estimation

In this section we provide analysis on the practical position of maximum
sensitivity, further details on amplitude estimation, and metrological gain vs
measurement repetitions.

In part I, we see that ideally maximum FI is achieved at $\beta\rightarrow 0$.
In the presence of imperfections, this is not the case. To exemplify this let
us first compute the FI of SCS for phase estimation. The probability of
overlap is given by

$\displaystyle p$ $\displaystyle=$
$\displaystyle|\langle\psi|\psi(\theta)\rangle|^{2}$ (9) $\displaystyle=$
$\displaystyle\frac{1}{4(1+e^{-\frac{\alpha^{2}}{2}})}|1+2e^{-\frac{\alpha^{2}}{2}}+e^{-\alpha^{2}(1-e^{i\theta})}|^{2}.$

From $p$, the theoretical FI can be derived analytically. This FI is plotted
in Fig. 7(a) vs $\theta$ for SCS with different average photon number
$\bar{n}$. We note that at $\beta\rightarrow 0$ (pink line), the FI saturates
the QFI. We also plotted the FI at the maximum slope $|\partial
p/\partial\theta|$ as red line. We note that the experimental data (blue
circles) follows this path closely. This supports a strategy that the
practical position of maximum sensitivity is around the maximum slope of
probability. The comparison of the corresponding FI is plotted in Fig. 7(b).

The supplemental analysis of amplitude estimation is shown in Fig. 8. Similar
to phase estimation, it is essential to gain insights into the mean
sensitivity above maximum FI of CS and the $\gamma$ offset at which the FI is
optimal. With the increase of photon number, the average FI increases linearly
while the optimal amplitude sensor point is moving to 0 as shown in Fig. 8(a).
The optimal amplitude sensor point can also be shifted to 0 by adding a
displacement to the initial state in the GRAPE pulse, which is crucial for
direct amplitude sensing. Furthermore, to estimate the amplitude estimation
error directly via $\Delta\gamma=\Delta p/|\partial p/\partial\gamma|$, we
present a benchmark in Fig. 3(c). The amplitude estimation error for the CS
remains constant even with an increase in the average photon number. In
contrast, the estimation error using SCS exhibits an almost linear
relationship with the average photon number and shows an optimal gain of
9.3(5)dB compared to the CS.

Furthermore, we also perform amplitude estimation with unbalanced SCS for
optimization. We note two distinct scenarios: one with a fixed average photon
number of 1 and another with a fixed $\alpha=1.6$, shown in Fig. 9(a) and Fig.
9(b), respectively. The former shows the less weight on the $\alpha$ component
of SCS yields a higher sensitivity. In the latter, the optimal sensitivity
with limited $\alpha$ is achieved by utilizing balanced components.

Figure 10: Metrological gain vs measurement repetitions. The gain of SCS over
CS for: (a) phase estimation and (b) amplitude estimation for different
measurement repetitions.

Finally, an important factor to consider for sensing applications is the
measurement repetitions, which also directly translates to the total duration
of the protocol. The highest metrological gain for phase and amplitude
estimation we report in the main text was done with $1000$ measurement
repetitions. The respective gain vs measurement repetitions are plotted in
Fig. 10. Notably, the gain for both phase and amplitude estimation stabilizes
at $\sim 600$ measurement repetitions. It shows that beyond this point, the
single shot measurements is enough to provide meaningful statistics.