# Multi-Objective Recommender Systems: Survey and Challenges

Dietmar Jannach<EMAIL_ADDRESS>University of KlagenfurtAustria

###### Abstract.

Recommender systems can be characterized as software solutions that provide
users convenient access to relevant content. Traditionally, recommender
systems research predominantly focuses on developing machine learning
algorithms that aim to predict which content is relevant for individual users.
In real-world applications, however, optimizing the accuracy of such relevance
predictions as a single objective in many cases is not sufficient. Instead,
multiple and often competing objectives have to be considered, leading to a
need for more research in multi-objective recommender systems. We can
differentiate between several types of such competing goals, including _(i)_
competing recommendation quality objectives at the individual and aggregate
level, _(ii)_ competing objectives of different involved stakeholders, _(iii)_
long-term vs. short-term objectives, _(iv)_ objectives at the user interface
level, and _(v)_ system level objectives. In this paper we review these types
of multi-objective recommendation settings and outline open challenges in this
area.

Recommender systems, Multi-objective optimization, Multistakeholder
recommendation, Short-term and long-term objectives

##  1\. Introduction

Generically defined, recommender systems can be characterized as _software
solutions that provide users convenient access to relevant content_. The types
of conveniences that such systems provide can be manifold. Historically,
recommender systems were mainly designed as information filtering tools, like
the early GroupLens system (Resnick et al., 1994) from 1994. Later on, various
other ways were investigated how such systems can create value, e.g., by
helping users _discover_ relevant content, by providing easy access to related
content (e.g., accessories), or by even taking automatic action like creating
and starting a music playlist.

While a recommender systems can serve various purposes and create value in
different ways (Jannach and Zanker, 2021), the predominant (implicit)
objective of recommender systems in literature today can be described as
“guide users to relevant items in situations of information overload”, or
simply “find good items” (Herlocker et al., 2000). The most common way of
operationalizing this information filtering problem is to frame the
recommendation task as a supervised machine learning problem. The core of this
problem is to learn a function from noisy data, which accurately predicts the
_relevance_ of a given item for individual users, sometimes also taking
contextual factors into account.

Although the actual relevance of recommended items can be assessed in
different ways (Gunawardana and Shani, 2015), data-based offline experiments
dominate the research landscape. In early years, rating prediction was
considered a central task of a recommender and the corresponding objective was
to minimize the mean absolute error (MAE), see (Shardanand and Maes, 1995) for
work using MAE in 1996. Nowadays, item ranking is mostly considered to be more
important than rating prediction, and a variety of corresponding ranking
accuracy measures are used today.

While the metrics changed over time, the research community has been working
on optimizing relevance predictions in increasingly sophisticated ways for
almost 30 years now. The main objective of such research is to minimize the
relevance prediction error or to maximize the accuracy of the recommendations.
The underlying assumption of these research approaches is that better
relevance predictions lead to systems that are more valuable for their users.
This seems intuitive for many practical applications, because a better
algorithm should surface more relevant items in the top-N lists shown to
users.

Such an assumption might however not always be true, and it was pointed out
many years ago that “being accurate is not enough” (McNee et al., 2006). A
recommender system might for example present users with obvious
recommendations, e.g., recommending new Star Wars sequels to a Star Wars
lover. The prediction error for such recommendations might be even close to
zero. But so will be the value of the recommendations to users, who most
probably know these movies already. Observations like this led to a multitude
of research efforts on “beyond-accuracy” measures like diversity, novelty, or
serendipity, see (Bradley and Smyth, 2001) for an early work from 2001. Which
beyond-accuracy dimension is relevant for a given setting depends on the
purpose the recommender is intended to serve (Jannach and Adomavicius, 2016).
Independent of the purpose, beyond-accuracy measures however often compete
with accuracy measures, leading to the problem that multiple objectives have
to be balanced when serving recommendations.

Historically, when considering the purpose of a recommender system, the focus
of research was on the value of such a system for _consumers_. Only in recent
years more attention was paid to the fact that recommender systems in practice
factually serve some business or organizational objectives. Considering these
provider-side aspects therefore requires that we see recommendation as a
problem where the interests and objectives of multiple stakeholders must be
considered (Abdollahpouri and Burke, 2022; Abdollahpouri et al., 2020), often
also taking different optimization time horizons into account.

Overall, while being able to predict the relevance of individual items for
users remains to be a central and relevant problem, considering only one type
of objectives, i.e., prediction accuracy, and the corresponding metrics may be
too simplistic and ultimately limit the impact of academic research efforts in
practice. One important way to escape the limitations of current research
practice is to consider multiple quality dimensions and stakeholder objectives
in parallel (Jannach and Bauer, 2020). Next, in Section 2, we will discuss
various forms of multi-objective recommender systems found in the literature.

##  2\. Types of Multi-Objective Recommendation Settings

On a very general level, we can define that “ _a multi-objective recommender
system (MORS) is a system designed to jointly optimize or balance more than
one optimization goal._ ” Figure 1 provides an taxonomy of different and
mostly orthogonal types of multi-objective recommendation settings. We note
that the various objectives to be pursued with a recommender system are in
many cases competing. Diversity goals, for instance, typically stand in
contrast with accuracy. However, not all objectives necessarily represent such
a contrast. Increasing the musical coherence of a playlist for example showed
to be advantageous in terms of accuracy in some cases in (Jannach et al.,
2015a). Also, when considering short-term and long-term objectives, taking
measures to increase interactivity and engagement with the system in the short
term is usually considered beneficial for customer retention in the long run
(Gomez-Uribe and Hunt, 2015). Note that while most discussed objectives are
_algorithmic_ ones, there may also be _non-algorithmic_ objectives at the user
experience level, i.e., objectives that are not tied to a specific underlying
algorithm.

Figure 1. Taxonomy of different types of multi-objective recommendation
settings.

###  2.1. Recommendation Quality Objectives

Under this category, we subsume problem settings, where more than one quality
objective of recommendations for users must be considered. We can
differentiate between the system considering such objectives at the level of
individual users or at an aggregate level, i.e., for the entire user base.

#### Individual level.

At the individual level, consumers can have specific (short-term) preferences,
e.g., regarding item features that should be considered in parallel. For
instance, a user of a hotel booking platform might be interested in a
relatively cheap hotel, which in addition is close to the city center. In such
a situation, a recommender system might therefore strive to find an offering
that respects both preferences as good as possible. Such problem settings
often occur in _interactive_ or _conversational_ recommendation scenarios.
Technically, a variety of Multi-Criteria Decision-Making (MCDM) methods such
as Analytic Hierarchical Process (AHP), the Weighted Sum Method (WSM) or
Multi-Attribute Utility Theory (MAUT) can be applied, see (Triantaphyllou,
2000; Manouselis and Costopoulou, 2007). Such methods can also be part of
interactive constraint-based recommendation approaches (Jannach, 2004) to rank
items. In addition, some of these constraint-based approaches support the
automatic relaxation of individual user preferences in case none of the
remaining items matches all consumer preferences (Felfernig et al., 2006).

A different way to consider multiple user preferences at the individual level
is called _calibration_. In such an approach, the idea usually is to match
certain beyond-accuracy aspects of the recommendation list with past
preference profiles of individual users. In an early work, Oh et al. (Oh et
al., 2011) tried to align the recommendations with the past popularity
tendencies of a user. Later, Jugovac et al. (Jugovac et al., 2017) extended
the approach for multiple optimization objectives. A more formal
characterization of calibration was introduced in (Steck, 2018) by Steck, and
(Abdollahpouri et al., 2021) represents another recent work into that
direction. In most cases the central idea of these approaches is to match two
distributions, e.g., the popularity distribution of items in the user profile
and the popularity distribution of recommendations. An alternative
optimization goal was used in (Jannach et al., 2015a) for the music domain,
where the objective was to find musically coherent playlist continuations
while preserving prediction accuracy.

#### Aggregate level.

The majority of published research on balancing different recommendation
quality aspects targets the aggregate level. The objective of such works is to
balance the recommendations for the entire user base, the corresponding
metrics are therefore usually averages. The most common beyond-accuracy
measures in the literature include diversity, novelty, serendipity, catalog
coverage, popularity bias or fairness, see, e.g., (Adomavicius and Kwon, 2012;
Kaminskas and Bridge, 2016; Vargas and Castells, 2011; Abdollahpouri et al.,
2017; Ekstrand et al., 2022). Most commonly, the goal is to balance accuracy
with exactly one of these measures, assuming that there is a trade-off between
these quality factors. Increasing diversity is for example commonly assumed
having a negative impact on accuracy metrics. A few works exist which consider
more than two factors. In an earlier work in this area (Rodriguez et al.,
2012), the authors describe an effort to build a _talent recommendation_
system at LinkedIn, which not only considers the semantic match between a
candidate profile and a job, but which also take side constraints into
account, for instance, the presumed willingness of a candidate to change
positions.

Technically, a variety of approaches to balance competing goals can be found
in the literature. Re-ranking accuracy-optimized lists is probably the most
common approach and was also used in early approaches for diversification
(Bradley and Smyth, 2001). Alternative techniques for creating a balanced
lists or for merging different lists include constraint optimization
techniques, graph-based and neighborhood-based approaches, methods that are
based on the concept of Pareto efficiency, methods that integrate different
objectives in their loss function, or bandit-based approaches that address the
explore-exploit problem for improved discovery through novel recommendations
(Zhang and Hurley, 2008; Jambor and Wang, 2010; Zhang et al., 2012; Said et
al., 2013; Ribeiro et al., 2015; de Souza Pereira Moreira et al., 2019;
McInerney et al., 2018; Zhou et al., 2010; Isufi et al., 2021).

###  2.2. Multistakeholder Objectives

The beyond-accuracy quality metrics discussed in the previous section were
historically mostly introduced to improve recommendations for end users.
Higher diversity, for example, should avoid monotonicity, and novelty should
support discovery. The underlying assumption—also of pure accuracy-oriented
works—is that improving different quality aspects for users would be at least
indirectly beneficial for the providers. Only in recent years, more attention
was paid in the literature to the fact that many recommendation scenarios in
the real world are situated in environments, where the objectives of multiple
stakeholders have to be considered. The common players in such
_multistakeholder_ recommendation problems include _end consumers_ ,
recommendation _service providers_ , item _suppliers_ , and sometimes even
parts of a broader _society_ (Abdollahpouri et al., 2020; Jannach and Bauer,
2020). In such settings, a recommender system may serve different purposes for
different stakeholders (Jannach and Adomavicius, 2016), and the related
objectives may stand in conflict.111In some cases there may even be subgroups
within the consumer stakeholder group, e.g., free vs. premium or new vs.
existing customers, for which different objectives may exist.

A typical problem setting in practice that involves _two stakeholders_ is that
of balancing consumer and provider objectives. In many cases, there may be a
potential trade-off between (a) recommending the _most relevant_ items for
consumers and (b) recommending items that are also somewhat relevant but
assumed to be favorable in terms of the provider’s business objectives. Some
of the discussed beyond-accuracy metrics can actually be seen as serving both
stakeholders. Making more novel recommendations not only potentially leads to
a better user experience, but also to more engagement with the service and
longer term customer retention (Anderson et al., 2020).

A number of research works however also consider business (or: organizational)
objectives more directly, in particular in the form of recommender systems
that are “‘price and profit aware” (Jannach and Adomavicius, 2017). In (Chen
et al., 2008), for instance, two heuristic profit-aware strategies are
proposed and the authors found that such methods can increase the profit from
cross-selling without losing much recommendation accuracy. A number of other
possible solutions to maximize provider profitability were proposed over the
years, using a variety of approaches including mathematical optimization with
side constraints, re-ranking accuracy-optimized lists based on profit
considerations, random walks over graphs or analytical modeling (Azaria et
al., 2013; Wang and Wu, 2009; Azaria et al., 2013; Panniello et al., 2016)

Besides such situations with potential consumer-provider trade-offs, there are
settings that involve even more stakeholders to consider. A problem that has
been studied for several years—even though not under the name multistakeholder
recommendation—is that of _group recommendation_ (Masthoff, 2015). In such
settings, the system’s goal is to determine a set of recommendations that suit
the preferences of a group of users, e.g., friends who want to watch a movie
together. A variety of strategies to aggregate the individual user preferences
were proposed over the years. Early works on the topic can be found in
(O’Connor et al., 2001) and (Masthoff, 2004). In (Masthoff, 2004), for
instance, Masthoff reports the outcomes of different user studies aimed to
understand how humans make choices for a group and finds that humans indeed
sometimes follow strategies inspired from Social Choice Theory. We note here
that the group recommendation setting differs from other multistakeholder
scenarios in that all stakeholders receive the same set of recommendations.

_Reciprocal recommendation_ is another specific set of problem settings
involving multiple stakeholders. Here, instead of recommending items to users,
the problem is to recommend users to users, also known as people-to-people
recommendation. Typical application scenarios are recommendations on dating
and recruiting platforms. A particularity of such settings is that the success
of a recommendation is not determined solely by the recipient of the
recommendation, but there must be a mutual preference match or compatibility
between the two people involved, see (Palomares et al., 2021) for an in-depth
discussion on the topic. The recommendation service provider therefore faces
additional complexities in the matching process and in parallel has to observe
its own business objectives and constraints. On a job recommendation platform,
for example, the provider may have to additionally ensure that each paid job
advertisement receives a minimum number of relevant impressions, i.e.,
exposure (Abel et al., 2017).

Similar considerations may generally apply when the recommendation platform
serves as a marketplace with multiple suppliers of identical or comparable
items. Let us consider again the example of a typical hotel booking platform,
which serves personalized recommendations to its users (Jannach and Bauer,
2020). Besides the consumer, who already might have competing objectives,
there are the property owners, who have their offerings listed on the booking
platform and pay a commission for each booking. The goal of the property
owners is that their offerings are exposed to as many matching customers as
possible in order to increase the chances of being booked. The booking
platform, finally, is not only interested in recommending matching hotels to
consumers, but might also seek to maximize their commission, e.g., by
recommending slightly more expensive hotels. In addition, to balancing these
objectives, the platform may furthermore have to ensure that _all_ listed
properties reach a sufficient level of exposure, i.e., chance of being booked.
This may be required to ensure a long-term relationship with property owners,
who might otherwise discontinue to list their offerings on the platform at
some stage (Krasnodebski and Dines, 2016). We discuss short-term and long-term
objectives next.

###  2.3. Time Horizon Objectives

In some application domains, it might be quite simple to increase short-term
Key Performance Indicators (KPIs). In the hotel booking scenario which we have
just discussed, boosting short-term revenue might be achieved by recommending
hotels with currently discounted rates, which maximizes the probability of a
transaction (Jannach et al., 2017). In the news domain, recommending articles
on trending topics, articles with click-bait headlines or generally popular
content such as celebrity gossip may lead to high click-through-rates (CTR).
In the music domain, recommending tracks of trending or popular artists, which
the user already knows, may be a safe strategy when the target metric is to
avoid “skip” events.

Such strategies that are successful in the short term may however be non-
optimal or even detrimental in the long run. The recommendation of discounted
hotel rooms may be bad for profit. News readers may be disappointed when
actually reading articles with a clickbait headline and may not trust these
recommendations in the future. Music listeners finally may have difficulties
to discover new artists over time and may quit using the service after some
time.

Most academic research is based on one-shot evaluations, typically focusing on
the prediction accuracy given a static dataset and a certain point in time.
Longitudinal effects of different recommendation strategies are much less
explored and there is also limited literature on the long-term effects of
recommender systems in industry. A/B tests in industry may last from a few
weeks to several months. In (Gomez-Uribe and Hunt, 2015), the case of Netflix
is discussed, where one main KPI is customer retention, which is oriented
towards the long-term perspective. In their case, attributing changes in the
recommender system to such long-term effects is reported to be challenging,
e.g., because of already high retention rates and the need for large user
samples. Other reports from real-world deployments a recommender systems can
be found in (Panniello et al., 2016) or (Lee and Hosanagar, 2019). In (Lee and
Hosanagar, 2019), the authors for example found that using a recommender
system led to decreased sales diversity compared to a situation without a
recommender. A similar effect was reported in (Anderson et al., 2020), where
the recommender system on a music streaming site led to a reduced consumption
diversity. A survey of other reports on real-world applications of recommender
systems can be found in (Jannach and Jugovac, 2019).

Given the limitations of one-shot evaluations, we have observed an increased
interest in longitudinal studies in recent years. One prominent line of
research lies in the area of reinforcement learning (RL) approaches in
particular in the form of contextual bandits, see e.g. (Li et al., 2010) for
an earlier work in the news domain. In such approaches, the system
sequentially selects items to recommend to users and then incorporates the
users’ feedback for subsequent recommendations. Different recommendation
algorithms can be evaluated offline with the help of simulators, e.g., (Rohde
et al., 2018; McInerney et al., 2021). A common challenge in this context is
to ensure that such evaluations are unbiased (Li et al., 2011; Huang et al.,
2020). We note that the consideration of temporal aspects such as different
time horizons or delayed feedback have been explored in the RL literature for
the related problem of computational advertising for several years
(Theocharous et al., 2015; Chapelle, 2014).

Reinforcement learning approaches typically aim at finding a strategy to
maximize the expected _reward_. During the last few years, a number of studies
that use other forms of simulations were published that focus on other
important long-term phenomena of recommender systems. These studies for
example focus on longitudinal effects of recommender systems on sales
diversity (Fleder and Hosanagar, 2009), potential reinforcement effects in
terms of popularity bias and other aspects for traditional and session-based
recommendations (Jannach et al., 2015b; Ferraro et al., 2020), longitudinal
performance effects of recommender systems and the “performance paradox”
(Zhang et al., 2019), differences in terms of long-term effects of consumer-
oriented and profit-oriented recommendation strategies (Ghanem et al., 2022).

###  2.4. User Experience Objectives

Going beyond specifics of individual algorithms, there can be also various
objectives to be pursued at the user interaction level of a recommender
system. The design space for the user interface of recommender systems is
actually large, see (Jugovac and Jannach, 2017), and there thus may be a
number of competing objectives at the user interface (UI) level.

Here, we only list a few examples of potential trade-offs that may be common
for many recommender system applications.

  * •

_Information Completeness vs. Information Overload_ : This, for instance,
refers to the question of how many items should be shown to users and if we
should completely filter out certain items from the result list. Showing too
few options may give users the feeling that the system holds back some
information. If there is too much information users will find themselves again
in a situation of information overload (Bollen et al., 2010; Aljukhadar et
al., 2012). Besides the question of how many options to show, a related
question is how much detail and additional information to show for each
recommendation.

  * •

_Transparency and User Control vs. Cognitive Effort_ : Transparency and
explanations are commonly considered to be trust-establishing factors in
recommender systems (Pu et al., 2011). A variety of different ways of
explaining recommendations were proposed in the literature (Tintarev and
Masthoff, 2011; Nunes and Jannach, 2017). Many of these academic proposals are
quite complex and may easily cognitively overload average end users. Similar
considerations apply for approaches that implement mechanisms for _user
control_ in recommender systems (Ekstrand et al., 2015; Jannach et al., 2016).

  * •

_Flexibility vs. Efficiency_ : This question arises in the context of modern
conversational recommender systems that are implemented in the form of
chatbots. Chatbots typically support two forms of interactions: a) natural
language input and b) form-based input (i.e., using buttons). While natural
language inputs may allow for more flexible interactions, the study in (Iovine
et al., 2020), for instance, indicated that a combination of interaction
modalities was most effective.

Several other more general design trade-offs may exist depending on the
specific application, e.g., regarding acceptable levels automating adaptivity
of the user interface, which may hamper usability (Paymans et al., 2004).

###  2.5. System Objectives

In this final category we discuss technical aspects and their potential trade-
offs. We call them “system objectives”, as they refer to more general system
properties.

One such trade-off in practice may lie in the complexity of the underlying
algorithms and the gains that one may obtain in terms of business-related
KPIs. Already in the context of the Netflix Prize (Bennett et al., 2007) we
could observe that the winning solutions were finally not put into production,
partly due to their complexity. Similar considerations can be made for today’s
sometimes computationally demanding methods based on deep learning. In some
cases there might be a diminishing return of deploying the most sophisticated
models in production, only because they lead to slightly better accuracy
values in offline testing. In some research works, it even turns out that
“embarrassingly shallow” models can be highly competitive in offline
evaluations (Steck, 2019).

With highly complex models, not only scalability issues may arise and monetary
costs for computing resources may increase, the complexity of the
architectures might also make such systems more difficult to maintain, debug,
and explain. On the other hand, solutions built upon modern deep learning
frameworks are sometimes reported to be advantageous over conceptually
simpler, but specialized solutions, because these frameworks and deep learning
architectures make it very easy to integrate various types of information into
the models (Steck et al., 2021).

##  3\. Summary and Challenges

Our review outlines that providing automated recommendations is a problem that
may require the consideration of more than one objective in many real-world
use cases. Such multi-objective settings may include competing objectives of
consumers, possible tensions between goals of different stakeholders,
conflicts when optimizing for different time horizons, competing design
choices at the UI level, as well as system-level and engineering-related
considerations. In this work, we reviewed the literature in this area and
provided a taxonomy to organize the various dimensions of multi-objective
recommendation. We note here that the categories of the taxonomy are not
mutually exclusive. For instance, a multi-objective recommendation approach
may address both aspects regarding different time horizons as well as the
possibly competing goals of the involved stakeholders.

In practice, one main challenge may usually lie in deciding on the right
balance between the competing goals from an organizational perspective.
Various stakeholders from different organizational units may have to agree on
such decisions, and corresponding KPIs need to be defined and monitored. Given
these KPIs, suitable optimization goals and possibly proxy measures have to be
implemented and validated at the technical level.

In academic settings, researchers typically abstract from the specifics of a
given application context, aimed at developing generalizable algorithmic
solutions to deal with multi-objective problem settings. This abstraction
process commonly involves the use of _offline evaluation approaches_ , the
establishment of certain assumptions, and the introduction of computational
metrics which should be optimized. After such an abstraction, one main
challenge, however, lies in the evaluation process and, in particular, in
making sure that improvements that are observed in terms of abstract
evaluation measures would translate to better systems in practice (Cremonesi
and Jannach, 2021).

Unfortunately, in many of today’s research works, we observe phenomena similar
to the “abstraction traps” described by Selbst et al. (Selbst et al., 2019) in
the context of research on algorithmic works in _Fair Machine Learning_. In
the case of competing individual-level quality goals, for example, how can we
be sure that a particular diversity metric, which we optimize such as an
intra-list similarity, matches human perceptions and what would be the right
balance for a given application setting or an individual user? How do we know
if calibrated recommendations are liked more by users, and what would be the
effects of calibration on organizational goals? Answering such questions
requires corresponding user studies to, e.g., validate that the computational
metrics are good proxies for human perceptions.

The problem however becomes even more challenging when not even the target
concepts are entirely clear. In recent years, a widely investigated multi-
objective problem setting is the provision of _fair_ recommendations (Ekstrand
et al., 2022). Unfortunately, optimizing for fairness turns out to be
challenging, as fairness is a societal construct. Researchers therefore came
up with various types of ways of operationalizing fairness constraints.
However, in many of such works, little or no evidence or argumentation is
provided why the chosen fairness metrics are meaningful in practice in general
or in a particular application setting, see (Deldjoo et al., 2022) for a
survey on the recent literature. In several cases, making fair recommendations
is simply equated with reducing the popularity bias of recommendations, e.g.,
by matching it with a target distribution, which is assumed to be given. In
reality, however, it is not clear if it would be fair to frequently recommend
items that are not popular—these items might simply be of poor quality—or if
users would even _perceive_ these recommendations to be fair.222Avoiding
popularity biases in general, i.e., beyond fairness considerations, certainly
is important for different reasons in practice, e.g., to ensure a certain
level of exposure for novel (cold-start) items.  Similar considerations apply
for many other types of multi-objective recommender systems.

Overall, these observations call for more studies involving humans in the
evaluation loop and industry partners in the research process. However, only
few works exist in that direction so far. An example of a user study can be
found in (Azaria et al., 2013), outcomes of field studies are described in
(Panniello et al., 2016). An offline evaluation with real-world data from
industry is done in (Mehrotra et al., 2018), but even in this case it is not
clear if the computational metrics truly correspond to the real-world goals,
e.g., if more listening events on the music platform lead to higher user
satisfaction as claimed. In a very recent work, Wang and colleagues from
Google (Wang et al., 2022) investigated the relationship between observed user
behavior—using data from a large platform—and desired long-term outcomes in
terms of user experience. With their work, they aim at providing guidance for
researchers and practitioners when selecting surrogate measures to address the
difficult problem of optimizing for long-term objectives.

Ultimately, despite such recent progress, multi-objective recommender systems
remains a highly important research area with a number of challenging research
questions. Addressing such questions will however help to pave the way towards
more impactful recommender systems research in the future.

## References

  * (1)
  * Abdollahpouri et al. (2020) Himan Abdollahpouri, Gediminas Adomavicius, Robin Burke, Ido Guy, Dietmar Jannach, Toshihiro Kamishima, Jan Krasnodebski, and Luiz Pizzato. 2020.  Multistakeholder Recommendation: Survey and Research Directions.  _User Modeling and User-Adapted Interaction_ 30 (2020), 127–158.
  * Abdollahpouri and Burke (2022) Himan Abdollahpouri and Robin Burke. 2022.  Multistakeholder Recommender Systems.  In _Recommender Systems Handbook_ , Francesco Ricci, Lior Rokach, and Bracha Shapira (Eds.). Springer US, New York, NY, 647–677.
  * Abdollahpouri et al. (2017) Himan Abdollahpouri, Robin Burke, and Bamshad Mobasher. 2017\.  Controlling Popularity Bias in Learning-to-Rank Recommendation. In _Proceedings of the Eleventh ACM Conference on Recommender Systems_ _(RecSys ’17)_. 42–46.
  * Abdollahpouri et al. (2021) Himan Abdollahpouri, Masoud Mansoury, Robin Burke, Bamshad Mobasher, and Edward C. Malthouse. 2021.  User-centered Evaluation of Popularity Bias in Recommender Systems. In _Proceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization, UMAP 2021_. 119–129.
  * Abel et al. (2017) Fabian Abel, Yashar Deldjoo, Mehdi Elahi, and Daniel Kohlsdorf. 2017.  RecSys Challenge 2017: Offline and Online Evaluation. In _Proceedings of the Eleventh ACM Conference on Recommender Systems, RecSys 2017_. 372–373.
  * Adomavicius and Kwon (2012) Gediminas Adomavicius and YoungOk Kwon. 2012.  Improving Aggregate Recommendation Diversity Using Ranking-Based Techniques.  _IEEE TKDE_ 24, 5 (2012), 896–911.
  * Aljukhadar et al. (2012) Muhammad Aljukhadar, Sylvain Senecal, and Charles-Etienne Daoust. 2012.  Using recommendation agents to cope with information overload.  _International Journal of Electronic Commerce_ 17, 2 (2012), 41–70.
  * Anderson et al. (2020) Ashton Anderson, Lucas Maystre, Ian Anderson, Rishabh Mehrotra, and Mounia Lalmas. 2020.  Algorithmic Effects on the Diversity of Consumption on Spotify. In _Proceedings of The Web Conference 2020_ _(WWW ’20)_. 2155–2165.
  * Azaria et al. (2013) Amos Azaria, Avinatan Hassidim, Sarit Kraus, Adi Eshkol, Ofer Weintraub, and Irit Netanely. 2013\.  Movie Recommender System for Profit Maximization. In _Proceedings of the 7th ACM Conference on Recommender Systems_ (Hong Kong, China) _(RecSys ’13)_. 121–128.
  * Bennett et al. (2007) James Bennett, Stan Lanning, et al. 2007\.  The Netflix Prize. In _Proceedings of KDD cup and workshop_ , Vol. 2007. 35.
  * Bollen et al. (2010) Dirk G. F. M. Bollen, Bart P. Knijnenburg, Martijn C. Willemsen, and Mark P. Graus. 2010\.  Understanding choice overload in recommender systems. In _Proceedings of the 4th Conference on Recommender Systems (RecSys ’10)_. 63–70.
  * Bradley and Smyth (2001) K. Bradley and B. Smyth. 2001.  Improving Recommendation Diversity. In _Proceedings of the 12th National Conference in Artificial Intelligence and Cognitive Science_. 75–84.
  * Chapelle (2014) Olivier Chapelle. 2014\.  Modeling Delayed Feedback in Display Advertising. In _Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining_ _(KDD ’14)_. 1097–1105.
  * Chen et al. (2008) Long-Sheng Chen, Fei-Hao Hsu, Mu-Chen Chen, and Yuan-Chia Hsu. 2008\.  Developing recommender systems with the consideration of product profitability for sellers.  _Information Sciences_ 178, 4 (2008), 1032–1048.
  * Cremonesi and Jannach (2021) Paolo Cremonesi and Dietmar Jannach. 2021.  Progress in recommender systems research: Crisis? What crisis?  _AI Magazine_ 42, 3 (2021), 43–54.
  * de Souza Pereira Moreira et al. (2019) Gabriel de Souza Pereira Moreira, Dietmar Jannach, and Adilson Marques da Cunha. 2019.  Contextual Hybrid Session-based News Recommendation with Recurrent Neural Networks.  _IEEE Access_ 7 (2019), 169185–169203.
  * Deldjoo et al. (2022) Yashar Deldjoo, Dietmar Jannach, Alejandro Bellogin, Alessandro Difonzo, and Dario Zanzonelli. 2022\.  A Survey of Research on Fair Recommender Systems.  https://arxiv.org/abs/2205.11127
  * Ekstrand et al. (2022) Michael D. Ekstrand, Anubrata Das, Robin Burke, and Fernando Diaz. 2022.  Fairness in Information Access Systems.  _Foundations and Trends® in Information Retrieval_ 16, 1-2 (2022), 1–177.
  * Ekstrand et al. (2015) Michael D. Ekstrand, Daniel Kluver, F. Maxwell Harper, and Joseph A. Konstan. 2015.  Letting Users Choose Recommender Algorithms: An Experimental Study. In _Proceedings of the 9th ACM Conference on Recommender Systems_ _(RecSys ’15)_. 11–18.
  * Felfernig et al. (2006) Alexander Felfernig, Gerhard Friedrich, Dietmar Jannach, and Markus Zanker. 2006.  An Integrated Environment for the Development of Knowledge-Based Recommender Applications.  _International Journal of Electronic Commerce_ 11, 2 (2006), 11–34.
  * Ferraro et al. (2020) Andres Ferraro, Dietmar Jannach, and Xavier Serra. 2020\.  Exploring longitudinal effects of session-based recommendations. In _Proceedings of the 2020 ACM Conference on Recommender Systems_. 474–479.
  * Fleder and Hosanagar (2009) Daniel Fleder and Kartik Hosanagar. 2009.  Blockbuster Culture’s Next Rise or Fall: The Impact of Recommender Systems on Sales Diversity.  _Management Science_ 55, 5 (2009), 697–712.
  * Ghanem et al. (2022) Nada Ghanem, Stephan Leitner, and Dietmar Jannach. 2022\.  Balancing Consumer and Business Value of Recommender Systems: A Simulation-based Analysis.  _E-Commerce Research and Applications_ forthcoming (2022).
  * Gomez-Uribe and Hunt (2015) Carlos A. Gomez-Uribe and Neil Hunt. 2015.  The Netflix Recommender System: Algorithms, Business Value, and Innovation.  _Transactions on Management Information Systems_ 6, 4 (2015), 13:1–13:19.
  * Gunawardana and Shani (2015) Asela Gunawardana and Guy Shani. 2015.  Evaluating Recommender Systems.  In _Recommender Systems Handbook_ , Francesco Ricci, Lior Rokach, and Bracha Shapira (Eds.). Springer, 265–308.
  * Herlocker et al. (2000) Jonathan L. Herlocker, Joseph A. Konstan, and John Riedl. 2000\.  Explaining Collaborative Filtering Recommendations. In _CSCW ’00_. 241–250.
  * Huang et al. (2020) Jin Huang, Harrie Oosterhuis, Maarten de Rijke, and Herke van Hoof. 2020.  Keeping Dataset Biases out of the Simulation: A Debiased Simulator for Reinforcement Learning Based Recommender Systems. In _Fourteenth ACM Conference on Recommender Systems_ _(RecSys ’20)_. 190–199.
  * Iovine et al. (2020) Andrea Iovine, Fedelucio Narducci, and Giovanni Semeraro. 2020\.  Conversational Recommender Systems and Natural Language: A study through the ConveRSE Framework.  _Decision Support Systems_ 131 (2020), 113250–113260.
  * Isufi et al. (2021) Elvin Isufi, Matteo Pocchiari, and Alan Hanjalic. 2021\.  Accuracy-diversity trade-off in recommender systems via graph convolutions.  _Information Processing & Management_ 58, 2 (2021), 102459\.
  * Jambor and Wang (2010) Tamas Jambor and Jun Wang. 2010.  Optimizing Multiple Objectives in Collaborative Filtering. In _Proceedings of the Fourth ACM Conference on Recommender Systems_ _(RecSys ’10)_. 55–62.
  * Jannach (2004) Dietmar Jannach. 2004\.  ADVISOR SUITE – A Knowledge-based Sales Advisory System. In _ECAI ’04_. 720–724.
  * Jannach and Adomavicius (2016) Dietmar Jannach and Gediminas Adomavicius. 2016.  Recommendations with a Purpose. In _Proceedings of the 10th ACM Conference on Recommender Systems_ (Boston, Massachusetts, USA) _(RecSys ’16)_. 7–10.
  * Jannach and Adomavicius (2017) Dietmar Jannach and Gediminas Adomavicius. 2017.  Price and Profit Awareness in Recommender Systems.  _Proceedings VAMS Workshop at ACM RecSys 2017_ CoRR abs/1707.08029 (2017).
  * Jannach and Bauer (2020) Dietmar Jannach and Christine Bauer. 2020.  Escaping the McNamara Fallacy: Towards more Impactful Recommender Systems Research.  _AI Magazine_ 41, 4 (2020), 79–95.
  * Jannach and Jugovac (2019) Dietmar Jannach and Michael Jugovac. 2019.  Measuring the business value of recommender systems.  _ACM TMIS_ 10, 4 (2019), 1–23.
  * Jannach et al. (2015a) Dietmar Jannach, Lukas Lerche, and Iman Kamehkhosh. 2015a.  Beyond ”Hitting the Hits” - Generating Coherent Music Playlist Continuations with the Right Tracks. In _Proceedings of the 9th ACM Conference on Recommender Systems (RecSys 2015)_. Vienna, Austria, 187–194.
  * Jannach et al. (2015b) Dietmar Jannach, Lukas Lerche, Iman Kamehkhosh, and Michael Jugovac. 2015b.  What recommenders recommend: an analysis of recommendation biases and possible countermeasures.  _User Modeling and User-Adapted Interaction_ 25, 5 (2015), 427–491.
  * Jannach et al. (2017) Dietmar Jannach, Malte Ludewig, and Lukas Lerche. 2017\.  Session-based item recommendation in e-commerce: on short-term intents, reminders, trends and discounts.  _User Modeling and User-Adapted Interaction_ 27, 3 (2017), 351–392.
  * Jannach et al. (2016) Dietmar Jannach, Sidra Naveed, and Michael Jugovac. 2016\.  User Control in Recommender Systems: Overview and Interaction Challenges. In _17th International Conference on Electronic Commerce and Web Technologies (EC-Web 2016)_.
  * Jannach and Zanker (2021) Dietmar Jannach and Markus Zanker. 2021.  Impact and Value of Recommender Systems.  In _Recommender Systems Handbook_ , Francesco Ricci, Bracha Shapira, and Lior Rokach (Eds.). Springer US.
  * Jugovac and Jannach (2017) Michael Jugovac and Dietmar Jannach. 2017.  Interacting with Recommenders - Overview and Research Directions.  _ACM Transactions on Intelligent Interactive Systems (ACM TiiS)_ 7, 3 (2017).
  * Jugovac et al. (2017) Michael Jugovac, Dietmar Jannach, and Lukas Lerche. 2017\.  Efficient Optimization of Multiple Recommendation Quality Factors According to Individual User Tendencies.  _Expert Systems With Applications_ 81 (2017), 321–331.
  * Kaminskas and Bridge (2016) Marius Kaminskas and Derek Bridge. 2016.  Diversity, Serendipity, Novelty, and Coverage: A Survey and Empirical Analysis of Beyond-Accuracy Objectives in Recommender Systems.  _ACM Transactions on Interactive Intelligent Systems_ 7, 1 (2016), 2:1–2:42.
  * Krasnodebski and Dines (2016) Jan Krasnodebski and John Dines. 2016.  Considering Supplier Relations and Monetization in Designing Recommendation Systems. In _Proceedings of the 10th ACM Conference on Recommender Systems_ _(RecSys ’16)_. 381–382.
  * Lee and Hosanagar (2019) Dokyun Lee and Kartik Hosanagar. 2019.  How Do Recommender Systems Affect Sales Diversity? A Cross-Category Investigation via Randomized Field Experiment.  _Information Systems Research_ 30, 1 (2019), 239–259.
  * Li et al. (2010) Lihong Li, Wei Chu, John Langford, and Robert E. Schapire. 2010\.  A Contextual-Bandit Approach to Personalized News Article Recommendation. In _Proceedings of the 19th International Conference on World Wide Web_ _(WWW ’10)_. 661–670.
  * Li et al. (2011) Lihong Li, Wei Chu, John Langford, and Xuanhui Wang. 2011\.  Unbiased Offline Evaluation of Contextual-Bandit-Based News Article Recommendation Algorithms. In _Proceedings of the Fourth ACM International Conference on Web Search and Data Mining_ (Hong Kong, China) _(WSDM ’11)_. 297–306.
  * Manouselis and Costopoulou (2007) Nikos Manouselis and Constantina I. Costopoulou. 2007\.  Analysis and Classification of Multi-Criteria Recommender Systems.  _World Wide Web_ 10, 4 (2007), 415–441.
  * Masthoff (2004) Judith Masthoff. 2004\.  Group Modeling: Selecting a Sequence of Television Items to Suit a Group of Viewers.  In _Personalized Digital Television: Targeting Programs to individual Viewers_. Springer Netherlands, 93–141.
  * Masthoff (2015) Judith Masthoff. 2015\.  Group Recommender Systems: Aggregation, Satisfaction and Group Attributes.  In _Recommender Systems Handbook_ , Francesco Ricci, Lior Rokach, and Bracha Shapira (Eds.). Springer.
  * McInerney et al. (2021) James McInerney, Ehtsham Elahi, Justin Basilico, Yves Raimond, and Tony Jebara. 2021.  Accordion: A Trainable Simulator for Long-Term Interactive Systems. In _Fifteenth ACM Conference on Recommender Systems_ _(RecSys ’21)_. 102–113.
  * McInerney et al. (2018) James McInerney, Benjamin Lacker, Samantha Hansen, Karl Higley, Hugues Bouchard, Alois Gruson, and Rishabh Mehrotra. 2018.  Explore, Exploit, and Explain: Personalizing Explainable Recommendations with Bandits. In _Proceedings of the 12th ACM Conference on Recommender Systems_ (Vancouver, British Columbia, Canada) _(RecSys ’18)_. 31–39.
  * McNee et al. (2006) Sean M. McNee, John Riedl, and Joseph A. Konstan. 2006\.  Being Accurate is Not Enough: How Accuracy Metrics Have Hurt Recommender Systems. In _CHI ’06 Extended Abstracts on Human Factors in Computing Systems_ _(CHI EA ’06)_. 1097–1101.
  * Mehrotra et al. (2018) Rishabh Mehrotra, James McInerney, Hugues Bouchard, Mounia Lalmas, and Fernando Diaz. 2018\.  Towards a Fair Marketplace: Counterfactual Evaluation of the Trade-off between Relevance, Fairness & Satisfaction in Recommendation Systems. In _Proceedings of the 27th ACM International Conference on Information and Knowledge Management_ _(CIKM ’18)_. 2243–2251.
  * Nunes and Jannach (2017) Ingrid Nunes and Dietmar Jannach. 2017.  A Systematic Review and Taxonomy of Explanations in Decision Support and Recommender Systems.  _User Modeling and User-Adapted Interaction_ 27, 3–5 (Dec. 2017), 393–444.
  * O’Connor et al. (2001) Mark O’Connor, Dan Cosley, Joseph A. Konstan, and John Riedl. 2001.  _PolyLens: A Recommender System for Groups of Users_.  Springer Netherlands, 199–218.
  * Oh et al. (2011) Jinoh Oh, Sun Park, Hwanjo Yu, Min Song, and Seung-Taek Park. 2011.  Novel Recommendation Based on Personal Popularity Tendency. In _ICDM ’11_. 507–516.
  * Palomares et al. (2021) Iván Palomares, Carlos Porcel, Luiz Pizzato, Ido Guy, and Enrique Herrera-Viedma. 2021.  Reciprocal Recommender Systems: Analysis of state-of-art literature, challenges and opportunities towards social recommendation.  _Information Fusion_ 69 (2021), 103–127.
  * Panniello et al. (2016) Umberto Panniello, Shawndra Hill, and Michele Gorgoglione. 2016.  The Impact of Profit Incentives on the Relevance of Online Recommendations.  _Electron. Commer. Rec. Appl._ 20, C (2016), 87–104.
  * Paymans et al. (2004) Tim F. Paymans, Jasper Lindenberg, and Mark Neerincx. 2004\.  Usability Trade-Offs for Adaptive User Interfaces: Ease of Use and Learnability. In _Proceedings of the 9th International Conference on Intelligent User Interfaces_ _(IUI ’04)_. 301–303.
  * Pu et al. (2011) Pearl Pu, Li Chen, and Rong Hu. 2011.  A User-centric Evaluation Framework for Recommender Systems. In _RecSys ’11_. 157–164.
  * Resnick et al. (1994) Paul Resnick, Neophytos Iacovou, Mitesh Suchak, Peter Bergstrom, and John Riedl. 1994.  GroupLens: An Open Architecture for Collaborative Filtering of Netnews. In _Proceedings of the 1994 ACM Conference on Computer Supported Cooperative Work_ _(CSCW ’94)_. 175–186.
  * Ribeiro et al. (2015) Marco Tulio Ribeiro, Nivio Ziviani, Edleno Silva De Moura, Itamar Hata, Anisio Lacerda, and Adriano Veloso. 2015.  Multiobjective Pareto-Efficient Approaches for Recommender Systems.  _ACM Trans. Intell. Syst. Technol._ 5, 4 (2015).
  * Rodriguez et al. (2012) Mario Rodriguez, Christian Posse, and Ethan Zhang. 2012\.  Multiple Objective Optimization in Recommender Systems. In _Proceedings of the Sixth ACM Conference on Recommender Systems_ _(RecSys ’12)_. 11–18.
  * Rohde et al. (2018) David Rohde, Stephen Bonner, Travis Dunlop, Flavian Vasile, and Alexandros Karatzoglou. 2018\.  RecoGym: A Reinforcement Learning Environment for the problem of Product Recommendation in Online Advertising.  _arXiv preprint arXiv:1808.00720_ (2018).
  * Said et al. (2013) Alan Said, Ben Fields, Brijnesh J. Jain, and Sahin Albayrak. 2013\.  User-Centric Evaluation of a K-Furthest Neighbor Collaborative Filtering Recommender Algorithm. In _Proceedings of the 2013 Conference on Computer Supported Cooperative Work_ _(CSCW ’13)_. 1399–1408.
  * Selbst et al. (2019) Andrew D. Selbst, Danah Boyd, Sorelle A. Friedler, Suresh Venkatasubramanian, and Janet Vertesi. 2019\.  Fairness and Abstraction in Sociotechnical Systems. In _Proceedings of the Conference on Fairness, Accountability, and Transparency_ _(FAT* ’19)_. 59–68.
  * Shardanand and Maes (1995) Upendra Shardanand and Pattie Maes. 1995.  Social Information Filtering: Algorithms for Automating “Word of Mouth”. In _Proceedings of the SIGCHI Conference on Human Factors in Computing Systems_ _(CHI ’95)_. 210–217.
  * Steck (2018) Harald Steck. 2018\.  Calibrated Recommendations. In _ACM RecSys ’18_. 154–162.
  * Steck (2019) Harald Steck. 2019\.  Embarrassingly Shallow Autoencoders for Sparse Data. In _The World Wide Web Conference_ _(WWW ’19)_. 3251–3257.
  * Steck et al. (2021) Harald Steck, Linas Baltrunas, Ehtsham Elahi, Dawen Liang, Yves Raimond, and Justin Basilico. 2021\.  Deep Learning for Recommender Systems: A Netflix Case Study.  _AI Mag._ 42, 3 (2021), 7–18.
  * Theocharous et al. (2015) Georgios Theocharous, Philip S. Thomas, and Mohammad Ghavamzadeh. 2015.  Personalized Ad Recommendation Systems for Life-Time Value Optimization with Guarantees. In _Proceedings of the 24th International Conference on Artificial Intelligence_ _(IJCAI’15)_. 1806–1812.
  * Tintarev and Masthoff (2011) Nava Tintarev and Judith Masthoff. 2011.  Designing and evaluating explanations for recommender systems.  In _Recommender systems handbook_. Springer, 479–510.
  * Triantaphyllou (2000) Evangelos Triantaphyllou. 2000\.  Multi-Criteria Decision Making Methods.  In _Multi-criteria Decision Making Methods: A Comparative Study_. 5–21.
  * Vargas and Castells (2011) Saúl Vargas and Pablo Castells. 2011.  Rank and Relevance in Novelty and Diversity Metrics for Recommender Systems. In _RecSys ’11_. 109–116.
  * Wang and Wu (2009) Hsiao-Fan Wang and Cheng-Ting Wu. 2009.  A mathematical model for product selection strategies in a recommender system.  _Expert Systems with Applications_ 36, 3, Part 2 (2009), 7299–7308.
  * Wang et al. (2022) Yuyan Wang, Mohit Sharma, Can Xu, Sriraj Badam, Qian Sun, Lee Richardson, Lisa Chung, Ed H. Chi, and Minmin Chen. 2022.  Surrogate for Long-Term User Experience in Recommender Systems. In _Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining_ _(KDD ’22)_. 4100–4109.
  * Zhang et al. (2019) Jingjing Zhang, Gediminas Adomavicius, Alok Gupta, and Wolfgang Ketter. 2019.  Consumption and Performance: Understanding Longitudinal Dynamics of Recommender Systems via an Agent-Based Simulation Framework.  _Information Systems Research_ forthcoming (2019).
  * Zhang and Hurley (2008) Mi Zhang and Neil Hurley. 2008.  Avoiding Monotony: Improving the Diversity of Recommendation Lists. In _Proceedings of the 2008 ACM Conference on Recommender Systems_ _(RecSys ’08)_. 123–130.
  * Zhang et al. (2012) Yuan Cao Zhang, Diarmuid Ó Séaghdha, Daniele Quercia, and Tamas Jambor. 2012\.  Auralist: Introducing Serendipity into Music Recommendation. In _Proceedings of the Fifth ACM International Conference on Web Search and Data Mining_ _(WSDM ’12)_. 13–22.
  * Zhou et al. (2010) Tao Zhou, Zoltán Kuscsik, Jian-Guo Liu, Matúš Medo, Joseph Rushton Wakeling, and Yi-Cheng Zhang. 2010.  Solving the apparent diversity-accuracy dilemma of recommender systems.  _Proceedings of the National Academy of Sciences_ 107, 10 (2010), 4511–4515.


$\frac{G_{2}\left(\frac{1}{3},\frac{\tau}{273},\frac{\sigma}{273},\frac{\rho}{273}\right)^{13}}{G_{2}\left(\frac{13}{3},\frac{\tau}{21},\frac{\sigma}{21},\frac{\rho}{21}\right)}\times\frac{G_{2}\left(\frac{1}{3},\frac{\tau^{\prime}}{278031},\frac{\sigma^{\prime}}{278031},\frac{\rho^{\prime}}{278031}\right)^{-13}}{G_{2}\left(\frac{13}{3},\frac{\tau^{\prime}}{21387},\frac{\sigma^{\prime}}{21387},\frac{\rho^{\prime}}{21387}\right)^{-1}}$

is close to the root $\approx 10.6409709...-i\cdot 5.9332732...$ of the
polynomial $x^{8}-18x^{7}+83x^{6}+396x^{5}+597x^{4}+396x^{3}+83x^{2}-18x+1$
which defines a subextension of
$\mathbb{K}_{8}^{+}(\mathfrak{f})/\mathbb{K}_{8}$. The computation time for
$1000$ digits is 2 minutes and 15 seconds.

####  5.2.3 Example 9

Keep $\mathbb{K}_{8}$, $\mathfrak{f}$, $\mathfrak{b}$, $\mathfrak{a}$ as in
example 8 and let us change our choice of fundamental units. We fix another
set of fundamental units for $\mathcal{O}^{+,\times}_{\mathfrak{f}}$:

$\varepsilon_{1}=\frac{z^{3}-21z^{2}+54z+11}{9},\leavevmode\nobreak\
\varepsilon_{2}=-z^{3}+19z^{2}-18z-8$

Then, we compute $\tilde{\lambda}_{1}=\tilde{\lambda}_{2}=1$,
$\tilde{t}_{1}=1$, $\tilde{t}_{2}=25$. In the second computation, a target
ideal is $\mathfrak{P}_{5}^{2}$ the square of the unique degree one prime
above $5$ and the ideals $qN/\mathfrak{a}$ and
$qN\mathfrak{P}_{5}^{2}/\mathfrak{a}$ are generated by the admissible vectors
$h_{1}=(-44z^{3}+843z^{2}-927z-232)/3$,
$h_{2}=(76z^{3}-1449z^{2}+1470z+344)/3$. The corresponding levels will be
$l_{1}=3\cdot 13$ and $l_{2}=3\cdot 5^{2}\cdot 13$. Computations show that the
correct orientations should be $\underline{\mu}=[1,1]$ and
$\underline{\nu}=[-1,1]$. Let us write the parameters

$\displaystyle\tau=\frac{2\varepsilon_{1}\varepsilon_{2}+109\varepsilon_{1}-3}{\varepsilon_{1}},$
$\displaystyle\tau^{\prime}=\frac{842\varepsilon_{1}\varepsilon_{2}+\varepsilon_{2}+3}{\varepsilon_{1}\varepsilon_{2}}$
$\displaystyle\sigma=\frac{-321\varepsilon_{1}\varepsilon_{2}-16\varepsilon_{1}+7}{\varepsilon_{1}\varepsilon_{2}},$
$\displaystyle\sigma^{\prime}=\frac{-7\varepsilon_{1}\varepsilon_{2}+4\varepsilon_{2}-63}{1}$
$\displaystyle\rho=\frac{-\varepsilon_{1}-47}{1},$
$\displaystyle\rho^{\prime}=\frac{-3\varepsilon_{1}\varepsilon_{2}+566\varepsilon_{2}-2}{\varepsilon_{2}}$

Then

$\frac{G_{2}\left(\frac{1}{3},\frac{\tau}{39},\frac{\sigma}{39},\frac{\rho}{39}\right)^{-13}}{G_{2}\left(\frac{13}{3},\frac{\tau}{3},\frac{\sigma}{3},\frac{\rho}{3}\right)^{-1}}\times\frac{G_{2}\left(\frac{1}{3},\frac{\tau^{\prime}}{975},\frac{\sigma^{\prime}}{975},\frac{\rho^{\prime}}{975}\right)^{13}}{G_{2}\left(\frac{13}{3},\frac{\tau^{\prime}}{75},\frac{\sigma^{\prime}}{75},\frac{\rho^{\prime}}{75}\right)}$

is close to the same root $\approx 10.6409709...-i\cdot 5.9332732...$ of the
polynomial $x^{8}-18x^{7}+83x^{6}+396x^{5}+597x^{4}+396x^{3}+83x^{2}-18x+1$.
The computation time for $1000$ digits is 11 seconds. We see that a good
choice for the fundamental units can have a strong impact on the computation
time.

####  5.2.4 Example 10

This example falls outside of ($\ast$) and ($\ast\ast$) because
$gcd(\tilde{t}_{1},\tilde{t}_{2},q=2)=2$. Still, we can find a pair of vectors
$h_{1},h_{2}$ for which the computations give interesting results. Let $z$ be
the complex root of the polynomial $x^{4}-12$ lying in the upper half-plane.
Then $\mathbb{K}_{10}=\mathbb{Q}(z)$ has class number $1$ and contains the
real quadratic field $\mathbb{Q}(\sqrt{3})$. Remember that this means that we
must carefully choose the fundamental units in this case to avoid totally real
elements. We choose $\mathfrak{f}^{4}=(2)$. We choose the fundamental units

$\varepsilon_{1}=\frac{z^{2}+2z+2}{4},\leavevmode\nobreak\
\varepsilon_{2}=\frac{-z^{3}+z^{2}+4z-2}{4}$

for $\mathcal{O}^{+,\times}_{\mathfrak{f}}$. We compute
$\tilde{\lambda}_{1}=\tilde{\lambda}_{2}=1$, $\tilde{t}_{1}=2\cdot 3$,
$\tilde{t}_{2}=2\cdot 11$. We may choose $\mathfrak{b}=(1)$ and
$\mathfrak{a}=\mathfrak{P}_{23}$ a degree one prime above $23$ in
$\mathbb{K}_{10}$. We search for target ideals and find $\mathfrak{P}_{3}$ the
degree one prime ideal above $3$ in the first case and $\mathfrak{P}_{11}$ one
of the degree one primes above $11$ in the second case. Then
$qN\mathfrak{P}_{3}/\mathfrak{a}$ and $qN\mathfrak{P}_{11}/\mathfrak{a}$ are
generated by the admissible vectors $h_{1}=(-4z^{3}+11z^{2}+10z+30)/2$ and
$h_{2}=(9z^{3}+4z^{2}-34z-56)/2$. The corresponding levels will be
$l_{1}=2\cdot 3\cdot 23$ and $l_{2}=2\cdot 11\cdot 23$. Fixing the
orientations $\underline{\mu}=[1,-1],\underline{\nu}=[-1,1]$ we compute
$I_{2,\mathfrak{f},\mathfrak{b},\mathfrak{a}}(\varepsilon_{1},\varepsilon_{2},\underline{h},\underline{\mu},\underline{\nu},\sigma_{\mathbb{C}})\approx
13.9102308...-i\cdot 24.0932265...$ with $t_{1}=t_{2}=2$, i.e. we compute a
product of 8 ordinary elliptic smoothed $G_{2}$ functions. The result is close
to a root of the polynomial
$x^{8}-28x^{7}+778x^{6}-112x^{5}-749x^{4}-112x^{3}+778x^{2}-28x+1$ which
defines $\mathbb{K}_{10}^{+}(\mathfrak{f})$. The computation time for $1000$
digits is 17 seconds.

###  5.3 One Quintic example

We have already successfully produced a dozen of quintic examples and we give
here a detailed presentation of our simplest one. Let $z$ be the complex root
of the polynomial $x^{5}-x^{4}-x^{3}-2x^{2}+x+1$ lying in the upper half-
plane. Then $\mathbb{K}_{11}=\mathbb{Q}(z)$ has class number $1$. We choose
$\mathfrak{f}=\mathfrak{P}_{3}$ the degree one prime above $3$. We choose the
fundamental units (once again take the fundamental units provided by Pari/GP
and then compute the simplest totally positive ones):

$\varepsilon_{1}=z^{4}-2z^{3}-z+3,\leavevmode\nobreak\
\varepsilon_{2}=2z^{4}-2z^{3}-3z+3,\leavevmode\nobreak\
\varepsilon_{3}=2z^{4}-3z^{3}-4z+4$

for $\mathcal{O}^{+,\times}_{\mathfrak{f}}$. We fix the order
$\\{Id,(32),(21),(231),(312),(31)\\}$ of $\mathfrak{S}_{3}$ given by Pari/GP.
We compute
$\tilde{\lambda}_{1}=\tilde{\lambda}_{2}=\tilde{\lambda}_{3}=\tilde{\lambda}_{4}=\tilde{\lambda}_{5}=\tilde{\lambda}_{6}=1$,
$\tilde{t}_{1}=7\cdot 37\cdot 137$, $\tilde{t}_{2}=31\cdot 53$,
$\tilde{t}_{3}=491$, $\tilde{t}_{4}=107$, $\tilde{t}_{5}=1$ and
$\tilde{t}_{6}=145637$ where we have written the prime number decomposition of
all the overflows $\tilde{t}$. We may choose $\mathfrak{b}=(1)$ and
$\mathfrak{a}$ the unique degree one prime above $11$ in $\mathbb{K}_{11}$. We
look for target ideals above the prime numbers listed above and find the
ideals
$\mathfrak{P}_{7},\mathfrak{P}^{\prime}_{37},\mathfrak{P}^{\prime}_{137},\mathfrak{P}_{31},\mathfrak{P}_{53},\mathfrak{P}^{\prime}_{491},\mathfrak{P}^{\prime}_{107},\mathfrak{P}_{145637}$
in the order provided by Pari/GP version 2.15.4. The ideals
$qN\mathfrak{P}_{7}\mathfrak{P}^{\prime}_{37}\mathfrak{P}^{\prime}_{137}/\mathfrak{a}$,
$qN\mathfrak{P}_{31}\mathfrak{P}_{53}/\mathfrak{a}$,
$qN\mathfrak{P}^{\prime}_{491}/\mathfrak{a}$,
$qN\mathfrak{P}^{\prime}_{107}/\mathfrak{a}$, $qN/\mathfrak{a}$,
$qN\mathfrak{P}_{145637}/\mathfrak{a}$ admit the following admissible
generators:

$\displaystyle h_{1}$ $\displaystyle=147z^{4}-135z^{3}-90z^{2}-234z+72$
$\displaystyle h_{2}$ $\displaystyle=-15z^{4}-30z^{3}+90z^{2}+36z+60$
$\displaystyle h_{3}$ $\displaystyle=42z^{4}-15z^{3}-87z^{2}-48z+30$
$\displaystyle h_{4}$ $\displaystyle=-21z^{4}+57z^{3}-6z^{2}-9z-48$
$\displaystyle h_{5}$ $\displaystyle=3z^{4}-6z^{3}+18z^{2}-6z+12$
$\displaystyle h_{6}$ $\displaystyle=108z^{4}-246z^{3}+111z^{2}-81z+195$

The corresponding levels are $l_{1}=3\cdot 11\cdot 7\cdot 37\cdot 137$,
$l_{2}=3\cdot 11\cdot 491$, $l_{3}=3\cdot 11\cdot 107$, $l_{4}=3\cdot 11\cdot
31\cdot 53$, $l_{5}=3\cdot 11$, $l_{6}=3\cdot 11\cdot 145637$. Computations
give the orientations
$\underline{\mu}=[1,-1,-1,1,1,-1],\underline{\nu}=[1,-1,-1,1,1,-1]$. We
compute the six quotients:

$\displaystyle\frac{G_{3}\left(\frac{-1}{3},\frac{\tau_{1}}{l_{1}},\frac{\sigma_{1}}{l_{1}},\frac{\rho_{1}}{l_{1}},\frac{\varpi_{1}}{l_{1}}\right)^{11}}{G_{3}\left(\frac{-11}{3},\frac{11\tau_{1}}{l_{1}},\frac{11\sigma_{1}}{l_{1}},\frac{11\rho_{1}}{l_{1}},\frac{11\varpi_{1}}{l_{1}}\right)}$
$\displaystyle\approx-6.9846353...10^{-18}+i\cdot 6.2764063...10^{-17}$
$\displaystyle\frac{G_{3}\left(\frac{-1}{3},\frac{\tau_{2}}{l_{2}},\frac{\sigma_{2}}{l_{2}},\frac{\rho_{2}}{l_{2}},\frac{\varpi_{2}}{l_{2}}\right)^{-11}}{G_{3}\left(\frac{-11}{3},\frac{11\tau_{2}}{l_{2}},\frac{11\sigma_{2}}{l_{2}},\frac{11\rho_{2}}{l_{2}},\frac{11\varpi_{1}}{l_{2}}\right)^{-1}}$
$\displaystyle\approx 1.7434761...10^{11}-i\cdot 1.7914686...10^{11}$
$\displaystyle\frac{G_{3}\left(\frac{1}{3},\frac{\tau_{3}}{l_{3}},\frac{\sigma_{3}}{l_{3}},\frac{\rho_{3}}{l_{3}},\frac{\varpi_{3}}{l_{3}}\right)^{-11}}{G_{3}\left(\frac{11}{3},\frac{11\tau_{3}}{l_{3}},\frac{11\sigma_{3}}{l_{3}},\frac{11\rho_{3}}{l_{3}},\frac{11\varpi_{3}}{l_{3}}\right)^{-1}}$
$\displaystyle\approx 0.2815024...-i\cdot 0.0665163...$
$\displaystyle\frac{G_{3}\left(\frac{1}{3},\frac{\tau_{4}}{l_{4}},\frac{\sigma_{4}}{l_{4}},\frac{\rho_{4}}{l_{4}},\frac{\varpi_{4}}{l_{4}}\right)^{11}}{G_{3}\left(\frac{11}{3},\frac{11\tau_{4}}{l_{4}},\frac{11\sigma_{4}}{l_{4}},\frac{11\rho_{4}}{l_{4}},\frac{11\varpi_{4}}{l_{4}}\right)}$
$\displaystyle\approx 9.8457700...-i\cdot 2.3506772...$
$\displaystyle\frac{G_{3}\left(\frac{-1}{3},\frac{\tau_{5}}{l_{5}},\frac{\sigma_{5}}{l_{5}},\frac{\rho_{5}}{l_{5}},\frac{\varpi_{5}}{l_{5}}\right)^{11}}{G_{3}\left(\frac{-11}{3},\frac{11\tau_{5}}{l_{5}},\frac{11\sigma_{5}}{l_{5}},\frac{11\rho_{5}}{l_{5}},\frac{11\varpi_{5}}{l_{5}}\right)}$
$\displaystyle\approx-0.2518907...-i\cdot 0.2274847...$
$\displaystyle\frac{G_{3}\left(\frac{-1}{3},\frac{\tau_{6}}{l_{6}},\frac{\sigma_{6}}{l_{6}},\frac{\rho_{6}}{l_{6}},\frac{\varpi_{6}}{l_{6}}\right)^{-11}}{G_{3}\left(\frac{-11}{3},\frac{11\tau_{6}}{l_{6}},\frac{11\sigma_{6}}{l_{6}},\frac{11\rho_{6}}{l_{6}},\frac{11\varpi_{6}}{l_{6}}\right)^{-1}}$
$\displaystyle\approx 100864.0193260...-i\cdot 767246.5816458...$

where the parameters are given in the table below. The product of these six
quotients is close to the root $\approx-11.6360077...+i\cdot 3.4634701...$ of
the polynomial
$x^{10}+24x^{9}+164x^{8}+99x^{7}-62x^{6}-89x^{5}-62x^{4}+99x^{3}+164x^{2}+24x+1$
which defines $\mathbb{K}_{11}^{+}(\mathfrak{f})$. The computation time for
$1000$ digits is 1 minute and 35 seconds, but the computation time for each of
the individual computations is not uniform at all. The fifth computation
requires 1 second whereas the sixth computation requires 58 seconds because of
the level difference $l_{5}=33$ versus $l_{6}=4806021$.

$\varepsilon_{1}\tau_{1}\hfill=$ | $229\varepsilon_{1}\varepsilon_{2}\varepsilon_{3}$ | $-1727\varepsilon_{1}\varepsilon_{2}$ | $-796386\varepsilon_{1}$ | $+3385$
---|---|---|---|---
$\varepsilon_{1}\varepsilon_{2}\sigma_{1}\hfill=$ | $-114\varepsilon_{1}\varepsilon_{2}\varepsilon_{3}$ | $-987551\varepsilon_{1}\varepsilon_{2}$ | $+718\varepsilon_{1}$ | $+949$
$\varepsilon_{1}\varepsilon_{2}\varepsilon_{3}\rho_{1}\hfill=$ | $-595138\varepsilon_{1}\varepsilon_{2}\varepsilon_{3}+$ | $12\varepsilon_{1}\varepsilon_{2}$ | $+1445\varepsilon_{1}$ | $-907$
$\varpi_{1}\hfill=$ | $-52\varepsilon_{1}\varepsilon_{2}\varepsilon_{3}$ | $+857\varepsilon_{1}\varepsilon_{2}$ | $-295\varepsilon_{1}$ | $+2067162$
$\varepsilon_{1}\tau_{2}\hfill=$ | $99\varepsilon_{1}\varepsilon_{2}\varepsilon_{3}$ | $-356\varepsilon_{1}\varepsilon_{3}$ | $+19304\varepsilon_{1}$ | $+57$
$\varepsilon_{1}\varepsilon_{3}\sigma_{2}\hfill=$ | $-17\varepsilon_{1}\varepsilon_{2}\varepsilon_{3}$ | $-70870\varepsilon_{1}\varepsilon_{3}$ | $-145\varepsilon_{1}$ | $+40$
$\varepsilon_{1}\varepsilon_{2}\varepsilon_{3}\rho_{2}\hfill=$ | $324170\varepsilon_{1}\varepsilon_{2}\varepsilon_{3}$ | $-85\varepsilon_{1}\varepsilon_{3}$ | $+197\varepsilon_{1}$ | $-111$
$\varpi_{2}\hfill=$ | $91\varepsilon_{1}\varepsilon_{2}\varepsilon_{3}$ | $-460\varepsilon_{1}\varepsilon_{3}$ | $+3\varepsilon_{1}$ | $-28725$
$\varepsilon_{1}\varepsilon_{2}\tau_{4}\hfill=$ | $-62\varepsilon_{1}\varepsilon_{2}\varepsilon_{3}$ | $-12636\varepsilon_{1}\varepsilon_{2}$ | $+20\varepsilon_{2}$ | $+3$
$\varepsilon_{1}\varepsilon_{2}\varepsilon_{3}\sigma_{4}\hfill=$ | $327915\varepsilon_{1}\varepsilon_{2}\varepsilon_{3}$ | $-37\varepsilon_{1}\varepsilon_{2}$ | $-1101\varepsilon_{2}$ | $+154$
$\rho_{4}\hfill=$ | $-44\varepsilon_{1}\varepsilon_{2}\varepsilon_{3}$ | $+314\varepsilon_{1}\varepsilon_{2}$ | $-65\varepsilon_{2}$ | $-52915$
$\varepsilon_{2}\varpi_{4}\hfill=$ | $-262\varepsilon_{1}\varepsilon_{2}\varepsilon_{3}$ | $-5\varepsilon_{1}\varepsilon_{2}$ | $-768869\varepsilon_{2}$ | $-19$
$\varepsilon_{2}\varepsilon_{3}\tau_{3}\hfill=$ | $-27\varepsilon_{1}\varepsilon_{2}\varepsilon_{3}$ | $-84\varepsilon_{2}\varepsilon_{3}$ | $+11\varepsilon_{2}$ | $+1$
$\varepsilon_{1}\varepsilon_{2}\varepsilon_{3}\sigma_{3}\hfill=$ | $2555\varepsilon_{1}\varepsilon_{2}\varepsilon_{3}$ | $+19\varepsilon_{2}\varepsilon_{3}$ | $-391\varepsilon_{2}$ | $+52$
$\rho_{3}\hfill=$ | $-11\varepsilon_{1}\varepsilon_{2}\varepsilon_{3}$ | $+49\varepsilon_{2}\varepsilon_{3}$ | $-51\varepsilon_{2}$ | $-491$
$\varepsilon_{2}\varpi_{3}\hfill=$ | $-80\varepsilon_{1}\varepsilon_{2}\varepsilon_{3}$ | $-23\varepsilon_{2}\varepsilon_{3}$ | $-1402\varepsilon_{2}$ | $-1$
$\varepsilon_{1}\varepsilon_{2}\varepsilon_{3}\tau_{5}\hfill=$ | $-456\varepsilon_{1}\varepsilon_{2}\varepsilon_{3}$ | $-7\varepsilon_{1}\varepsilon_{3}$ | $-2\varepsilon_{3}$ | $+3$
$\sigma_{5}\hfill=$ | $-4\varepsilon_{1}\varepsilon_{2}\varepsilon_{3}$ | $+17\varepsilon_{1}\varepsilon_{3}$ | $+12\varepsilon_{3}$ | $+566$
$\varepsilon_{3}\rho_{5}\hfill=$ | $9\varepsilon_{1}\varepsilon_{2}\varepsilon_{3}$ | $-39\varepsilon_{1}\varepsilon_{3}$ | $+1265\varepsilon_{3}$ | $+13$
$\varepsilon_{1}\varepsilon_{3}\varpi_{5}\hfill=$ | $\varepsilon_{1}\varepsilon_{2}\varepsilon_{3}$ | $-252\varepsilon_{1}\varepsilon_{3}$ | $-3\varepsilon_{3}$ | $+2$
$\varepsilon_{1}\varepsilon_{2}\varepsilon_{3}\tau_{6}\hfill=$ | $9785210\varepsilon_{1}\varepsilon_{2}\varepsilon_{3}$ | $+2609\varepsilon_{2}\varepsilon_{3}$ | $-523\varepsilon_{3}$ | $-729$
$\sigma_{6}\hfill=$ | $479\varepsilon_{1}\varepsilon_{2}\varepsilon_{3}$ | $+987\varepsilon_{2}\varepsilon_{3}$ | $-13874\varepsilon_{3}$ | $-19886676$
$\varepsilon_{3}\rho_{6}\hfill=$ | $1713\varepsilon_{1}\varepsilon_{2}\varepsilon_{3}$ | $-8328\varepsilon_{2}\varepsilon_{3}$ | $-13204073\varepsilon_{3}$ | $+764$
$\varepsilon_{2}\varepsilon_{3}\varpi_{6}\hfill=$ | $4002\varepsilon_{1}\varepsilon_{2}\varepsilon_{3}$ | $+7773526\varepsilon_{2}\varepsilon_{3}$ | $+1749\varepsilon_{3}$ | $-2296$

## References

  * [1] N. Bergeron, P. Charollois, and L. E. García.  Elliptic units for complex cubic fields.  https://arxiv.org/abs/2311.04110, 2023.
  * [2] M. Bhargava.  Higher composition laws III: the parametrization of quartic rings.  Annals of Mathematics, 159:1329–1360, 2004.
  * [3] P. Charollois and H. Darmon.  Arguments des unités de Stark et périodes de séries d’Eisenstein.  Algebra & Number Theory, 2.6:655–688, 2008.
  * [4] H. Cohen.  A Course in Computational Algebraic Number Theory.  Springer-Verlag, 1993.
  * [5] D. A. Cox.  Primes of the form $x^{2}+ny^{2}$.  John Wiley & Sons, Ltd, 2013.
  * [6] S. Dasgupta and M. Greenberg.  Lecture notes on the Rank One Abelian Stark Conjecture, Arizona Winter School, 2011.
  * [7] S. Dasgupta and M. Kakde.  Brumer-Stark Units and Explicit Class Field Theory.  Duke Math., 2023.
  * [8] G. Eisenstein.  Beiträge zur Theorie der elliptischen Functionen, part VI.  Crelles Journal, 35:153–274, 1847.
  * [9] G. Felder, A. Henriques, C. A. Rossi, and C. Zhu.  A gerbe for the elliptic gamma function.  Duke Math., 141:1–74, 2008.
  * [10] G. Felder and A. Varchenko.  The elliptic gamma function and $\mathrm{SL}(3,\mathbb{Z})\ltimes\mathbb{Z}^{3}$.  Adv. Math., 156:44–76, 2000.
  * [11] The LMFDB Collaboration.  The L-functions and modular forms database.  https://www.lmfdb.org, 2024.
  * [12] A. Narukawa.  The modular properties and the integral representations of the multiple elliptic gamma functions.  Adv. Math., 189(2):247–267, 2004.
  * [13] M. Nishizawa.  An elliptic analogue of the multiple gamma function.  J. Phys. A: Math. Gen., 34:7411–7421, 2001.
  * [14] Vicenţiu Paşol and Wadim Zudilin.  A study of elliptic gamma function and allies.  Research in the Mathematical Sciences, 5(4), 2018.
  * [15] A. R. Rankin.  Modular Forms and Functions.  Cambridge University Press, 1977.
  * [16] T. Ren and R. Sczech.  A refinement of Stark’s conjecture over complex cubic number fields.  Journal of number theory, 129:831–857, 2009.
  * [17] G. Robert.  Unités elliptiques.  Bull. Soc. Math. France, Mémoire 36, 1973.
  * [18] S. N. M. Ruijsenaars.  First order analytic difference equations and integrable quantum systems.  J. Math. Phys, 38(2):1069–1146, 1997.
  * [19] Sczech.  Eisenstein group cocycles for $\mathrm{GL}_{n}$ and values of $l$-functions.  Inventiones mathematicae, 113:581–616, 1993.
  * [20] H. Stark.  L-functions at s = 1. IV. First Derivatives at s = 0.  Advances in Mathematics, 35:197–235, 1980.
  * [21] The PARI Group, Univ. Bordeaux.  PARI/GP version 2.15.4, 2023.  available from http://pari.math.u-bordeaux.fr/.
  * [22] A. Weil.  Elliptic functions according to Eisenstein and Kronecker, volume 88 of Ergebnisse der Mathematik und ihrer Grenzgebiete.  Springer-Verlag, first edition, 1976.
  * [23] Jacob Winding.  Multiple elliptic gamma functions associated to cones.  Adv. Math., 325:56–86, 2018.