# The Initial Screening Order Problem

Jose M. Alvarez  0000-0001-9412-9013<EMAIL_ADDRESS>Scuola Normale
Superiore, University of PisaPisaItaly ,  Antonio Mastropietro
0000-0002-8823-0163 University of PisaPisaItaly and  Salvatore Ruggieri
0000-0002-1917-6087<EMAIL_ADDRESS>University of PisaPisaItaly

(2024)

###### Abstract.

We investigate the role of the initial screening order (ISO) in candidate
screening processes, such as employee hiring and academic admissions. The ISO
refers to the order in which the screener evaluates the candidate pool. It has
been largely overlooked in the literature, despite its potential impact on the
optimality and fairness of the chosen set, especially under a human screener.
We define two problem formulations: the best-$k$, where the screener selects
the $k$ best candidates, and the good-$k$, where the screener selects the $k$
first good-enough candidates. To study the impact of the ISO, we introduce a
human-like screener and compare it to its algorithmic counterpart. The human-
like screener is conceived to be inconsistent over time due to fatigue. Our
analysis shows that the ISO, in particular, under a human-like screener
hinders individual fairness despite meeting group level fairness. This is due
to the position bias, where a candidate’s evaluation is affected by its
position within the ISO. We report extensive simulated experiments exploring
the parameters of the best-$k$ and good-$k$ problem formulations both for the
algorithmic and human-like screeners. This work is motivated by a real world
candidate screening problem studied in collaboration with a large European
company.

fair set selection, position bias, candidate screening, human-in-the-loop

††copyright: acmlicensed††journalyear: 2024††doi: XXXXXXX.XXXXXXX††conference:
2024 ACM Conference on Equity and Access in Algorithms, Mechanisms, and
Optimization; October 29–31, 2024; San Luis Potosí, Mexico††booktitle: 2024
ACM Conference on Equity and Access in Algorithms, Mechanisms, and
Optimization (EAAMO ’24), October 29–31, 2024, San Luis Potosí, Mexico††ccs:
Information systems Retrieval models and ranking††ccs: Human-centered
computing Interaction design theory, concepts and paradigms

##  1\. Introduction

Candidate screening is a complex, human-dependent process. It consists of a
decision-maker, which we refer to as the screener, tasked with choosing $k$
suitable candidates from a candidate pool. Common candidate screening
processes include the evaluation of resumes for a job interview (Pisanelli,
2022) or application packages for university admission (Sukumar et al., 2018).
The screener usually evaluates the candidate pool using limited information
and under strict time constraints. Because of this, as with other repetitive
tasks performed by humans (Kahneman, 2011; Kahneman et al., 2021), candidate
screening is prone to biased decision-making (Pisanelli, 2022; Echterhoff et
al., 2022). An industry around automating candidate screening using Machine
Learning (ML) has emerged in recent years (Will et al., 2023), though with
limited results in terms of fairness (Wilson et al., 2021; Köchling and
Wehner, 2020; Sonderling et al., 2022; Raghavan et al., 2020; Rhea et al.,
2022; Kazim et al., 2021; Sloane et al., 2022; Fabris et al., 2023). The
future of automated decision-making (ADM) for candidate screening is an open
question as ADM systems for this task are increasingly required by
stakeholders, such as regulators, to consider how humans interact with their
output (Carriço, 2018; Ruggieri et al., 2023).

In this paper, we investigate the role of the initial screening order (ISO) in
the fair set selection problem within candidate screening. The ISO refers to
the order in which the candidates appear in the candidate pool. It can be
chosen by or provided to the screener. We find that the ISO has an impact in
terms of optimality (choosing the best candidates) and fairness (treating
similar candidates similarly) on the selected set of candidates by the
screener, especially when the screener is human (e.g., (Pei et al., 2023;
Echterhoff et al., 2022)). This is mainly because of the position bias
inherent to the ISO. Position bias refers to the penalty (or premium) each
candidate experiences due to where it falls on the ISO (Baeza-Yates, 2018).
Humans are predisposed to favor the items being placed at the top of a list as
we tend to read from top to bottom (Baeza-Yates, 2018; Athey and Ellison,
2011). Clearly, there is a benefit for those candidates that appear at the
start of the ISO. We further motivate the ISO problem in Section 1.1.

Our results are based on a modeling framework for understanding how a screener
interacts with an ISO along with a simulations framework that implements
extensively the problem formulations for practical insights on the influence
of the ISO. We start with Section 2, in which we describe how a screener
explores the ISO in search of an optimal and fair (at the group-level w.r.t. a
representational quota $q$) set of $k$ candidates. We define two problem
formulations. In the best-$k$ the screener selects the $k$ top candidates,
while in the good-$k$ the screener selects the $k$ first good-enough
candidates w.r.t. some minimum score for measuring candidate quality. We
devise algorithmic solutions for both problem formulations. The good-$k$ is
noteworthy as it allows the screener to partially search the candidate pool;
set selection problem formulations often assume a full search by the screener.
We discuss this further in Sections 1.2 and 2.5.

To better understand the impact of the ISO when a human screener is involved,
in Section 3 we introduce and analyse the algorithmic and human-like
screeners. The former refers to a consistent screener (like a ML model); the
latter refers to an inconsistent screener (like a human) whose evaluation of
candidates suffers over time due to fatigue (see, e.g., Kahneman (2011)). We
enhance our analysis of these two screeners in Section 4 through simulated
experiments that mimic multiple screening settings and help us understand the
impact of the ISO. The results, although specific to our assumptions, confirm
the role of position bias inherent to the ISO. We find, for instance, that the
human-like screener can violate individual fairness (Dwork et al., 2012) by
not evaluating similar candidates similarly while still meeting $q$. We also
find that the algorithmic screener can miss the best candidate depending on
its search procedure. In short, we find that where a candidate falls within
the ISO can determine its chances of getting selected, highlighting the
influence of position bias. We conclude in Section 5 by discussing the
limitations of our work, future work, and key takeaways.

Our main contributions are threefold. First, we formalize the role of the ISO
in two possible objectives of the screener with the best-$k$ and good-$k$
problems. Second, we formulate a human-like screener and compare it to its
algorithmic counterpart both theoretically and with experiments. Third, with
the simulations framework, we provide a useful, open source tool for studying
the ISO problem and inform stakeholders without needing to run real world
screening scenarios, which can be both costly and time consuming.

###  1.1. Qualitative Background

Our work borrows from a previous collaborative effort at a European Fortune
Global 500. We refer to this company as G. The purpose was to study G’s hiring
process as an algorithmic fairness problem. We worked closely with Advanced
Analytics (AA) and Human Resources (HR) teams, focusing on candidate
screening. In this phase of G’s hiring process, an HR officer reduces the
candidate pool for a job opening into a smaller pool of suitable candidates,
determining candidate suitability based on each candidate’s profile using a
set of minimum basic requirements. The candidate pool is stored in Taleo by
Oracle, a platform specific to hiring. For more details on our collaboration,
see Appendix A.

Candidate screening at G represented a time-consuming, repetitive task prone
to human error (Kahneman, 2011) and a sensitive, high-risk task requiring
human oversight (Bringas Colmenarejo et al., 2022). Therefore, the option of
full automation was not possible. The focus was, thus, on understanding and
modeling the search of the HR officer since the viable options involved either
the presence or absence of an algorithmic aid. In particular, under the
realistic risk of position bias (Baeza-Yates, 2018) affecting candidate
evaluations, as shown empirically in similar screening settings (Pan et al.,
2007; Grotov et al., 2015; Richardson et al., 2007; Echterhoff et al., 2022;
Pei et al., 2023), we wanted to study the influence of the ISO on the set of
suitable candidates chosen by the HR officer.

The following stylized facts summarize key practices by the HR officers
(henceforth, screeners) that motivate the ISO problem in Sections 2 and 3.
These are: G1 Varying ISOs. Screeners chose the ISO. The choice was restricted
by the sorting fields of the platform, such as using the candidates’ last
name. G2 Two ways to search the candidate pool. Two search practices became
apparent: full or partial search based on the desired number of suitable
candidates. G3 Meeting the set of minimum basic requirements. Although
screeners were able to differentiate candidates relative to each other, their
focus was on finding candidates that met these requirements. G4 Representative
suitable candidates. Fairness goals already existed in the form of
representation quotas, often around gender, that were enforced by the
screeners. G5 A consistent notion of time. Screeners aimed at spending one
minute per candidate.

Although G1-5 are specific to G, they highlight salient aspects of a real-
world candidate screening problem that are, we argue, likely to hold in
similar settings involving humans searching a pool of candidates. We note
that, while G4-5 are standard to the fair set selection problem, especially
under an algorithmic screener (Section 2.5), G1-3 introduce new considerations
to such problem formulation, especially under a human screener (Section 1.2).

###  1.2. Related Work

In formalizing the ISO problem, our work revives the role of position bias
within a screening process involving human screeners. Works on search engine
click models were the first to formalize (Craswell et al., 2008) and test (Pan
et al., 2007; Grotov et al., 2015; Richardson et al., 2007) how users search
over an ISO. These works modeled the different clicking procedures observed in
users and provided experimental evidence for the existence of position bias.
Hence, it is not surprising that today product owners are willing to pay
premiums to search engines and similar platforms for the first spots in the
search results of a user (Athey and Ellison, 2011). Different from these
works, we consider a user that chooses (i.e., “clicks”) more than one item.
Further, we formalize such users under a utility-maximizing framework with
fairness constraints, which relates to past set selections works (Stoyanovich
et al., 2018).

Within algorithmic fairness, focus has been mainly on learning an algorithm
that provides a fair ISO (Zehlike et al., 2023a, b; Pitoura et al., 2022).
Works on probability-based (Yang and Stoyanovich, 2017; Zehlike et al., 2017)
and exposure-based (Baeza-Yates, 2018; Joachims et al., 2017) fairness tackle
position bias by (re-)arranging the ISO so that is fair according to,
respectively, some model for user attention and spot allocations. These works,
however, avoid formalizing the user for which the fair ISO is meant, e.g.,
assuming a complete search of the (re-)arranged ISO by the user. Based on the
click models and our own experience with G, we argue that it is important to
model the human user to fully grasp the ISO problem. The fair set selection
literature, including the fair ranking, is vast; we position our work within
this literature in Section 2.5 after we have formulated the ISO problem.

We highlight two recent works that provide evidence for position bias and the
implicit role of the ISO. Echterhoff et al. (2022) collaborate with a college
to study anchoring bias (Kahneman, 2011) in admissions; anchoring bias
(Tversky and Kahneman, 1974) occurs when a candidate’s evaluation is
conditioned by the quality of the previously evaluated candidates. This paper
finds that the same candidate is better off if it is preceded by worst
candidates than better candidates, which violates individual fairness (Dwork
et al., 2012), and proposes an algorithm for capturing and balancing anchoring
bias. Pei et al. (2023) work with a college to study how the platforms used by
professors for evaluating homework affects the students’ grades. This paper
runs a set of experiments varying the order in which the homeworks are
presented by the platform. The experiments shows that the default sorting
order, which is in alphabetical order, unfairly rewards students with the same
work quality depending on their last names, which violates individual fairness
(Dwork et al., 2012).

Both papers provide evidence of the ISO problem (w.r.t. fairness) and, similar
to our work, highlight the role of the human using the ISO. However, we differ
from these works in that we provide a simulations framework supported by our
problem formulation flexible enough to capture multiple screening scenarios.
Gathering experimental data or having access to the real process are costly
and limited to stakeholders studying questions of fairness, which positions
simulations as useful tools for answering these questions (Ionescu et al.,
2021; Jr. et al., 2020; Bountouridis et al., 2019; Bokányi and Hannák, 2020;
Schelling, 1971). We come back to this point in Section 5.

##  2\. Searching the Pool of Candidates

The goal is to formulate the fair set selection problem, where a decision-
maker selects a set of items from a population, by considering the initial
screening order (ISO) in which the items are presented to it. Here, the
candidates for a job represent the items and the screener evaluating their
profiles represents the decision-maker.

###  2.1. Setting

Let us consider a candidate pool $\mathcal{C}$ of $n$ candidates. Each
candidate $c$ is described by the vector of $p$ attributes
$\mathbf{X}_{c}\in\mathbb{R}^{p}$ and the protected attribute $W_{c}$. To
simplify our analysis, we assume that $W$ encodes the membership to the
protected group and is binary with $W_{c}=1$ if $c$ belongs to the protected
group and $W_{c}=0$ otherwise. We can relax this assumption if needed. The
candidates are evaluated by a screener $h\in\mathcal{H}$, where $\mathcal{H}$
denotes the set of available screeners. The following variables refer to a
specific $h$. The goal of $h$ is to obtain a set of $k$ selected candidates
$S^{k}\in[\mathcal{C}]^{k}$, with $[\mathcal{C}]^{k}$ denoting the set of
$k$-subsets of $\mathcal{C}$ (i.e., subsets with cardinality $k$), based on
each candidate’s application profile as summarized by the tuple
$(\mathbf{X}_{c},W_{c})$. We model candidate evaluation by assuming that $h$
uses an individual scoring function $s\colon\mathbb{R}^{p}\to[0,1]$, such that
$s(\mathbf{X}_{c})$ returns the score of $c$ according to $h$. Here, $h$ does
not (or, equivalently, cannot) use $W_{c}$ when scoring any $c$. The higher
the score, the better the candidate fits the job.

The screener $h$ explores the candidate pool $\mathcal{C}$ in a specific
order. We denote the set of total orderings of candidates in $\mathcal{C}$ by
$\Theta$. It represents all the possible ways in which the $n$ candidates in
$\mathcal{C}$ can be arranged. An order $\sigma\in\Theta$ maps an integer
$i\in\\{1,\dots n\\}$ to a candidate $c\in\mathcal{C}$, indicating that $c$
occupies the $i$-th position according to $\sigma$, with notation
$\sigma(i)=c$ and vice-versa $\sigma^{-1}(c)=i$.

Importantly, the screener explores the candidate pool $\mathcal{C}$ under the
initial screening order $\theta\in\Theta$, which represents the order chosen
by or, alternatively, provided to $h$ before starting the exploration of
$\mathcal{C}$ (recall, G1 in Section 1.1). The screener is not required to
explore the entirety of $\mathcal{C}$, meaning $h$ can either fully or
partially explore $\mathcal{C}$ given $\theta$ (recall, G2 in Section 1.1). We
assume that the screener respects $\theta$, meaning:

(1) a candidate $c_{1}\in\mathcal{C}$ is evaluated before
$c_{2}\in\mathcal{C}$ only if $\theta^{-1}(c_{1})<\theta^{-1}(c_{2})$.

###  2.2. Group Fairness

To address the fairness goal of the screener $h$ (recall, G4 in Section 1.1),
we assume a quota $q$ for the protected group $W=1$. We introduce the fraction
$f\big{(}S^{k}\big{)}\in[0,1]$ of protected candidates in the selected set
$S^{k}$ as:

(2) $f\big{(}S^{k}\big{)}=\frac{\left|\\{c\in S^{k}\text{ s.t.
}W_{c}=1\\}\right|}{k}$

and we denote by $q\in[0,1]$ the desired fraction of selected protected
candidates in $S^{k}$. The fair $h$ will be constrained to meet the
representational quota $q$ when deriving $S^{k}$ by satisfying the condition
$f\big{(}S^{k}\big{)}\geq q$. The unconstrained version, in which no
representativeness requirement is assumed, is achieved by considering $q=0$.
We view $q$ as a policy enforced by $h$ to achieve a diverse $S^{k}$. It is a
statement on the composition of $S^{k}$, not a statement on the ordering of
protected candidates within $S^{k}$. For instance, for $k=10$ and $q=0.5$, the
fair screener would need to derive $S^{k}$ with $50\%$ of protected candidates
though in no particular order within $S^{k}$.

###  2.3. Two Problem Formulations

How might we account for the two ways $h$ can search $\mathcal{C}$ as
stipulated in our setting? We proceed to formulate two fair set selection
problems for $h$, in which we distinguish between best-$k$ and good-$k$. Under
the best-$k$ formulation, the set $S^{k}$ represents the fair best $k$
candidates in $\mathcal{C}$ according to $h$. We denote it accordingly as
$S^{k}_{\text{{best}}}$. Under the good-$k$ formulation, the set $S^{k}$
represents the fair first good-enough $k$ candidates in $\mathcal{C}$
according to $h$. We denote it accordingly as $S^{k}_{\text{{good}}}$. For
both formulations we define the objective of $h$ in terms of achieving an
optimal and fair selection of candidates. How we define optimality, as we
show, derives the best-$k$ and good-$k$ problems; we already defined fairness
in the previous subsection. The key difference between these two problem
formulations is that the best-$k$ requires a full search of $\mathcal{C}$
while the good-$k$ allows for a partial search of $\mathcal{C}$ under an ISO
$\theta$.

####  2.3.1. Best-$k$.

We first focus on the screener $h$ that finds the set of best $k$ candidates
in $\mathcal{C}$ given the fairness constraint $q$ and while respecting the
ISO $\theta$ (1). Here, $h$ needs to evaluate the complete $\mathcal{C}$ since
$h$ must score and rank all candidates according to the individual scoring
function $s$ before choosing the ones with the highest scores and that satisfy
$q$.

We view the goal in terms of maximizing a utility for $h$. We define utility
as the benefit derived by $h$ from selecting $k$ suitable candidates under
$\theta$. Formally, utility is a function
$U^{k}\colon[\mathcal{C}]^{k}\,\times\,\Theta\to\mathbb{R}$. The simplest
expression for defining $U^{k}$ is to add the scores of the selected
candidates:

(3) $U^{k}_{\text{{add}}}\big{(}S^{k},\theta\big{)}=\sum_{c\in
S^{k}}s\big{(}\mathbf{X}_{c}\big{)}$

rationalizing that $h$ will maximize its utility by selecting the $k$ most
suitable candidates for the job. Notice that $\theta$ in (3) does not affect
the evaluation of $S^{k}$ because of the commutative property of addition.
Under (3), we define the fair best-$k$ set selection problem or, simply, the
best-k problem as:

(4) $\displaystyle\operatorname*{arg\,max}_{S^{k}\in[\mathcal{C}]^{k}}$
$\displaystyle\quad U^{k}_{\text{{add}}}\big{(}S^{k},\theta\big{)}$ s. t.
$\displaystyle\quad f(S^{k})\geq q$

for which we denote its solution as $S^{k}_{\text{{best}}}$. In the presence
of tied scores, the solution may not be unique. In such a case, we consider
any solution. We emphasize that (3) is not the only possible model for the
utility of $h$ and alternative models, such as exposure discounting (Singh and
Joachims, 2018), can be considered for (4). We leave this for future work.

####  2.3.2. Good-$k$

We now focus on the screener $h$ that finds $k$ candidates in $\mathcal{C}$
that meet a set of minimum basic requirements $\psi$ (recall, G3 in Section
1.1) given the fairness constraint $q$ and while respecting the ISO $\theta$
(1). We represent $\psi$ as a minimum score, such that $h$ deems candidate
$c\in\mathcal{C}$ as eligible for being selected if
$s(\mathbf{X}_{c})\geq\psi$. Here, unlike the best-$k$ formulation, $h$ is not
required to evaluate the whole $\mathcal{C}$ as it is enough to find the first
$k$ candidates that are good enough according to $\psi$ and that satisfy $q$.

We still view the goal in terms of maximizing a utility for $h$. We need,
however, to define an alternative utility function to (3) that specifically
ensures that $h$ stops searching $\mathcal{C}$ after finding the $k$-th good-
enough candidate according to $\psi$. We define the following expression for
$U^{k}_{\psi}$:

(5)
$U^{k}_{\psi}\big{(}S^{k},\theta\big{)}=\left\\{\begin{array}[]{ll}k-\sum_{c\in
S^{k}}p(c,S^{k},\theta)&\text{if,}\ \forall c\in S^{k},\
s(\mathbf{X}_{c})\geq\psi\\\ 0&\text{otherwise.}\end{array}\right.$

with the penalty function $p(c,S^{k},\theta)=\mathbb{1}(\exists\
c^{\prime}\in\mathcal{C}\setminus S^{k}\,\mbox{s.t.}\
\,\theta^{-1}(c^{\prime})<\theta^{-1}(c)\wedge
s(\mathbf{X}_{c^{\prime}})\geq\psi\wedge W_{c^{\prime}}=W_{c})$. Under (5),
$h$ wants to find as quick as possible the $k\text{-{th}}$ suitable candidate
without wanting to check whether the $k\text{-{th}}+1$ candidate is also
suitable. This is because, for a candidate $c$, $p(c,S^{k},\theta)$, looks for
another candidate of the same group as $c$ and meeting $\psi$, who occurs
before $c$ under $\theta$ but who has not been selected into $S^{k}$. The
penalty function models the “wasted effort” in choosing a candidate occurring
after another one meeting all the same requirements. At worst, there are $k$
penalties. Under (5), we define the fair good-$k$ set selection problem or,
simply, the good-k problem as:

(6) $\displaystyle\operatorname*{arg\,max}_{S^{k}\in[\mathcal{C}]^{k}}$
$\displaystyle\quad U^{k}_{\psi}\big{(}S^{k},\theta\big{)}$ s. t.
$\displaystyle\quad f(S^{k})\geq q$

for which we denote its solution as $S_{\text{{good}}}^{k}(\psi)$ or, if there
is no ambiguity on $\psi$, simply as $S_{\text{{good}}}^{k}$. If the fairness
constraint is strengthened to a fixed quota, i.e., $f(S^{k})=q$, it can be
shown that the solution is unique. In the general case, i.e., $f(S^{k})\geq
q$, there can be two solutions but with different fractions of the protected
group. For this last point, see Example B.1 in Appendix B.1. Further, we
emphasize that (5) is not the only utility model for (6). Other models are
possible as long as they properly describe the partial search allowed within
the good-$k$ problem. We leave this for future work. For more details, also
see Appendix B.1 where we motivate (5) by presenting a simpler utility model
(read, without the penalty) and showing its failure to stop $h$ from
evaluating more candidates.

###### Remark 1.

The ISO $\theta$ can influence the screening process under the good-$k$
problem (6) due to the potential partial search of $\mathcal{C}$ by $h$. This
holds without assuming anything about $h$. To observe this point, let $k=1$
and assume two candidates such that $s(\mathbf{X}_{c_{1}})\geq\psi$ and
$s(\mathbf{X}_{c_{2}})\geq\psi$. A $\theta$ such that $\theta^{-1}(c_{1})=1$
and $\theta^{-1}(c_{2})=2$ would imply that $c_{1}$ is considered eligible
before even evaluating $c_{2}$. Conversely, a reverse $\theta$ such that
$\theta^{-1}(c_{1})=2$ and $\theta^{-1}(c_{2})=1$ would imply eligible $c_{2}$
before $c_{1}$. Since $h$ will stop after finding $k=1$ good-enough
candidates, $\theta$ affects which candidates are selected.

###  2.4. Two Search Procedures

We present two search procedures for the best-$k$ and good-$k$ problems. These
two algorithmic implementations, with the ExaminationSearch solving for
$S_{\text{{best}}}^{k}$ and the CascadeSearch solving for
$S_{\text{{good}}}^{k}$, allow us to operationalize the two problem
formulations given the ISO $\theta$ of candidates chosen by or provided to
$h$.

The ExaminationSearch procedure, Algorithm 1, solves the best-$k$ setting,
returning $S^{k}_{\text{{best}}}$ for given $n$ (candidates), ISO $\theta$,
and parameters $k$ (subset size) and $q$ (fairness group constraint). First,
line 2 calculates the minimum number $q^{*}$ of candidates from the protected
group to be selected, and the maximum number of candidates $r^{*}$ not in that
quota. Then, candidates are considered by descending scores, using the
argsortdesc procedure (lines 2, 3). The loop in lines 5-13 iterates until $k$
candidates are found. The loop adds candidates to the sets $Q$ and $R$: $Q$
are candidates in the quota of the protected group; $R$ are candidates not in
that quota (can be non-protected or protected). An non-protected candidate can
be only added to the $R$ set, thus line 7 checks if there is still room in $R$
to do this. A protected candidate is added to the quota set $Q$ if there is
room (lines 10-11) or to the other set $R$ otherwise (lines 12-13). Finally,
the procedure returns the candidates in the quota set $Q$ or in the other set
$R$. The result of the ExaminationSearch procedure maximizes (4), as
candidates are added in decreasing score, while keeping the fairness
constraint through the quota management.

The CascadeSearch procedure, Algorithm 2, solves the good-$k$ setting,
returning $S^{k}_{\text{{good}}}$ for given $n$, ISO $\theta$, and parameters
$k$, $q$ and $\psi$ (minimum basic requirement). The difference with the
ExaminationSearch procedure consists in strictly following the ISO $\theta$
(line 4), and in checking $\psi$ (line 8) before adding a candidate to the
quota set $Q$ or to the other set $R$. The result of the CascadeSearch
procedure maximizes (6), as no penalty is accumulated in the loop. In fact, a
non-protected candidate ($W_{c}=0$) is not added only because there is no room
in $R$ – and $R$ never gets smaller to allow for more room later on. A
protected candidate ($W_{c}=1$) is not added only if not meeting the minimum
basic requirement (line 8), hence it cannot be counted for the penalty.
Formally, $U^{k}_{\psi}\big{(}S^{k}_{\text{{good}}},\theta\big{)}=k$ for the
solution $S^{k}_{\text{{good}}}$ returned by CascadeSearch.

Algorithm 1 ExaminationSearch

1:$n$, $\theta$, $k$, $q$

2:$S^{k}_{\text{{best}}}$

3:$q^{*}\,\leftarrow$ round($q\cdot k$); $r^{*}\,\leftarrow k-q^{*}$

4:$\mathit{scores}\,\leftarrow[s(\mathbf{X}_{c})\ \mathit{for}\
c=\theta(1),\ldots,\theta(n)]$

5:$\tau\leftarrow\texttt{argsortdesc(}\mathit{scores}$)

6:$i\,\leftarrow 1$; $k^{*}\,\leftarrow 0$; $Q\,\leftarrow\\{\\}$;
$R\,\leftarrow\\{\\}$

7:while $k^{*}<k$ do

8: $c\;\leftarrow\theta(\tau(i))$; $i\,\leftarrow i+1$

9: if $W_{c}==0$ and len($R$)$==r^{*}$ then

10: continue

11: $k^{*}\,\leftarrow k^{*}+1$

12: if $W_{c}==1$ and $\texttt{len(}Q\texttt{)}<q^{*}$ then

13: $Q\,\leftarrow Q\cup\\{c\\}$

14: else

15: $R\,\leftarrow R\cup\\{c\\}$ return $Q\cup R$

Algorithm 2 CascadeSearch

1:$n$, $\theta$, $k$, $q$, $\psi$

2:$S_{\text{{good}}}^{k}$

3:$q^{*}\,\leftarrow$ round($q\cdot k$); $r^{*}\,\leftarrow k-q^{*}$

4:$i\,\leftarrow 1$; $k^{*}\,\leftarrow 0$; $Q\,\leftarrow\\{\\}$;
$R\,\leftarrow\\{\\}$

5:while $k^{*}<k$ do

6: $c\;\leftarrow\theta(i)$; $i\,\leftarrow i+1$

7: if $W_{c}==0$ and len($R$)$==r^{*}$ then

8: continue

9: $Y_{c}\;\leftarrow s(\mathbf{X}_{c})$

10: if $Y_{c}\geq\psi$ then

11: $k^{*}\,\leftarrow k^{*}+1$

12: if $W_{c}==1$ and $\texttt{len(}Q\texttt{)}<q^{*}$ then

13: $Q\,\leftarrow Q\cup\\{c\\}$

14: else

15: $R\,\leftarrow R\cup\\{c\\}$ return $Q\cup R$

###  2.5. Additional Related Work

We presented two formulations of fair set selection problems. Here, we can now
compare our formulations with the fair ranking and set selection literature,
where, in the latter, $h$ cares about the order within the chosen $S^{k}$.

The reference ranking problem is the top-$k$ selection (Fagin et al., 2003)
representing the standard setting in the fairness literature (Zehlike et al.,
2023a, b). The reference set selection problem is the Secretary Problem (SP),
inspired by the hiring process (Ferguson, 1989): the goal is to select the
maximum valued element in a randomly ordered sequence of $n$, with the
condition of committing irrevocably to the acceptance or rejection decision
after each evaluation. The past literature has already analyzed the fairness
implication of the SP, even in the $k$-choice extension (e.g., (Stoyanovich et
al., 2018)). Our best-$k$ and good-$k$ formulations, differently from the SP,
focus on the screening process, not the interview phase. More generally, we
assume an offline set selection, that is, $h$ evaluates the applicants
individually according to the ISO and each decision is not irrevocable.
Instead, the SP is online, i.e., a decision must be made as each applicant is
presented. Notably, past works focusing on the online setting assume the
additive utility we employ in the best-$k$ formulation (Stoyanovich et al.,
2018; Mehrotra and Celis, 2021). This fact shows how the peculiar good-$k$ set
selection formulation, inspired by G’s hiring process, differs from the
previously analyzed settings.

We emphasize that our research question focuses on the impact of the ISO
$\theta$ on fair screening practices. With few exceptions (recall, Section
1.2), the impact of $\theta$ is neglected by the past literature to the best
of our knowledge. For this reason, we move from the optimality analysis of the
two algorithms (e.g., (Fagin et al., 2003)), toward modeling the behaviour of
a screener solving the two problem formulations. For this reason, we do not
aim to provide novel optimal algorithms for the two problem formulations. The
two algorithms we discuss are inspired by the click-model literature. They are
inherently sequential and can be applied online because we aim to model a
human-like screener, described in the next section, that operates
sequentially. Yet, in both settings, the applicants are disclosed according to
$\theta$ and not to the score, differently from past works (Stoyanovich et
al., 2018). We add to this line of work by introducing $\theta$ as a parameter
of interest.

##  3\. The Human-Like Screener

We want to model a screener $h$ prone to error when evaluating the candidate
pool $\mathcal{C}$ given the ISO $\theta$, allowing to study the consequences
of an ISO on a human screener solving for the best-$k$ (4) or good-$k$ (6)
problems.

We distinguish two kinds of screeners $h$: the algorithmic screener, denoted
by $h_{a}\in\mathcal{H}_{a}$, if it can consistently evaluate $\mathcal{C}$;
the human-like screener, denoted by $h_{h}\in\mathcal{H}_{h}$, if it is
conditioned by a fatigue component that hinders the consistency of its
evaluation as it explores $\mathcal{C}$. We introduce a time component to
study these screeners. Let $t$ denote the discrete unit of time that
represents how long $h$ takes to evaluate a candidate $c\in\mathcal{C}$. We
assume that $t$ is constant (recall G5 Section 1.1), implying time itself
cannot be optimized by $h$. We track time along $\theta$, meaning $h$
evaluates the first candidate that appears in $\theta$ at time $t=1$, and so
on. Thus, time $t$ ranges from $0$ to $n$ at maximum.

###  3.1. Fatigue and Fatigued Scores

Formally, we introduce a fatigue component $\phi(t)$ specific to the human-
like screener $h_{h}$ as a function of time $t$ and model the accumulated
fatigue $\Phi\colon\\{0,\ldots,n\\}\to\mathbb{R}$, with $\Phi(0)=0$. The
discrete derivative of $\Phi$, that is, $\phi(i)=\Phi(i)-\Phi(i-1)$, defined
for $t\geq 1$, is the effort of $h_{h}$ to examine the $t$-th candidate.
Clearly, how we define $\Phi$ conditions the effect of fatigue on our analysis
of $h_{h}$. We make the simplest modeling choice to define $\phi$ by assuming
that fatigue accumulates linearly over time, or $\phi(t)=\lambda$ so that
$\Phi(t)=\lambda\cdot t$. This is one possible formulation based on our
interpretation that $h_{h}$ becomes tired at a constant pace. We leave the
study of other $\Phi$ formulations for future work.

How does fatigue materialize for $h_{h}$ when searching $\mathcal{C}$ under
$\theta$? We model the effect of fatigue when $h_{h}$ evaluates $c$ through
the fatigued score $s_{h_{h}}(\mathbf{X}_{c})+\epsilon$, where $\epsilon$ is a
random variable depending on the fatigue $\Phi$ which quantifies the deviation
from the truthful score $s_{h}(\mathbf{X}_{c})$. We model $\epsilon$ according
to _two modeling choices_. The first choice defines $\epsilon_{1}$ as a
centered Gaussian variable, and the fatigue affects only its variance.
Formally, at a given $t$, $\epsilon_{1}\sim\mathcal{N}(0,\,v(\Phi(t-1)))$,
where $v\colon\mathbb{R}\to\mathbb{R}$ defines the variance of $\epsilon_{1}$
as an increasing function of the accumulated fatigue. The second choice
assumes that a negative bias can affect each applicant’s evaluation. The more
fatigue, the more the screener $h_{h}$ tends to underscore the candidates.
Hence, we model $\epsilon_{2}$ as an uncentered Gaussian, whose mean is a
decreasing function of the fatigue. Formally,
$\epsilon_{2}\sim\mathcal{N}(\mu(\Phi(t-1),\,v(\Phi(t-1))$, where
$\mu\colon\mathbb{R}\to\mathbb{R}$ is a decreasing function rather than a
constant.

###### Remark 2.

What $\Phi$ means in practice is that $h_{h}$ evaluates the identical
candidates $c_{1}$ and $c_{2}$ differently under the ISO $\theta$ at times
$t_{1}$ and $t_{2}$, as long as $\Phi(t_{1})\neq\Phi(t_{2})$ and regardless
whether $h_{h}$ is solving for the best-$k$ or good-$k$ problem.

We assume that $h_{h}$ is unaware of its fatigue as it goes over
$\mathcal{C}$, representing an unconscious decision-making bias due, e.g., to
mental heuristics (Tversky and Kahneman, 1974). Such a setting allows
capturing what occurs in real life, where $h_{a}$ is consistent in its
evaluation of $\mathcal{C}$ whereas $h_{h}$ loses its consistency over time
(Echterhoff et al., 2022; Pei et al., 2023). Algorithms 1 and 2 represent
$h_{a}$ as there is no notion of fatigue nor biased scores. To represent the
human-like screener, we need to track the accumulated fatigue $\Phi$ over the
time and draw $\epsilon$ accordingly to compute the fatigued scores of
$h_{h}$. The main change is, thus, to line 2 in Algorithm 1 and line 7 in
Algorithm 2, where the score computed for candidate $c$ is biased by
$\epsilon$ sampled at that time. We present the human-like versions of these
search procedures in Appendix B.2 as Algorithms 3 and 4.

###  3.2. Position Bias Implications

Importantly, no two candidates can occupy the same position in the ISO. With
this in mind, we now analyse the fairness and optimality implications of the
position bias implicit to the ISO. For concreteness, let us consider the next
two assumptions. A1: We assume that $\theta$ is independent of the protected
attribute $W$, meaning the way in which candidates are sorted in $\theta$
contains no information about membership to either category in $W$. A2: We
assume that the individual scoring functions $s$ for either screener is able
to evaluate any candidate $c$ fairly and truthfully, meaning
$s(\mathbf{X}_{c})$ captures no information about $W_{c}$ and all information
about the suitability of $c$. Under A1 and A2 we can control for other biases,
such as measurement error in $s$, and focus on the bias coming from the ISO.

Let us start with the fairness implications in the best-$k$ and good-$k$
problems. Here, it is important to distinguish between the group-level
fairness constraining $h$ given $\theta$ and the potential individual fairness
violations under $\theta$, in terms of $h$ failing to evaluate similar
candidates similarly (Dwork et al., 2012). Regarding group-level fairness,
both $h_{a}$ and $h_{h}$ are fair in solving for $S_{\text{{best}}}^{k}$ and
$S_{\text{{good}}}^{k}$. Indeed, both fair $h_{a}$ and $h_{h}$ must meet the
condition $f\big{(}S^{k}\big{)}\geq q$. This point is clear for $h_{a}$ (i.e.,
Algorithms 1 and 2) as there is no fatigue involved. The same holds for
$h_{h}$ (i.e., Algorithms 3 and 4) because the error on the score does not
affect the evaluation of the representational quota $q$. Formally, the
expected error $\mathbb{E}[\epsilon\mid
W_{c}=1,\theta]=\mathbb{E}[\epsilon\mid W_{c}=0,\theta]$, regardless of
$\epsilon$ being $\epsilon_{1}$ or $\epsilon_{2}$. The fatigue and, thus, the
fatigued scores are, on average, shared across protected and non-protected
candidates.

The distinction between $h_{a}$ and $h_{h}$ becomes important under
individual-level fairness because $h_{a}$ ensures it, while $h_{h}$ violates
individual fairness for a specific candidate $c$. Indeed, the candidate’s
position in $\theta$ influences the amount of error made by $h_{h}$ when
evaluating that candidate. Therefore, similar candidates will not be evaluated
similarly due to the unequal accumulation of fatigue experienced by $h_{h}$,
because of their different positions in $\theta$. For $\epsilon_{2}$, e.g.,
even if $\mathbf{X}_{c_{j}}=\mathbf{X}_{c_{i}}$, but $j>i$, the score of $j$
is less, on average, than the one of $i$, so that $i$ has an unfair premium
from $\theta$.

Let us now consider the optimality implications of the best-$k$ and good-$k$
problems. Here, we need to keep in mind that each problem, due to its utility
model, has different optimal solutions, with best-$k$ requiring the top-$k$
candidates in $\mathcal{C}$ and good-$k$ the first good-enough $k$ candidates
in $\mathcal{C}$. It follows that $h_{a}$ reaches the optimal solution as the
absence of fatigue enable $h_{a}$ to consistently judge suitable candidates,
thus to obtain the optimal selected set. The opposite holds for $h_{h}$ due to
the inconsistent scoring of candidates ascribed by the accumulated fatigue.

We can summarize the impact of the position bias in the ISO, if the $q$
defines the fairness constraint. We find that $h_{a}$ reaches the optimal fair
solution of both fair best-$k$ and good-$k$ problems. Moreover, $h_{a}$
guarantees individual fairness in both settings. Instead, we find that $h_{h}$
reaches the fair but, potentially, sub-optimal solutions for both fair
best-$k$ and good-$k$ problems. Moreover, $h_{h}$, potentially, does not
guarantee individual fairness in both settings. We purposely include
“potentially” for $h_{h}$ because this analysis depends on how we model
fatigue and whether it impacts the selected set.

###  3.3. The Initial Screening Order Problem

The ISO problem amounts to a combination of factors that are likely to occur
in real-world screening scenarios. As our analysis shows, the ISO has no
effect under an algorithmic screener $h_{a}$. This is a trivial result,
especially given the ideal assumptions A1 and A2. Under these conditions,
automating the screening process is clearly the preferred option. In practice,
however, even under A1 and A2, it is likely for the screening process to
require the decision of a human-like algorithmic screener $h_{h}$, as it was
the case for G (Section 1.1) and the other similar case studies (Section 1.2).

Significantly under the $h_{h}$, the position of a candidate within the chosen
or provided ISO $\theta$ matters. It has impact for determining whether a
candidate, depending on the search strategy, is evaluated at all (Remark 1)
and, if so, is evaluated fairly (Remark 2). Note that these results only
worsen when relaxing the assumptions A1 and A2. The best position within
$\theta$ to be as a candidate is the first spot, and less from there. Being
first implies meeting a non-fatigued $h_{h}$. To an extent, this result of
position bias is also trivial. What is difficult to answer is _how exactly the
fatigue of the $h_{h}$ evolves over $\theta$_. The answer to this question
passes through our two problem formulations and their respective algorithmic
implementations, as illustrated with the simulations in the next section.

##  4\. Experiments

We now explore the implications of the best-$k$ and good-$k$ problems through
Monte Carlo experiments for both the algorithmic $h_{a}$ and human-like
$h_{h}$ screeners. Moreover, this section shows how our framework can explore
different screening scenarios involving the initial screening order (ISO). The
algorithms 1–4 and simulation procedures are developed in R (R Core Team,
2024).111See the GitHub repository for the code; https://github.com/cc-
jalvarez/initial-screening-order-problem/tree/main.

###  4.1. Experimental Setup

####  4.1.1. Generating the sample.

We assume a sample consisting of $n$ triplets
$\\{(\theta({c_{i}}),s(\mathbf{X}_{c_{i}}),W_{c_{i}})\\}_{i=1}^{n}$ drawn from
a probability distribution with domain
$\mathcal{G}_{n}\times\mathbb{R}^{n}\times\\{0,1\\}^{n}$ where
$\mathcal{G}_{n}$ is the set of all permutations of $\\{1,\ldots,n\\}$. Each
sample represents a specific candidate pool $\mathcal{C}$ sorted according to
an ISO $\theta$.

Regarding $s(\mathbf{X}_{c_{i}})$, we consider three possible distributions of
candidate scores. All are based on the truncated normal distribution family
$tN(\mu,\sigma)$ (Botev, 2017) with values bounded in the interval $[0,1]$.
Here, we wish to model scenarios in which very good candidates, as in those
with top scores, occur with different probabilities. All three scenarios are
shown in Figure 1 (left). A symmetric distribution of scores (in red) defined
by $tN(0.5,0.02)$ with mean/median of $0.5$ implies that top candidates occur
with a very low probability. An asymmetric distribution of scores (in blue)
defined by $tN(0.8,0.05)$ implies that top candidates occur with a higher
probability and median value ($\approx 0.75$) compared to the previous
scenario. An increasing distribution of scores (in green) defined by
$tN(1,0.05)$ implies that top candidates occur with an even higher probability
and median value ($\approx 0.85$).

The score scenarios have implications, in particular, for the good-k problem
where we must set the minimum score $\psi$ and the screener is not required to
explore all of $\mathcal{C}$ under $\theta$ (recall Remark 1). Setting a large
$\psi$ makes the screening process highly selective in the first setting, less
selective in the second setting, and not selective at all in the third
setting. These settings represent a range of candidate pools with different
candidate quality on average.

Regarding $\theta$, we consider two possible variants for the ISO. First: the
ISO is randomly and independently generated from the candidate scores. This
setting models the case in which $\theta$ brings no information regarding
candidate quality, and the screener prefers the alphabetical order or performs
a random shuffle of candidates. We denote such a scenario as
$\theta\mathrel{{\scalebox{1.07}{$\perp\mkern-10.0mu\perp$}}}s$. Second: The
ISO is randomly generated generated with a given Spearman’s rank correlation
$\rho$ with the candidate scores. Formally, $\rho$ is the Spearman’s rank
correlation of the pairs
$\\{(\theta(c_{i}),s(\mathbf{X}_{c_{i}}))\\}_{i=1}^{k}$. It assesses how well
$\theta$ monotonically relates to the scores of candidates.222To generate
correlated initial ordering and scores, we rely on copulas – see e.g.,
(Embrechts et al., 2003, Section 3.4). We denote such scenario as
$\theta\not\\!\perp\\!\\!\\!\perp s$. In particular, for $\rho=-1$, it means
that $\theta$ ranks candidates by descending scores similar to giving a ranked
list of candidates to the screener. With these two variants, we can consider a
$\theta$ chosen by or provided to the screener, respectively.

Regarding $W$, we initially consider the sample of candidates drawn from
$Ber(\mathit{pr})$ such that $\mathit{pr}=0.2$ is the fraction of protected
group candidates in the population. The sample is independently drawn from
both the scores and the ISO, according to the assumptions A1 and A2 from
Section 3.2. Here, we are determining the diversity of $\mathcal{C}$. Later
on, we increase $\mathit{pr}$ to study a more diverse $\mathcal{C}$ and its
effect on the screener reaching the representational quota $q$.

Beyond the triplet
$\big{(}\theta({c_{i}}),s(\mathbf{X}_{c_{i}}),W_{c_{i}}\big{)}$, for the
fatigued scores of $h_{h}$, we fix $\lambda=1$, hence $\Phi(t)=t$, and define:
$\epsilon_{1}\sim\mathcal{N}(0,\,(0.005\cdot(t-1))^{2})$, hence with constant
expectation and with standard deviation of $0.005\cdot(t-1)$; and
$\epsilon_{2}\sim\mathcal{N}(-0.005\cdot(t-1),\,(0.001\cdot(t-1))^{2})$, hence
with a decreasing expectation and with a smaller standard deviation than
$\epsilon_{1}$. Recall Section 3.1 for details.

####  4.1.2. Evaluation metrics.

We consider the solution $S^{k}_{\text{{best}}}$ of the best-$k$ problem (4)
for $h_{a}$ (Algorithm 1) as the baseline solution. We compare it with the
analysis of the solutions for the good-$k$ problem (6) under $h_{a}$
(Algorithm 2) and of the solutions for the best-$k$ and good-$k$ problems
under $h_{h}$ (respectively, Algorithms 3 and 4). We introduce two comparison
metrics that capture how close is the compared solution to the baseline
solution.

The ratio to baseline (RtB) is defined as the ratio of $U^{k}_{\text{{add}}}$
between the compared solution and the baseline solution. For the solution
$S_{\text{{good}}}^{k}$ under $h_{a}$, e.g., it is
$U^{k}_{\text{{add}}}(S_{\text{{good}}}^{k})/U^{k}_{\text{{add}}}(S^{k}_{\text{{best}}})$.
The closer the ratio is to $1$, the better the compared solution approximates
the best-$k$ solution under $h_{a}$ in terms of $U^{k}_{\text{{add}}}$
utility. Here, when calculating the utility of $h_{h}$, we use the truthful
scores, not the fatigued scores, to be able to compare w.r.t. the baseline.
The Jaccard similarity (JdS) is defined as the proportion of candidates in
both the compared and baseline solutions over those in at least one of the
two. For the solution $S_{\text{{good}}}^{k}$ under $h_{a}$, e.g., it is
$|S_{\text{{good}}}^{k}\cap
S^{k}_{\text{{best}}}|\,/\,|S_{\text{{good}}}^{k}\cup S^{k}_{\text{{best}}}|$.
Such a metric quantifies the share of candidates between the two solutions.
Essentially, the RtB metric captures whether the compared solution achieves
the same utility as the baseline solution as measured by
$U^{k}_{\text{{add}}}$, while the JdS metric captures the overlap in
candidates between the compared solution and the baseline solution.

####  4.1.3. Simulations.

For each setting of the parameters ($n$, $k$, $q$, $\rho$, $\psi$ or others),
we run 10,000 times the experiments by randomly generating $n$ triplets at
each run. The runs for which a solution of the problem does not exist are
discarded. This mainly occurs in the good-$k$ problem when there are not
enough $k$ candidates with scores greater or equal than $\psi$. The plots
report the mean output based on the evaluation metrics over all the runs.

###  4.2. Experiments without Fatigue

Figure 1.  Left: density of experimental candidate score (i.e., candidate
quality) distributions, with scores ranging from 0 to 1. Center: RtB for
different score distributions and the setting $n=120$, $k=6$, $q=0.5$, and
$\theta\mathrel{{\scalebox{1.07}{$\perp\mkern-10.0mu\perp$}}}s$ at the
variation of the minimum score $\psi$. Right: JdS for different score
distributions and the setting $n=120$, $k=6$, $q=0.5$, and
$\theta\mathrel{{\scalebox{1.07}{$\perp\mkern-10.0mu\perp$}}}s$ at the
variation of the minimum score $\psi$.

We start by exploring the settings without fatigue, meaning we consider
$h_{a}$, which allows for clarifying the relation between the best-$k$
(Algorithm 1) and good-$k$ (Algorithm 2) solutions. Given these two problem
formulations, here we are mainly interested if their solutions differ in
practice due to the ISO, especially since the best-$k$ requires a full search
of $\mathcal{C}$ while the good-$k$ allows for a partial search of
$\mathcal{C}$.

We study the impact of the score distributions on the metrics at the variation
of $\psi$. We consider the case of $n=120$, $k=6$, $q=0.5$, and
$\theta\mathrel{{\scalebox{1.07}{$\perp\mkern-10.0mu\perp$}}}s$. Note that
these parameters are shared by both best-$k$ and good-$k$ problems. Instead,
$\psi$ is specific to the good-$k$, which is why we focus on it. We find that,
as $\psi$ increases and screening becomes more selective, the good-$k$
approximates the best-$k$ when there is a low probability of having good
candidates in $\mathcal{C}$. Figure 1, under the RtB (center) and the JdS
(right) metrics, illustrates this point for the three score distribution
scenarios. In particular, the symmetric distribution (in red) allows the
good-$k$ to better approximate the best-$k$ for medium-to-high $\psi$ values.
This result is expected. Having few good candidates forces $h_{a}$ to explore
more of $\mathcal{C}$ under $\theta$, especially as $\psi$ increases and the
$k$ first good-enough candidates essentially become the $k$ top candidates.

The opposite holds for the other two distributions (in green and blue), which
are more resilient to $\psi$ as each represents a higher concentration of good
candidates. It follows that having many good candidates makes it difficult for
$h_{a}$ to select the $k$ top candidates under a partial search. As the RtB
and JdS metrics show in Figure 1 (center, right), $h_{a}$ still achieves
significant utilities under the other two distributions but is unlikely to
derive the same selected set of candidates under a partial search w.r.t. a
full search of $\mathcal{C}$. These results are also expected, but worth
emphasizing. Having many good candidates means that $h_{a}$ can still
partially search $\mathcal{C}$ despite having a highly selective $\psi$: i.e.,
$h_{a}$ finds $k$ good-enough candidates with high-enough scores as the $k$
top candidates but not the same ones.

Figure 1 illustrates how the two problems materialize differently when
implemented due to the ISO $\theta$. Clearly, as noted back in Remark 1, where
the $k$ top candidates appear in $\theta$ can determine if they are selected
or not by $h_{a}$ under a partial search. The position bias in the ISO becomes
more prevalent under many good candidates as, e.g., even the $\mathcal{C}$’s
best candidate may never be selected by $h_{a}$ under a partial search if it
lies at the bottom of $\theta$.

We also study what occurs when $\theta\not\\!\perp\\!\\!\\!\perp s$. We
present these results in Appendix C.1 for $n=120$, $k=6$, and $q=0.5$. Here,
we briefly discuss these results as they further illustrate the role of
$\theta$. Recall that $\rho$ represents the correlation between $\theta$ and
the scores, with a negative $\rho$ implying a descending order. We find that
the good-$k$ solution approximates quite well the best-$k$ solution already
for $\rho=-0.5$; for $\rho=-1$, the two solutions are the same. These results
are expected as $\theta$ essentially represents the best-$k$ solution or an
approximation of it depending on $\rho$’s strength. Under $\rho=-1$, e.g., the
$\theta$ searched by $h_{a}$ is already sorted by the candidate scores and, in
turn, the $k$ first good-enough candidates are also the $k$ best candidates in
the candidate pool. See Figure 4 (center, right) for details.

In Appendix C.1 we also study the impact of the number $n$ candidates in
$\mathcal{C}$ and the number of $k$ candidates to be selected. We find that
under $\theta\mathrel{{\scalebox{1.07}{$\perp\mkern-10.0mu\perp$}}}s$, the
ratio $k/n$ is positively correlated with the ability of best-$k$ to
approximate good-$k$. Intuitively and unsurprisingly, it means that the more
candidates we can select from $\mathcal{C}$, the better the chance to include
top ones under a partial search. Clearly, the influence of $\theta$ diminishes
as $k/n$ increases. See Figure 4 (left) for details. Similarly, we study the
role of changing the representational quota $q$. Here, results are expected
given our setup and underlying assumptions (A1 and A2 from Section 3.2),
finding that under
$\theta\mathrel{{\scalebox{1.07}{$\perp\mkern-10.0mu\perp$}}}s$, $q$ does not
affect the relative strengths of best-$k$ and good-$k$ solutions. See Figure 5
(all) for details.

###  4.3. Experiments with Fatigue

We now focus on $h_{h}$. First, we consider whether fatigue impacts utility
w.r.t. the baseline solution, namely the solution of best-$k$ without fatigue
(Algorithm 1). We compare to such a baseline both the best-$k$ with fatigue
(Algorithm. 3) and good-$k$ with fatigue (Algorithm. 4).

Figure 2.  RtB for different score distributions with $n=120$, $k=6$,
$\theta\mathrel{{\scalebox{1.07}{$\perp\mkern-10.0mu\perp$}}}s$ setting. Left:
good-$k$ solution for fatigue with $\epsilon_{1}$ and setting $q=0.5$ at the
variation of the minimum score $\psi$. Center: good-$k$ solution for fatigue
with $\epsilon_{2}$ and setting $q=0.5$ at the variation of the minimum score
$\psi$. Right: best-$k$ solution for fatigue with $\epsilon_{1}$ at the
variation of the representational quota $q$.

Figure 3.  RtB for different $\rho$’s with setting $n=120$, $k=6$, and $q=0.5$
at the variation of the minimum score $\psi$. Left: good-$k$ solution for
fatigue with $\epsilon_{1}$ and the $tN(0.5,0.02)$ score distribution. Center:
good-$k$ solution for fatigue with $\epsilon_{2}$ and the $tN(0.5,0.02)$ score
distribution. Right: good-$k$ solution for fatigue with $\epsilon_{1}$ and the
$tN(1,0.05)$ score distribution.

Figure 2 (left) shows the RtB metric for the three score distributions for the
good-$k$ solution with fatigue based on $\epsilon_{1}$ for the fatigued
scores. Based on Figure 1 (center), for the asymmetric (in blue) and
increasing (in green) distributions, which are the scenarios with many good
candidates in $\mathcal{C}$, there is not much difference w.r.t. the case
without fatigue. For the symmetric distribution (in red), instead, there is a
considerable decrease for high $\psi$ values. This can be attributed to the
low number of top scores, for which the perturbation due the $\epsilon_{1}$’s
has a large effect. For the other two distributions, instead, there are
sufficiently many top scores, for which perturbation does not dramatically
change the score distribution for top scores. Intuitively, since the RtB
metric captures achieving the utility of the baseline model, under a partial
search $h_{h}$ is able to reach high utility solutions when $\mathcal{C}$ has
many good candidates because $h_{h}$ can avoid evaluating all candidates. This
is unlikely to be the case when few good candidates are in $\mathcal{C}$. As
$\psi$ increases and $h_{h}$ becomes more selective, it also becomes more
tired under $\theta$ as it needs to evaluate more and more candidates to
achieve $k$.

Figure 2 (center) is analogous to (left), but considers the fatigued scores
based on $\epsilon_{2}$. The effect for the symmetric distribution (in red) is
not present in such a case, due to the lower standard deviation of
$\epsilon_{2}$. The bias of $\epsilon_{2}$ does not impact too much, apart
from high values of $\psi$ where it causes the problem not to have a solution
as fatigued scores are smaller than scores of an already low number of top
candidates. In summary, under
$\theta\mathrel{{\scalebox{1.07}{$\perp\mkern-10.0mu\perp$}}}s$, variance
appears more relevant than bias in the case of low probability of top scores.
This result illustrates the importance of how we define fatigue. It can also
inform how the a human screener should behave in practice to diminish the role
of position bias within $\theta$. Given these results, e.g., we would be
interested in exploring under what settings would the human screener
experience $\epsilon_{1}$ over $\epsilon_{2}$.

Figure 2 (right) shows the RtB for the best-$k$ solution with fatigue at the
variation of the quota $q$. There is a considerable and constant loss in
utility under fatigue, which is more consistent for the symmetric distribution
(in red). Interestingly enough, the RtB is lower than in the case of the
good-$k$ with fatigue (see left) for $\psi\geq 0.5$. This result means that,
for the symmetric distribution, the good-$k$ solution with fatigue has better
utility than the best-$k$ solution with fatigue. This noteworthy result can
inform the screening practice.

Finally, we consider the impact of the correlation $\rho$ on the ISO $\theta$
for the good-$k$ solution with fatigue. Figure 3 (left) considers the
symmetric distribution (in red). For the lower half of $\psi$’s, lines are
similar to the analogous case without fatigue shown in Figure 4 (center). For
the higher half of $\psi$’s, instead, there is a decrease in the metric. This
is, again, due to the low probability of top scores, for which the effects of
the variability of $\epsilon_{1}$ is not counter-balanced by correlation of
the scores and $\theta$. Such an effect does not appear for $\epsilon_{2}$ nor
for $\epsilon_{1}$ with the increasing distribution. In fact, the plots in
Figure 3 (center) and (right) closely resemble those in Figure 4 (center) and
(right) respectively. This is an interesting result on its own. It means that,
in the presence of correlation, fatigue does not have an impact on utility of
the good-$k$ solution if there are sufficiently many top scores or a
sufficiently small variability of the fatigue.

Moreover, we believe this last result points at the importance in practice of
providing a $\theta$ to the human screener that has some information about
candidate quality. Intuitively, under a partial search procedure and the
threat of position bias materializing through $\theta$, we would like to
decrease $h_{h}$’s fatigue by minimizing its need to search more of
$\mathcal{C}$. A way to do is to already provide to $h_{h}$ a sorted $\theta$.
Note that this point excludes the difficulty behind deriving such a sorted
$\theta$ in the first place, which is the main goal of the fair set selection
literature. This point, however, hints at an interesting line of future work
focused on human screeners and their interactions with an algorithmic aid.

##  5\. Discussion

In this work, we presented the initial screening order (ISO) as a parameter of
interest; defined two formulations under distinct utility models of the fair
set selection problem, the best-$k$ and the good-$k$, with their corresponding
algorithmic implementations; and introduced a human-like screener to study the
effects of the ISO as though we were studying a human user in a candidate
screening problem. We also provided a simulation framework flexible enough to
account and model for other screening scenarios. Our analysis confirms the
fairness and optimality impact of the ISO, motivated by the risk of position
bias, on the set of $k$ candidates chosen by an algorithmic or human-like
screener. Extensive simulations showed complex relations between best-$k$ and
good-$k$, which heavily depend on the score distribution, on their correlation
with the ISO, and on the fatigue modeling.

We once again emphasize that the assumptions made–both in terms of functional
forms and parameter choice–condition our analysis, in particular, our
experimental results. Some of these choices are probably less contested than
others. For instance, the utility models chosen in Section 2 for defining the
best-$k$ and good-$k$ problems are intuitive and reflective of the screener’s
evaluation criteria in each setting and, overall, recall common modeling
practices within the literature (e.g., (Zehlike et al., 2023a, b; Stoyanovich
et al., 2018)). Further, both problem formulations are not restricted to these
specific utility models and future work should explore other models that
ensure the desired screener behavior. Under such alternative utility models,
though, Algorithms 1 and 2 and, in turn, Remark 1 should still hold, thus,
illustrating the importance of distinguishing between these two problem
formulations.

Other assumptions, like the choices, both function- (Section 3.1) and
parameter-wise (Section 4.1) regarding the human-like screener’s fatigue are
likely to be contested and limit the implications of our analysis. We
recognize this limitation, though we argue that we made such choices mainly to
study the human-like screener (with the fatigue term) under an ISO chosen or
provided. We made the simplest modeling assumptions for the fatigue, but
always keeping in mind what we observed at G and know from previous empirical
works (Echterhoff et al., 2022; Pei et al., 2023). Indeed, under a different
fatigue model, Remark 2 may not hold and $h_{h}$ may not violate individual
fairness; similarly, under different fatigue parameters, the experimental
results in Section 4.3 may motivate a different analysis. Note, though, that
the ISO would still play a central role in any variation of the current
assumptions.

We stress that our modeling and simulation frameworks are flexible enough to
account for other screening settings. The experimental setup studied here is
one of many possible screening scenarios that we could consider. Similarly,
the framework could consider the role of the ISO under a different fatigue
term. What matters is studying the role of the ISO and, in particular, its
effect on the human screener. It is costly and time-consuming to run real
experiments with human screeners and candidates, especially, at the same scale
(10000 runs per setting) of Section 4. We view our work as another example
(Ionescu et al., 2021; Jr. et al., 2020; Bountouridis et al., 2019; Bokányi
and Hannák, 2020; Schelling, 1971) of how simulations are useful tools for
studying the fairness and optimality of real-world decision-making processes.

For instance, based on the experiments presented in Section 4, a company like
G could revisit the current screening practices of its HR officers. Informed
by the simulations, instead of running the sequential, partial search
(Algorithm 2) for the good-$k$, the company could, e.g., recommend its HR
officers to take breaks after screening a certain number of candidates in the
ISO to ensure that the fatigue does not accumulate. Similarly, such a new
search procedure for the good-$k$ problem could be implemented and tested
within our simulation framework as it would still be addressing the ISO
problem. Overall, such an approach would be a better alternative for the
company to experiment on real screeners and candidates. Note that the company
could carry out these analyses considering an ISO that is either chosen by or
provided to the screener; in turn, it would allow for studying settings
involving human screeners alone and human screeners relying on some form of
algorithmic aid (Mosqueira-Rey et al., 2023).

Thus, future work could explore alternative utility models and fatigue terms
while relying on the current simulations framework. For instance, an
interesting alternative formulation for fatigue could involve a human-like
screener that rests while searching the candidate pool under the ISO. We see
recurrent survival models (Chandar et al., 2022) well suited for this task.
Further, defining a human-like screener is not limited to fatigue. Future work
could also explore theories on human decision-making (see (Jannach et al.,
2010, Ch. 10) and (Caraban et al., 2019; Adomavicius et al., 2013)).
Furthermore, the simulations framework could be extended to test for optimal
parameters in the ISO problem. For example, finding an optimal minimum score
$\psi$ for which the two problem formulations coincide and making the search
procedure less influential.

To conclude, we summarize three main takeaways from our analysis of the ISO
problem. First, defining the adequate problem formulation is important for
understanding the impact of the ISO on the selected set of candidates (Remark
1), which reflects the search procedures of the screeners regardless of the
kind. Second, once the search procedure is clear, it is important to
understand how the screener behaves as it searches over the ISO. This is most
important for the human-like screener (with fatigue) and the fair individual
treatment of each candidate in the pool (Remark 2). Third, simulations are
beneficial for understanding the implications of the interaction between the
screener and the ISO on the selected candidates. For example, the composition
of the candidate pool can amplify or diminish the influence of the position
bias in the ISO (as in Sections 4.2 and 4.3). All three takeaways have direct
implications for practitioners. These insights might seem obvious ex-post,
though they are supported by an extensive analysis that accounts for several
factors that only become clear under modeling and experiments.

###### Acknowledgements.

Jose M. Alvarez and Salvatore Ruggieri received funding from the European
Union’s Horizon 2020 research and innovation program under Marie Sklodowska-
Curie Actions (grant agreement number 860630) for the project ”NoBIAS -
Artificial Intelligence without Bias”. Antonio Mastropietro and Salvatore
Ruggieri received funding from the European Union’s Horizon Europe (grant
agreement number 101070212) for the FINDHR project. This work reflects only
the authors’ views and the European Research Executive Agency is not
responsible for any use that may be made of the information it contains. No
other funding was received by the authors.

## References

  * (1)
  * Adomavicius et al. (2013) Gediminas Adomavicius, Jesse C. Bockstedt, Shawn P. Curley, and Jingjing Zhang. 2013.  Do Recommender Systems Manipulate Consumer Preferences? A Study of Anchoring Effects.  _Inf. Syst. Res._ 24, 4 (2013), 956–975.
  * Athey and Ellison (2011) Susan Athey and Glenn Ellison. 2011.  Position Auctions with Consumer Search.  _The Quarterly Journal of Economics_ 126, 3 (2011), 1213–1270.
  * Baeza-Yates (2018) Ricardo Baeza-Yates. 2018.  Bias on the web.  _Commun. ACM_ 61, 6 (2018), 54–61.
  * Bokányi and Hannák (2020) Eszter Bokányi and Anikó Hannák. 2020.  Understanding inequalities in ride-hailing services through simulations.  _Scientific reports_ 10, 1 (2020), 1–11.
  * Botev (2017) Z. I. Botev. 2017.  The normal law under linear restrictions: simulation and estimation via minimax tilting.  _Journal of the Royal Statistical Society Series B_ 79, 1 (2017), 125–148.
  * Bountouridis et al. (2019) Dimitrios Bountouridis, Jaron Harambam, Mykola Makhortykh, Mónica Marrero, Nava Tintarev, and Claudia Hauff. 2019.  SIREN: A Simulation Framework for Understanding the Effects of Recommender Systems in Online News Environments. In _FAT_. ACM, 150–159.
  * Bringas Colmenarejo et al. (2022) Alejandra Bringas Colmenarejo, Luca Nannini, Alisa Rieger, Kristen M Scott, Xuan Zhao, Gourab K Patro, Gjergji Kasneci, and Katharina Kinder-Kurlanda. 2022.  Fairness in agreement with European values: An interdisciplinary perspective on ai regulation. In _AIES_. ACM, 107–118.
  * Caraban et al. (2019) Ana Karina Caraban, Evangelos Karapanos, Daniel Gonçalves, and Pedro F. Campos. 2019.  23 Ways to Nudge: A Review of Technology-Mediated Nudging in Human-Computer Interaction. In _CHI_. ACM, 503.
  * Carriço (2018) Gonçalo Carriço. 2018.  The EU and artificial intelligence: A human-centred perspective.  _European View_ 17, 1 (2018), 29–36.
  * Chandar et al. (2022) Praveen Chandar, Brian St. Thomas, Lucas Maystre, Vijay Pappu, Roberto Sanchis-Ojeda, Tiffany Wu, Ben Carterette, Mounia Lalmas, and Tony Jebara. 2022.  Using Survival Models to Estimate User Engagement in Online Experiments. In _WWW_. ACM, 3186–3195.
  * Craswell et al. (2008) Nick Craswell, Onno Zoeter, Michael J. Taylor, and Bill Ramsey. 2008.  An experimental comparison of click position-bias models. In _WSDM_. ACM, 87–94.
  * Dwork et al. (2012) Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard S. Zemel. 2012.  Fairness through awareness. In _ITCS_. ACM, 214–226.
  * Echterhoff et al. (2022) Jessica Maria Echterhoff, Matin Yarmand, and Julian J. McAuley. 2022.  AI-Moderated Decision-Making: Capturing and Balancing Anchoring Bias in Sequential Decision Tasks. In _CHI_. ACM, 161:1–161:9.
  * Embrechts et al. (2003) Paul Embrechts, Filip Lindskog, and Alexander Mcneil. 2003.  Modelling Dependence with Copulas and Applications to Risk Management.  In _Handbook of Heavy Tailed Distributions in Finance_ , Svetlozar T. Rachev (Ed.). Vol. 1. North-Holland, Amsterdam, 329–384.
  * Fabris et al. (2023) Alessandro Fabris, Nina Baranowska, Matthew J. Dennis, Philipp Hacker, Jorge Saldivar, Frederik J. Zuiderveen Borgesius, and Asia J. Biega. 2023.  Fairness and Bias in Algorithmic Hiring.  _CoRR_ abs/2309.13933 (2023).
  * Fagin et al. (2003) Ronald Fagin, Amnon Lotem, and Moni Naor. 2003.  Optimal aggregation algorithms for middleware.  _J. Comput. Syst. Sci._ 66, 4 (2003), 614–656.
  * Ferguson (1989) Thomas S Ferguson. 1989.  Who solved the secretary problem?  _Statist. Sci._ 4, 3 (1989), 282–289.
  * Grotov et al. (2015) Artem Grotov, Aleksandr Chuklin, Ilya Markov, Luka Stout, Finde Xumara, and Maarten de Rijke. 2015.  A Comparative Study of Click Models for Web Search. In _CLEF_ _(Lecture Notes in Computer Science, Vol. 9283)_. Springer, 78–90.
  * Ionescu et al. (2021) Stefania Ionescu, Anikó Hannák, and Kenneth Joseph. 2021.  An Agent-based Model to Evaluate Interventions on Online Dating Platforms to Decrease Racial Homogamy. In _FAccT_. ACM, 412–423.
  * Jannach et al. (2010) Dietmar Jannach, Markus Zanker, Alexander Felfernig, and Gerhard Friedrich. 2010.  _Recommender Systems - An Introduction_.  Cambridge University Press.
  * Joachims et al. (2017) Thorsten Joachims, Laura A. Granka, Bing Pan, Helene Hembrooke, and Geri Gay. 2017.  Accurately Interpreting Clickthrough Data as Implicit Feedback.  _SIGIR Forum_ 51, 1 (2017), 4–11.
  * Jr. et al. (2020) Donald Martin Jr., Vinodkumar Prabhakaran, Jill Kuhlberg, Andrew Smart, and William S. Isaac. 2020.  Extending the Machine Learning Abstraction Boundary: A Complex Systems Approach to Incorporate Societal Context.  _CoRR_ abs/2006.09663 (2020).
  * Kahneman (2011) Daniel Kahneman. 2011.  _Thinking, Fast and Slow_.  Farrar, Straus and Giroux.
  * Kahneman et al. (2021) Daniel Kahneman, Olivier Sibony, and Cass Sunstein. 2021.  _Noise: A Flaw in Human Judgment_.  William Collins.
  * Kazim et al. (2021) Emre Kazim, Adriano Soares Koshiyama, Airlie Hilliard, and Roseline Polle. 2021.  Systematizing Audit in Algorithmic Recruitment.  _Journal of Intelligence_ 9, 3 (2021).
  * Köchling and Wehner (2020) Alina Köchling and Marius Claus Wehner. 2020.  Discriminated by an algorithm: a systematic review of discrimination and fairness by algorithmic decision-making in the context of HR recruitment and HR development.  _Business Research_ 13, 3 (2020).
  * Mehrotra and Celis (2021) Anay Mehrotra and L Elisa Celis. 2021.  Mitigating bias in set selection with noisy protected attributes. In _Proceedings of the 2021 ACM conference on fairness, accountability, and transparency_. 237–248.
  * Mosqueira-Rey et al. (2023) Eduardo Mosqueira-Rey, Elena Hernández-Pereira, David Alonso-Ríos, José Bobes-Bascarán, and Ángel Fernández-Leal. 2023.  Human-in-the-loop machine learning: A state of the art.  _Artif. Intell. Rev._ 56, 4 (2023), 3005–3054.
  * Pan et al. (2007) Bing Pan, Helene Hembrooke, Thorsten Joachims, Lori Lorigo, Geri Gay, and Laura A. Granka. 2007.  In Google We Trust: Users’ Decisions on Rank, Position, and Relevance.  _J. Comput. Mediat. Commun._ 12, 3 (2007), 801–823.
  * Pei et al. (2023) Jiaxin Pei, Zhihan Helen Wang, and Jun Li. 2023.  30 Million Canvas Grading Records Reveal Widespread Sequential Bias and System-Induced Surname Initial Disparity.  (2023).
  * Pisanelli (2022) Elena Pisanelli. 2022.  Your resume is your gatekeeper: Automated resume screening as a strategy to reduce gender gaps in hiring.  _Economics Letters_ 221 (2022), 110892.
  * Pitoura et al. (2022) Evaggelia Pitoura, Kostas Stefanidis, and Georgia Koutrika. 2022.  Fairness in rankings and recommendations: an overview.  _VLDB J._ 31, 3 (2022), 431–458.
  * R Core Team (2024) R Core Team. 2024.  _R: A Language and Environment for Statistical Computing_.  R Foundation for Statistical Computing, Vienna, Austria.  https://www.R-project.org/
  * Raghavan et al. (2020) Manish Raghavan, Solon Barocas, Jon M. Kleinberg, and Karen Levy. 2020.  Mitigating bias in algorithmic hiring: evaluating claims and practices. In _FAT*_. ACM, 469–481.
  * Rhea et al. (2022) Alene K. Rhea, Kelsey Markey, Lauren D’Arinzo, Hilke Schellmann, Mona Sloane, Paul Squires, Falaah Arif Khan, and Julia Stoyanovich. 2022.  An external stability audit framework to test the validity of personality prediction in AI hiring.  _Data Min. Knowl. Discov._ 36, 6 (2022), 2153–2193.
  * Richardson et al. (2007) Matthew Richardson, Ewa Dominowska, and Robert Ragno. 2007.  Predicting clicks: estimating the click-through rate for new ads. In _WWW_. ACM, 521–530.
  * Ruggieri et al. (2023) Salvatore Ruggieri, José M. Álvarez, Andrea Pugnana, Laura State, and Franco Turini. 2023.  Can We Trust Fair-AI?. In _AAAI_. AAAI Press, 15421–15430.
  * Schelling (1971) Thomas C Schelling. 1971.  Dynamic models of segregation.  _Journal of Mathematical Sociology_ 1, 2 (1971), 143–186.
  * Singh and Joachims (2018) Ashudeep Singh and Thorsten Joachims. 2018.  Fairness of Exposure in Rankings. In _KDD_. ACM, 2219–2228.
  * Sloane et al. (2022) Mona Sloane, Emanuel Moss, and Rumman Chowdhury. 2022.  A Silicon Valley love triangle: Hiring algorithms, pseudo-science, and the quest for auditability.  _Patterns_ 3, 2 (2022), 100425.
  * Sonderling et al. (2022) Keith E. Sonderling, Bradford J. Kelley, and Lance Casimir. 2022.  The Promise and The Peril: Artificial Intelligence and Employment Discrimination.  _U. Miami Law Review_ 77, 1 (2022), 3.
  * Stoyanovich et al. (2018) Julia Stoyanovich, Ke Yang, and H. V. Jagadish. 2018.  Online Set Selection with Fairness and Diversity Constraints. In _EDBT_. OpenProceedings.org, 241–252.
  * Sukumar et al. (2018) Poorna Talkad Sukumar, Ronald A. Metoyer, and Shuai He. 2018.  Making a Pecan Pie: Understanding and Supporting The Holistic Review Process in Admissions.  _Proc. ACM Hum. Comput. Interact._ 2, CSCW (2018), 169:1–169:22.
  * Tversky and Kahneman (1974) Amos Tversky and Daniel Kahneman. 1974.  Judgment Under Uncertainty: Heuristics and Biases.  _Science_ 185, 4157 (1974), 1124–1131.
  * Will et al. (2023) Paris Will, Dario Krpan, and Grace Lordan. 2023.  People versus machines: introducing the HIRE framework.  _Artif. Intell. Rev._ 56, 2 (2023), 1071–1100.
  * Wilson et al. (2021) Christo Wilson, Avijit Ghosh, Shan Jiang, Alan Mislove, Lewis Baker, Janelle Szary, Kelly Trindel, and Frida Polli. 2021.  Building and Auditing Fair Algorithms: A Case Study in Candidate Screening. In _FAccT_. ACM, 666–677.
  * Yang and Stoyanovich (2017) Ke Yang and Julia Stoyanovich. 2017.  Measuring Fairness in Ranked Outputs. In _SSDBM_. ACM, 22:1–22:6.
  * Zehlike et al. (2017) Meike Zehlike, Francesco Bonchi, Carlos Castillo, Sara Hajian, Mohamed Megahed, and Ricardo Baeza-Yates. 2017.  FA*IR: A Fair Top-k Ranking Algorithm. In _CIKM_. ACM, 1569–1578.
  * Zehlike et al. (2023a) Meike Zehlike, Ke Yang, and Julia Stoyanovich. 2023a.  Fairness in Ranking, Part I: Score-Based Ranking.  _ACM Comput. Surv._ 55, 6 (2023), 118:1–118:36.
  * Zehlike et al. (2023b) Meike Zehlike, Ke Yang, and Julia Stoyanovich. 2023b.  Fairness in Ranking, Part II: Learning-to-Rank and Recommender Systems.  _ACM Comput. Surv._ 55, 6 (2023), 117:1–117:41.

##  Appendix A Collaborating with G

##### Overall experience.

The collaboration lasted for four months. Due to the COVID-19 pandemic, it was
a hybrid collaboration. During this time, we mostly interviewed the HR
officers to understand their tasks, constraints, and methodologies, often
shadowing them during screening sessions. We were also granted access to Taleo
by Oracle, the platform used by HR for managing the hiring pipeline. This
allowed us to experience for ourselves the patterns we observed among the HR
officers. These patterns resulted in the stylised facts of Section 1.1.

We interacted, in particular, with five HR officers specialized in screening
applications for technical roles within G, such as the roles of data scientist
and front-end developer. These HR officers had to process considerable amounts
of information within a time constraint. Based on what we observed, most
candidate pools (except for very senior profiles like, e.g., director of data
science) consisted of hundred of applications. These HR officers were involved
in screening multiple candidate pools with similar deadlines within the same
week. It became apparent to us, specially for candidate screening, how time-
consuming and human-dependent was the hiring process.

We discussed our observations, often on a bi-weekly basis, to members of both
the HR and AA teams. We emphasize that we simply collaborated with these teams
as equals, sharing the goal of understanding G’s hiring process and whether it
was suitable or not for (fair) automation. We specifically use the wording
“stylized facts” in Section 1.1 to emphasize that we draw inspiration from the
collaboration with G in formulating the ISO problem rather than “hard facts”
about G derived from an observational study. In fact, the first draft of this
paper occurred about one year after our collaboration with G had concluded.

##### Deliverables.

We concluded the collaboration with a report to AA that formalized G’s
candidate screening process as a ranking problem, evaluated the potential
fairness implications, and assessed the risk and benefits of automation. The
report was discussed with a wider audience within G in a series of
presentations hosted by the AA and HR teams. The report focused mainly on
candidate screening. It is worth mentioning also that throughout the
collaboration, we followed G’s strict ethical guidelines at all times.
Further, at no point did we receive monetary compensation from G. Our research
has been funded by public European institutions. The views reflected in this
paper are entirely our own. No sensitive data (or data at all) from G was used
for this work. This paper is not a deliverable specific to G.

##### The hiring pipeline.

Hiring at G consists mainly of three phases. With respect to the stylized
facts, these are concerned with the candidate screening phase or phase two.
These phases are the following:

  * •

In phase one, the HR builds a candidate pool for the job opening. Candidates
submit their CVs, complete a multiple-choice questioner, and write a
motivation letter. Sensitive information, such as gender, ethnicity, and age,
is also provided or it can be inferred. The candidate pool is stored in a
database platform.

  * •

In phase two, the HR officer reduces the candidate pool into a smaller pool of
suitable candidates. The HR officer determines candidate suitability based on
each candidate’s profile using a set of minimum basic requirements.

  * •

In phase three, the chosen candidates are interviewed by HR and the team
offering the job. It is common for the hiring team to also prepare a use case
for the candidates. The best candidates receive an offer. If no candidates are
hired, HR resorts to the runner-up candidates from phase two and repeat phase
three.

We note that the above represent G’s hiring pipeline during the collaboration.
We do not know if this hiring pipeline is still the case today, though this is
not important for the purpose of this paper.

##  Appendix B Supplementary Material

###  B.1. Section 2.3.2: Additional Discussion on the Utility Model

First, we present Example B.1 below. It shows that for the utility model (5)
for the good-$k$ problem, in the general case, i.e., $f(S^{k})\geq q$, there
can be two solutions, but with different fractions of the protected group.

###### Example B.1.

Consider $n=3,k=2,q=0.5$. Assume three eligible candidates and
$\theta(1)=c_{1},\theta(2)=c_{2},\theta(3)=c_{3}$ with $W_{1}=0$ and
$W_{2}=W_{3}=1$. Both $S^{\prime}=\\{c_{1},c_{2}\\}$ and
$S^{\prime\prime}=\\{c_{2},c_{3}\\}$ are solutions of (6) with
$U^{k}_{\psi}\big{(}S^{\prime},\theta\big{)}=U^{k}_{\psi}\big{(}S^{\prime\prime},\theta\big{)}=2$.
However, $f(S^{\prime})=0.5$ and $f(S^{\prime\prime})=1$. Intuitively,
$S^{\prime}$ is obtained by strictly iterating over $\theta$.

We now motivate the alternative utility model (5) used in the good-$k$
problem. As a possible utility function in the good-$k$ setting consider the
following utility model:

(7)
$\hat{U}^{k}_{\psi}\big{(}S^{k},\theta\big{)}=\left\\{\begin{array}[]{ll}n-\max_{c\in
S^{k}}\theta^{-1}(c)&\text{if}\ \forall c\in S^{k}\
s(\mathbf{X}_{c})\geq\psi\\\ 0&\text{otherwise.}\end{array}\right.$

where we use $\hat{U}_{\psi}^{k}$ (i.e., the $U$ hat) to differentiate from
the utility model $U_{\psi}^{k}$ considered in (5).

Intuitively, in the above utility definition the screener wants to find as
quickly as possible a set of $k$ eligible candidates. Therefore, if $S^{k}$
contains only eligible candidates, the utility of $h$ selecting $S^{k}$ under
$\theta$ is expressed by the number of candidates past the last one who was
screened, i.e., the “saved effort” of the screener $h$. Despite the simplicity
of (7), the above utility model (7) is not suitable to properly account for
the intended good-$k$ problem. To observe this last point, consider Example
B.2.

###### Example B.2.

Let $n=3,k=2,q=0.5$. Assume three eligible candidates and
$\theta(1)=c_{1},\theta(2)=c_{2},\theta(3)=c_{3}$ with $W_{1}=W_{2}=0$ and
$W_{3}=1$. It turns out that both $S^{\prime}=\\{c_{1},c_{3}\\}$ and
$S^{\prime\prime}=\\{c_{2},c_{3}\\}$ maximize the utility and satisfy the
fairness constraint $q$.

Following up on Example B.2, why should have been $c_{2}$ considered, and then
returned in $S^{\prime\prime}$, if $c_{1}$ already meets the minimum basic
requirement? A reason for doing that is a variant of our good-$k$ problem in
which the screener $h$ keeps evaluating non-protected candidates in
$\mathcal{C}$, even if their quota is reached but the one of protected
candidates is not yet reached, for the purpose of keeping the best ones found
so far. We do not consider such a variant in this paper. For this reason, we
introduce the penalty function in (5) presented in Section 2.3.2.

###  B.2. Search Procedures under a Human-Like Screener

We update the ExaminationSearch and CascadeSearch and their corresponding
Algorithm 1 and Algorithm 2 from Section 2.4 under the human-like screener
$h_{h}$ from Section 3. We incorporate the fatigued scores formulation from
Section 3.1 into both algorithms, resulting in a human-like
HuamnExaminationSearch (Algorithm 3) and a human-like HumanCascadeSearch
(Algorithm 4). In comparison to the algorithmic screener $h_{a}$, the main
difference here is that both algorithms compute the fatigued score for
candidate $c\in\mathcal{C}$:

(8) $Y_{c}=s(\mathbf{X}_{c})+\epsilon$

where $\epsilon$ is a random variable depending on the accumulated fatigue
$\Phi$ of $h_{h}$. In both algorithms 3 and 4, by requiring the fatigue
component $\Phi$, we also require a specific modeling choice for $\epsilon$
which is a probabilistic function of the accumulated fatigue $\Phi$. As
discussed in Section 3.1, $\epsilon$ can be modeled as either $\epsilon_{1}$
or $\epsilon_{2}$.

We stress once again that other formulations for $\epsilon$ are possible.
These formulations are compatible with Algorithms 3 and 4 as long as
$\epsilon$ is treated as a random variable that is drawn each time $h_{h}$
evaluates candidate $c$. These formulations can be implemented as with
$\epsilon_{1}$ and $\epsilon_{2}$, meaning by providing the corresponding
probability distribution for the intended accumulated fatigue $\Phi$.

Algorithm 3 HumanLikeExaminationSearch

1:$n$, $\theta$, $k$, $q$, $\Phi$

2:$S^{k}_{\text{{best}}}$

3:$q^{*}\,\leftarrow$ round($q\cdot k$); $r^{*}\,\leftarrow k-q^{*}$

4:$\mathit{scores}\,\leftarrow[s(\mathbf{X}_{\theta(t)})+\epsilon(\Phi(t-1))\
\mathit{for}\ t=1,\ldots,n]$

5:$\tau\leftarrow\texttt{argsortdesc(}\mathit{scores}$)

6:$i\,\leftarrow 1$; $k^{*}\,\leftarrow 0$; $Q\,\leftarrow\\{\\}$;
$R\,\leftarrow\\{\\}$

7:while $k^{*}<k$ do

8: $c\;\leftarrow\theta(\tau(i))$; $i\,\leftarrow i+1$

9: if $W_{c}==0$ and len($R$)$==r^{*}$ then

10: continue

11: $k^{*}\,\leftarrow k^{*}+1$

12: if $W_{c}==1$ and $\texttt{len(}Q\texttt{)}<q^{*}$ then

13: $Q\,\leftarrow Q\cup\\{c\\}$

14: else

15: $R\,\leftarrow R\cup\\{c\\}$ return $Q\cup R$

Algorithm 4 HumanLikeCascadeSearch

1:$n$, $\theta$, $k$, $q$, $\psi$, $\Phi$

2:$S_{\text{{good}}}^{k}$

3:$q^{*}\,\leftarrow$ round($q\cdot k$); $r^{*}\,\leftarrow k-q^{*}$

4:$i\,\leftarrow 1$; $k^{*}\,\leftarrow 0$; $Q\,\leftarrow\\{\\}$;
$R\,\leftarrow\\{\\}$; $t\,\leftarrow 1$

5:while $k^{*}<k$ do

6: $c\;\leftarrow\theta(i)$; $i\,\leftarrow i+1$

7: if $W_{c}==0$ and len($R$)$==r^{*}$ then

8: continue

9: $Y_{c}\;\leftarrow s(\mathbf{X}_{c})+\epsilon(\Phi(t-1)$); $t\,\leftarrow
t+1$

10: if $Y_{c}\geq\psi$ then

11: $k^{*}\,\leftarrow k^{*}+1$

12: if $W_{c}==1$ and $\texttt{len(}Q\texttt{)}<q^{*}$ then

13: $Q\,\leftarrow Q\cup\\{c\\}$

14: else

15: $R\,\leftarrow R\cup\\{c\\}$ return $Q\cup R$

##  Appendix C Additional Experiments

###  C.1. Experiments without Fatigue

In this section, we present the additional figures and corresponding
discussions relating to the experimental analysis of the algorithmic screener
$h_{a}$ from Section 4.2.

First, we consider the impact of the correlation $\rho$ between $\theta$ and
the scores. Recall that $\rho=-1$ means that the candidates are ordered by
descending scores. Under such a condition, the good-$k$ and best-$k$
procedures return the same solution. This result is apparent in Figure 4
(center, right) where we report the ratio to baseline for the symmetric (left)
and the increasing (right) score distributions. The plots show that even a
moderate correlation of $\rho=-0.5$ leads the good-$k$ solution to approximate
the best-$k$ one quite well. For the increasing distribution (right), the
ratio to baseline is around 95%. In summary, initial orders that negatively
correlate to the score greatly reduce the difference in utility between the
good-$k$ and best-$k$ solutions.

Second, we consider the impact of the number $n$ candidates in $\mathcal{C}$
and the number of $k$ candidates to be selected. We focus only on the
symmetric distribution (in red) and the RtB metric, but the results are
similar for the other two distributions (in blue and green) and the JdS
metric. Figure 4 (left) compares the case $n=120,k=6$ considered earlier to
two other scenarios in terms of $n$ and $k$. The first scenario increases
$k=20$, but leaves the ratio of selected $k/n=0.05$ the same by also
increasing $n=400$. The second scenario, instead, leaves $k=6$ the same, but
it increases $k/n=0.2$ by decreasing $n=30$. The plot shows that changes in
the ratio $k/n$ affect the metric, in particular a larger ratio ($n=30$,
$k=6$) leads good-$k$ to better approximate best-$k$ for the same $\psi$. In
other words, the more candidates we can select from the pool, the better the
chance to include top ones. In summary, under
$\theta\mathrel{{\scalebox{1.07}{$\perp\mkern-10.0mu\perp$}}}s$, the ratio
$k/n$ is positively correlated with the ability of best-$k$ to approximate
good-$k$.

Finally, let us now consider the quota parameter $q$, thus far set to $q=0.5$
over a population with a fraction of protected candidates set to
$\mathit{pr}=0.2$. Since we assumed that $W$ is independent from both scores
and the ISO, the fraction of protected group in the solutions of best-$k$ and
good-$k$ is, on average, $\mathit{min}\\{q,\mathit{pr}\\}$. Figure 5 (left)
shows this result in the solution for good-$k$. A less trivial question is
whether $q$ is also not affecting the evaluation metrics: e.g., whether the
quota $q$ changes the ratio to baseline? Figure 5 (center, right) show that
this is not the case in two experimental settings. Again, this result is
theoretically implied by the independence of $W$ with scores and initial
order. In summary, under
$\theta\mathrel{{\scalebox{1.07}{$\perp\mkern-10.0mu\perp$}}}s$, the $q$ in
the best-$k$ and good-$k$ problem does not affect the relative strengths of
their solutions.

Figure 4.  Left: RtB for different $k/n$ ratios, for the $tN(0.5,0.02)$ score
distribution and with setting $q=0.5$,
$\theta\mathrel{{\scalebox{1.07}{$\perp\mkern-10.0mu\perp$}}}s$. Center: RtB
for different $\rho$’s for the $tN(0.5,0.02)$ score distribution and with
setting $n=120$, $k=6$, and $q=0.5$. Right: RtB for different $\rho$’s for the
$tN(1,0.05)$ score distribution and with setting $n=120$, $k=6$, $q=0.5$.

Figure 5.  Left: fraction of protected candidates in the solution of good-$k$
for different representational quotas $q$, for the $tN(0.5,0.02)$ score
distribution and with setting $n=400$, $k=20$,
$\theta\mathrel{{\scalebox{1.07}{$\perp\mkern-10.0mu\perp$}}}s$. Center: RtB
for different representational quotas $q$, for the $tN(0.5,0.02)$ score
distribution and with setting $n=400$, $k=20$,
$\theta\mathrel{{\scalebox{1.07}{$\perp\mkern-10.0mu\perp$}}}s$. Right: RtB
for different representational quotas $q$, for the $tN(1,0.05)$ score
distribution and with setting $n=400$, $k=20$,
$\theta\mathrel{{\scalebox{1.07}{$\perp\mkern-10.0mu\perp$}}}s$.