Paper Title: Robust Restaking Networks.


 



















theoremTheorem
lemmaLemma
corollaryCorollary
claimClaim
caseCase


solution
  
  
  


amatrix[1]
  [[ ]

































































Robust Restaking Networks
    Naveen DurvasulaColumbia University. Email: . Tim RoughgardenColumbia University & a16z crypto. Email: .
    August 5, 2024
============================================================================================================



§ ABSTRACT 
We study the risks of validator reuse across multiple services in a
restaking protocol. We characterize the robust security of a restaking
network as a function of the buffer between the costs and profits from
attacks.  For example, our results imply that if attack costs
always exceed attack profits by 10%, then a sudden loss of .1% of
the overall stake (e.g., due to a software error) cannot result in the
ultimate loss of more than 1.1% of the overall stake.
We also provide local analogs of these overcollateralization conditions
and robust security guarantees that apply specifically for a target
service or coalition of services. All of our bounds on worst-case
stake loss are the best possible. Finally, we bound the
maximum-possible length of a cascade of attacks.

Our results suggest measures of robustness that could be exposed to
the participants in a restaking protocol. We also suggest
polynomial-time computable sufficient conditions that can proxy for
these measures.




§ INTRODUCTION




 §.§ Sharing Validators Across Services


Major blockchain protocols such as Bitcoin or Ethereum are
“decentralized,” meaning that transaction execution is carried
out by a large and diverse set of “validators.” 
Such protocols offer a form of “trusted computation,” in the sense
that, because they are decentralized, no one individual or entity can
easily interfere with their execution. A decentralized and
Turing-complete smart contract platform such as Ethereum can then be
viewed as a trusted programmable computer capable of performing
arbitrary computations.

While Turing-complete, the computing functionality offered by Ethereum
smart contracts suffers from limitations imposed by design decisions
in the underlying consensus protocol. Most obviously, computation and
storage in the Ethereum virtual machine is scarce, with perhaps 15–20
transactions processed per second.  Could the Ethereum protocol be
somehow bypassed, opening the door for different or more powerful
computing functionality, while retaining at least some of the
protocol's decentralization?  Or, what about applications that are not
compatible with all Ethereum validators, perhaps due to demanding
hardware requirements or regulatory constraints?

One natural approach to addressing these challenges is to allow the
reuse of a blockchain protocol's validators across multiple
services, where a “service” is some task that could be carried out
by some subset of validators. (The initial blockchain protocol can be viewed as the
canonical service, performed by all validators.)
For example, such services
could include alternative consensus protocols
(perhaps with higher throughput, a different virtual machine, or
different consistency-liveness trade-offs), storage (“data
availability”), or verifiable off-chain computation (“zk
coprocessors”).[
Several projects, in various stages of production, are currently
exploring this idea; the Eigenlayer restaking protocol <cit.>
is perhaps the most well known of them. We stress that our goal here
is to develop a model that isolates some of the fundamental challenges
and risks of validator reuse, independent of any specific
implementation of the idea.]

The obvious danger of validator reuse is an increased risk of
a validator deviating from its intended behavior (e.g., due to overwhelming computational
responsibilities). Our focus here is deliberate validator
deviations in response to economic incentives, such as the profits that could
be obtained by corrupting one or more services. The goal of this paper is to
quantify such risks:

Under what conditions can validators be safely reused across
multiple services?







 §.§ Cryptoeconomic Security


We first review the usual “cryptoeconomic” approach to answering a
more basic question: when is a blockchain protocol, without any
additional services, “safe from attack”?  The idea is to perform a
cost-benefit analysis from the perspective of an attacker, and declare
the protocol safe if the cost of carrying out an attack
exceeds the profit that the attacker can expect from it; see also Figure <ref>.
In the
specific case of a proof-of-stake blockchain protocol with slashing
(such as Ethereum), the cost can be estimated as the value of the
validator stake that would be lost to slashing following an attack. 
For example, let V denote the set of validators of a proof-of-stake
blockchain protocol and σ_v the stake of validator v ∈ V
(e.g., 32 ETH in Ethereum). In a typical PBFT-type protocol in which
double-voting validators lose all of their stake, the 
cost of an attack (i.e., causing a consistency violation) can be
bounded below by 13∑_v ∈ Vσ_v. In this case,
the protocol can be regarded as cryptoeconomically secure provided
the estimated profit π of an attack is less than this quantity.
To the extent that there is a “buffer” between 13∑_v
  ∈ Vσ_v and π, the protocol can be treated as
“robustly secure,” meaning secure even after a sudden loss of some
amount of stake (e.g., due to slashing caused by a software error).







 §.§ Our Results: Robustly Secure Validator Reuse


Viewing the basic scenario of a blockchain protocol as a star graph
(with one “service” representing the protocol connected to all the
validators running it), the more complex scenario of validators reused
across multiple services can be viewed as an arbitrary bipartite
graph (Figure <ref>). As before, we suppose that each
validator v ∈ V has some
stake σ_v that can be confiscated in the event that v
participates in an attack on a service that it has agreed to
support. There is now a set S of services, with π_s denoting the
profit an attacker would obtain by compromising the service s ∈
S.[We follow <cit.> and assume that the estimates on
  attack profitability (the π_s's) are given. Developing tools to
  help produce such estimates in practice is an important open
  research direction.]
We assume that compromising a service s ∈ S requires the
participation of an α_s fraction of the overall validator stake
supporting v (e.g., α_s = 1/3).
With this expanded network formalism to capture multiple services,
when should we consider a network to be “secure”?



Intuitively, a network is insecure if there is a set of services that
can be profitably attacked. This requires two conditions to be
satisfied: the validators B ⊆ V carrying out the attack must
control sufficient stake to do so (for each service s in the
attacked set A, the validators of B control at least a α_s
fraction of the staked pledged to s), and they must profit from it
(i.e., ∑_s ∈ Aπ_s > ∑_v ∈ Bσ_v).
We call such a pair (A,B) a valid attack and a network secure if there are no valid attacks; see also Figure <ref>.
With multiple services,
security is an inherently combinatorial (as opposed to binary)
notion. For example, the computational problem of checking whether a network is secure is 
as hard as the (coNP-hard) problem of verifying the expansion of a bipartite graph (see <cit.>).



When is a network “robustly secure,” in the sense that the sudden
loss of a small amount of stake cannot enable a catastrophic attack?
Unlike the “all-or-nothing” version of this question with a single
service, with multiple services, the following more fine-grained question is the appropriate one: given an initial shock in the
form of a sudden loss of a ψ fraction of the overall stake, what
is the total fraction of stake that might be lost following any
consequent valid attacks?

As in the case of a single service, some amount of “buffer” in stake
(relative to attack profits) is necessary for robust security. We
parameterize this overcollateralization factor via a parameter γ and suppose that,
whenever B ⊆ V is a subset of validators capable of
corrupting all the services in A ⊆ S, the total stake
of B is at least a 1+γ factor larger than the total profit from
corrupting all of A. For example, this condition holds in the
network in Figure <ref>(b) for γ=1/2 (but not for larger
values of γ).

Our first main result (<ref>) precisely characterizes the
worst-case (over bipartite graphs and shocks, as a function
of γ) fraction of the overall
stake that can be lost due to a shock
of size ψ: 1+1γψ. Because the network
was secure prior to the shock, the value of a
1+1γψ fraction of the overall stake is also
an upper bound on the total profit obtained from all of the attacked
services.

We also show that our result is tight
in a strong sense (<ref>, <ref>, and
<ref>): for every ψ, γ, and ϵ
greater than zero such that 0 ≤1 + 1/γψ - ϵ≤ 1,
there exists a restaking graph in which a ψ
fraction of the overall stake can disappear in a shock that results in
the loss of at least a 1 + 1/γψ - ϵ fraction
of the overall stake.[After slightly reducing the validator
  stakes, the network
  in Figure <ref>(b) already shows that the bound is tight for
  the special case in which ψ = 1/5 and γ is arbitrarily
  close to 1/2. (Consider a shock that knocks out the validator that
is connected to all four services.)]

Qualitatively, this result implies that a constant-factor
strengthening of the obvious necessary condition for security
automatically implies robust security. For example, if attack costs
always exceed attack profits by 10%, then a sudden loss of .1% of
the overall stake cannot result in the ultimate loss of more than
1.1% of the overall stake. 

Our result suggests a “risk measure” that could be exposed to the
participants in a restaking protocol, namely the maximum value of the
buffer parameter γ that holds with respect to the current
restaking network. We also suggest easy-to-check
sufficient conditions that can proxy for this risk measure (<ref>). These conditions are similarly tight, as shown in <ref>, <ref>, and  <ref>.



 §.§ Our Results: Local Robust Security Guarantees




The results described in Section <ref> are “global” with
respect to the network structure, in three distinct senses: (i) the
overcollateralization condition is assumed to hold for every
subset B  V of validators and A  S of services that B is
capable of corrupting; (ii) the initial shock can affect any subset of
validators, subject to the assumed bound of ψ on the total
fraction of stake lost; and (iii) any subset of validators might lose
stake following the initial shock, subject to our upper bound of
(1+1γ)ψ on the total fraction of lost stake.






  
Local guarantees.
We next pursue more general “local” guarantees, which are
parameterized by a set C of services. (The global guarantees will
correspond to the special case in which C=S.)
For example, C might be a set of closely related services that share
a number of dedicated validators. The operators of such a set C
might object to both the assumptions and the conclusion of our global
guarantee:



  * How can we be sure that the overcollateralization factor holds for
  services and validators that we know nothing about?


  * And even if we could, 
how can we be sure that random validators that we have nothing
  to do with won't suddenly lose their stake (e.g., because they
  supported a malicious or buggy service), resulting in an initial
  shock the causes the loss of more than a ψ fraction of the
  overall stake?


  * And even if we could, how can we be sure that our validators
  won't be the ones that lose their stake following a shock that is
  purely the fault of other services and/or validators?



To address these concerns, we next consider a refined version of the
basic model, parameterized by a set C of services.  We denote
by Γ(C) the validators that are exclusive to C, meaning that
they contribute to no services outside C. (In the special case in
which C=S, Γ(C)=V and we recover the original model.)
Intuitively, the validators in Γ(C) are the ones that services
in C are “counting on”; other validators support (potentially
malicious or buggy) services outside C and, from C's perspective,
could disappear at any time.  We then restrict attention to initial
shocks in which at most a ψ fraction of the total stake
controlled by the validators of Γ(C) is lost. (The shock can affect
validators outside of Γ(C) arbitrarily.)  The goal is then to
identify overcollateralization conditions guaranteeing that, no matter
what the initial shock and subsequent attacks, the fraction of stake
ultimately lost by the validators in Γ(C) is bounded (by a
function of the shock size ψ and an overcollateralization
parameter γ).

Generalizing our global guarantees to local guarantees is not
straightforward and, as the next two examples show, requires
additional compromises. The first example shows that protection can be
guaranteed only against a subset of valid attacks, a natural and well
motivated subset that we call “stable attacks.” The second example
shows that, even when restricting attention to stable attacks,
overcollateralization is required not only for potential attacks, but
more generally for what we call “attack headers.”









  
Necessity of restricting to stable attacks.
In more detail, in the first example (depicted on the left of
<ref>), we
consider a restaking network with two services and three
validators. (Vertices in this figure correspond to validators, which
were previously represented as the right-hand side vertices of a
bipartite graph; each service is now identified with its neighborhood
of validators.) 
The service highlighted in green on the left has 100
times as much stake securing it as its profit from corruption, and
shares no validators with other services, so one might hope that it
would be protected from any shocks that affect only the other
validators.
However, if the validator
outlined in red disappears, the two validators outlined in yellow
can attack both services for
a profit of 102-101 > 0.
But this example is unsatisfying: the validator with stake 1 could
have attacked the service with profit from corruption 101 on its own
to yield a profit of 100. The addition of the validator with stake
100 added 100 to the cost of the attack, but only added 1 (from
the corruption of the service highlighted in green) to the profit. We
show in <ref> that the issue suggested by this
example is fundamental: without further restrictions on attacks that
rule out contrived examples such as this one, there does not exist a
local condition that guarantees local security.
In response, we confine
attention to sequences of what we call stable attacks in which
all of the attacking validators contribute positively to the profit of
the attack (as opposed to free riding on the profits attributable to
other attacking validators). The attack in the example is not
stable, as the validator with stake 100 was not a profitable
addition to the attack that could have been carried out by the
validator with stake 1.






  
Necessity of overcollateralizing attack headers.
Even if we restrict our attention to stable attacks, simple
overcollateralization is insufficient to guarantee local security. To
appreciate the issue, consider the restaking network shown on the
right in <ref>. In this example, there are three
services, each with a profit from corruption of 1, and three
services each with stake 1. Each validator is used to secure two
different services. The service outlined in green is
overcollateralized in that its profit from corruption is 1, but two
units of stake stake are securing it. Despite this, if the unrelated
validator outlined in red disappears, the two validators outlined in
yellow can attack all three services for a profit of 3-2 > 0.
Furthermore, if all validators are required to attack each service
(i.e., α_s=1), then this attack is stable, as the inclusion
of both validators in the attack is profitable. Thus, despite being
overcollateralized, a stable attack can be launched on the service
highlighted in green even if stake unrelated to the service disappears
in a shock.

We find that to guarantee robust security for a coalition of services
C using only “local” information, it is necessary to
overcollateralize not only pairs (A,B) where B is a set of
validators capable of attacking all services in A, but a more
general collection of pairs that we call attack
  headers. Informally, this amounts to requiring that there is some
“buffer” in stake for any potential attack on some services in C
even if we were to allow every validator that is also securing a
service outside of C to join the attack without considering their
profitability. Our main result here (<ref>) formally
provides a local condition guaranteeing that, whenever an initial
shock knocks out at most a ψ fraction of the stake that provides
security exclusively to C, the worst-case loss of such stake, after
an arbitrary sequence of stable attacks, is at most a
(1+1γ)ψ fraction. We show that our bounds are
tight and indeed require overcollateralization of all attack headers
(<ref>, <ref>, and
<ref>), and again provide easily computable
sufficient (and similarly tight) conditions that can proxy for the
overcollateralization condition (<ref>).  Our local condition
generalizes the global overcollateralization condition, with the latter
corresponding to the special case of the former in which C=S.




 §.§ Our Results: Cascading Attacks


An attack on a restaking network results in a loss of stake (of the
attacking validators), and this may introduce new opportunities for
other sets of validators to carry out profitable attacks. That is, an
initial shock may set off an entire cascade of attacks. (All of
the bounds on stake loss described in Sections <ref>
and <ref> hold for cascades of attacks of arbitrary length.)
Our final result concerns the maximum-possible length of such a
cascade, and shows that this quantity is also governed in part by the
overcollateralization factor γ.

Precisely, we define the reference depth of a cascade of attacks
as a measure of the “long-range dependence” between different
attacks in an attack sequence. For example, if each attack in the
sequence is directly enabled by the loss of the validators slashed in
the previous attack, then the reference depth of the sequence
is 1. Our main result here (<ref>) bounds the
maximum-possible attack length as a function of the reference
depth k, the shock size ψ, the overcollateralization
factor γ, and the minimum stake ϵ held by a validator:
k ( 1 + log_1 + γ(ψ· [total
    stake]/ϵγ)).
For example, in the case of constant reference depth and equal
validator stake amounts, the worst-case attack length is logarithmic
in the number of validators with overcollateralization and linear
without it.



 §.§ Related Work


Our focus on the risks of cascading failures following a small shock
echoes some of themes in the literature on systemic risk in financial
networks. For example, Eisenberg and Noe <cit.>
that study the existence and structure of inter-firm payments in a
financial network following a default.  This work is extended by
Glasserman and Young <cit.>, who study how
“connectedness,” meaning the fraction of liabilities that a firm
externally owes, affects contagion risk.  Acemoglu et
al. <cit.> build further on this work and study
network “stability,” meaning the propensity for shocks to propagate;
they show that connectivity initially improves stability, but then at
a phase transition, denser connectivity leads to increased shock
propagation.  In a subsequent paper, Acemoglu et
al. <cit.> unify a number of the preceding
results. 

A separate line of work, beginning with Chen et
al. <cit.>, aims to axiomatically characterize
systemic risk measures.  The model in <cit.> can
capture, in particular, a contagion model characterized by a matrix of
profits and losses over different firms and outcomes. The results
in <cit.> characterize the global measures of risk
(operating on the matrix) that satisfies certain sets of desirable
axioms.  This is work is expanded upon in Kromer et
al. <cit.>, where the authors consider general
outcome measure spaces, as well as in Feinstein et
al. <cit.>, where the authors consider
set-valued risk measures.  Battison et al.<cit.>
study systemic risk measurement when a regulator has limited
information about contracts made between financial network
participants (e.g., the fraction of the face value that can likely be
recovered if a counterparty defaults), and show how small errors in
knowledge can lead to large errors in systemic risk measurement.

Our work also shares some conceptual similarity with the well-known
work of Diamond and Dybvig <cit.> on bank runs and
of Brunnermeier et al. <cit.> on the risks of re-hypothecation.

The model in the present work differs substantially from those
considered in the aforementioned papers, in large part because of the
idiosyncrasies of restaking networks, including their combinatorial
and bipartite nature and their susceptibility to economically
motivated attacks.

Restaking has also, to a limited extent, been studied in its own
right. The EigenLayer team introduces restaking in <cit.>. The
framing of an economically motivated attacker that trades off stake
loss with profits from corruption appears in <cit.>, and is used
also by Deb et al. <cit.>.  Chitra and
Neuder <cit.> discuss restaking risk from a validator
perspective, comparing restaking with investments in other financial
instruments (e.g. bonds).  Alexander <cit.>
considers the interplay between existing leveraging schemes and liquid
restaking tokens, which can amplify large-scale credit risk.



§ MODEL




  
Validators and Services. We consider a setting in which there is a set V of validators and a set S of services. Each service s ∈ S has some profit from corruption π_s, and each validator v ∈ V has some stake σ_v. We also associate with each service s a parameter α_s that denotes the fraction of stake required to corrupt/launch an attack on s. We call a bipartite graph G = (S, V, E, π, σ, α) a restaking graph; an edge is drawn between a validator v ∈ V and a service s ∈ S if v is restaking for s. For a given set of vertices A in a graph G, we use the notation N_G(A) to denote the neighbors of A. 



  
Attack Dynamics. For simplicity, we assume that validators lose their full stake σ_v if they launch an attack on a service. As such, for a given collection of services A ⊆ S, and a given collection of validators B ⊆ V restaking for those services, we say that (A, B) is an attacking coalition for a restaking graph G if the validators in B possess enough stake to corrupt the services A:

    ∑_v ∈ B ∩ N_G(s)σ_v_Total stake in s owned by validators B≥α_s ·∑_v ∈ N_G(s)σ_v_Total amount restaked in s      ∀ s ∈ A

We further say that (A, B) is a valid attack if it is an attacking coalition that has an incentive to launch an attack:

    ∑_s ∈ Aπ_s_Total profit from corrupting A > ∑_v ∈ Bσ_v_Total stake owned by validators B

If a valid attack (A, B) is carried out, we denote by G ↘ B the induced subgraph G⟨S, V ∖ B|. The graph G ↘ B denotes the state of the restaking graph after the attack is carried out. If no valid attacks exist on the graph, we call it secure. To simplify notation, for any subset of validators B ⊆ V, we will use the shorthand σ_B to denote ∑_v ∈ Bσ_v. Similarly, for any A ⊆ S, we will use π_A as shorthand for ∑_s ∈ Aπ_s.



  
EigenLayer sufficient conditions. We note in passing that, in their whitepaper <cit.>, EigenLayer proposes some efficiently verifiable sufficient conditions for security that they check to ensure that an attack does not exist.
[EigenLayer Sufficient Conditions, from Appendix B.1 of the EigenLayer Whitepaper <cit.>]
A restaking graph G is secure if for each validator v ∈ V,

    ∑_s ∈ N_Gvσ_v /σ_N_Gs·π_s/α_s≤σ_v



It is shown in Appendix B.1 of the whitepaper that if <ref> holds for G, then the graph is secure (i.e. no valid attacks (A,B) satisfying <ref> exist).




  
Cascading attacks. Our goal is to understand when a small shock can result in the loss of a large fraction of the overall stake. Small shocks can turn into large shocks by means of a cascading attack. Formally, we say that a disjoint sequence (A_1, B_1), …, (A_T, B_T) ∈ 2^S × 2^V is a valid cascade of attacks on a restaking graph G = (S, V, E, π, σ, α) if for each t ∈ [T], (A_t, B_t) is a valid attack on G ↘⋃_i = 1^t-1  B_i. We denote by C(G) the set of all such sequences of valid cascading attacks. 



  
Worst-case stake loss. We now define a metric that measures the total potential loss of stake due to a sequence of cascading attacks. In our model, we first suppose that an initial small shock decreases the amount of stake. Formally, we define, for a given restaking graph G = (S, V, E, π, σ, α),


    𝔻_ψ(G) := D ⊆ V |σ_D/σ_V≤ψ

to be the set of validator coalitions that constitute at most a ψ-fraction of all stake. Given some D ∈𝔻_ψ(G), we use the notation G ↘ D := G⟨S, V ∖ D| to denote the induced subgraph of the restaking graph when we delete the validators D.

We now define 

    R_ψ(G) := ψ_Initial shock + max_D ∈𝔻_ψ(G)max_(A_1, B_1), …, (A_T, B_T) ∈ C(G↘ D)σ_⋃_t = 1^T B_t/σ_V_Stake lost from cascading attacks


This quantity represents the worst-case total fraction of stake lost due to an initial ψ-fraction of the stake disappearing. By construction, ψ≤ R_ψ(G) ≤ 1.



§ OVERCOLLATERALIZATION PROVIDES ROBUST SECURITY


In this section, we show that “scaling up” the definition of security automatically results in robust security, meaning bounded losses from cascading attacks that follow an initial shock. We first show that, without loss of generality, it suffices to consider single valid attacks (A, B) ∈ C(G↘ D) instead of more general cascading attacks.


    Let G = (S, V, E, π, σ, α) be an arbitrary restaking graph, and further suppose that (A, B) is an attacking coalition on G ↘ D, where D ⊆ V. Then, (A, B ∪ D) is an attacking coalition on G. 


    Because (A, B) is an attacking coalition on G ↘ D, we must have by <ref> that
    
    σ_B ∩ N_Gs≥α_s ·σ_N_G s∖ D      ∀ s ∈ A

    It follows that for any s ∈ A,
    
    σ_B ∪ D∩ N_Gs   = σ_B ∩ N_Gs + σ_D ∩ N_Gs
       ≥α_s ·σ_ N_Gs∖ D  + σ_D ∩ N_Gs
       ≥α_s ·σ_N_Gs

and the desired result follows.


Let G = (S, V, E, π, σ, α) be an arbitrary restaking graph, and further suppose that (A_1, B_1), …, (A_T, B_T) ∈ C(G) is a valid sequence of cascading attacks on G. Then, ⋃_t = 1^T A_t, ⋃_t = 1^T B_t is also a valid attack on G. 


    By repeatedly applying Lemma <ref>, we find that for each t ∈ [T], A_t, ⋃_i = 1^t B_t is an attacking coalition on G. It follows by inspection of <ref> that we must therefore have that ⋃_t A_t, ⋃_t B_t is an attacking coalition on G. To finish the result, it suffices to show that <ref> holds for this attacking coalition on the original graph G. This follows from the disjointness of the A_t's and of the B_t's:
    
    π_⋃_t A_t = ∑_t = 1π_A_t > ∑_t = 1^T σ_B_t  =  σ_⋃_t B_t

    where in the inner inequality we use that for each t ∈ [T], π_A_t > σ_B_t by <ref>, as (A_t, B_t) is a valid attack on G ↘⋃_i=1^t-1 B_i. It follows that ⋃_t A_t, ⋃_t B_t is a valid attack on G.




  
Adding Multiplicative Slack. Our condition is given by adding multiplicative slack to <ref>. Formally, we say that a restaking graph G is secure with γ-slack if for all attacking coalitions (A, B) on G,

    (1 + γ) π_A_Total profit from corrupting A≤σ_B_Total stake owned by validators B





    Suppose that a restaking graph G = (S, V, E, π, σ, α) is secure with γ-slack for some γ > 0. Then, for any ψ > 0, R_ψ(G) < 1 + 1/γψ.



Take any ψ > 0 and any D ∈𝔻_ψ(G) for some restaking graph G where (<ref>) holds. Let (A_1, B_1) …, (A_T, B_T) ∈ C(G↘ D) be arbitrary. Applying <ref>, we must have that (⋃_t A_t, ⋃_t B_t) ∈ C(G↘ D) as well. Defining A := ⋃_t A_t and B := ⋃_t B_t, we must therefore have that (A, B) is an attacking coalition on G ↘ D, and furthermore that

    π_A    > σ_B

 By <ref>, we must also have that (A, B ∪ D) is an attacking coalition on the original graph G. It then follows that as G is secure with γ-slack, <ref> must hold on (A, B ∪ D), whence

    (1 + γ)  π_A ≤σ_B ∪ D = σ_B + σ_D

Putting this together with <ref>, we find that

    (1 + γ) σ_B < (1 + γ) π_A ≤σ_B + σ_D  ≤σ_B + ψ·σ_V

It follows that 

    γ·σ_B < ψ·σ_V    σ_B /σ_V <  ψ/γ
       ψ +  σ_B /σ_V < 1 + 1/γψ

As we took A_1, …, A_T to be arbitrary, we find that R_ψ(G) < 1 + 1/γψ, as desired.


The EigenLayer sufficient conditions (<ref>) can be similarly “scaled up” to yield efficiently checkable sufficient conditions for security with γ-slack (and hence, by <ref>, robustness to cascading attacks).

Let G be a restaking graph such that, for all validators v ∈ V,
    
    ∑_s ∈ N_G(v)σ_v /σ_N_Gs·(1 + γ) π_s/α_s≤σ_v

Then, R_ψ(G) < 1 + 1/γψ.


    This follows from <ref> and <ref>. Noting that the γ-slack condition holds precisely iff no valid attacks exist when profits from corruption π_s are inflated by a multiplicative factor of (1+γ), it suffices to apply EigenLayer's sufficient conditions from <ref> with modified profits from corruption (1+γ)π_s.

Note that, given a restaking network, it is straightforward to compute the minimum value of γ such that the condition in (<ref>) holds. 
This value can then be interpreted as an easily computed “risk measure” of such a network.



§ LOWER BOUNDS FOR GLOBAL SECURITY


In this section, we show that the upper bounds from the previous section are tight. We first show that if there is no multiplicative slack (i.e. γ = 0), then very small shocks can cause all stake to be lost in the worst case. This holds even under the EigenLayer conditions (<ref>)[The graph exhibited in the proof of <ref> has π_x/σ_a →∞ as ϵ→ 0. It is possible to construct a counterexample with similar properties while maintaining that π_s/σ_v is greater than some universal constant for any s ∈ S and v ∈ V. This is done in <ref>.].

    For any 0 < ϵ < 1, there exists a restaking graph G that is secure and meets the EigenLayer condition (<ref>), but has R_ψ(G) = 1 for all ψ≥ϵ. 


    We construct a restaking graph G = (S, V, E, π, σ, α) with one service S = x and two validators V = a,b, where an edge exists between each validator and the service (i.e. E := (x, a), (x, b)). We then let σ_a := ϵ, σ_b := 1 - ϵ, π_x := 1, and α_x := 1. Without loss of generality, we may assume ψ < 1 as R_1(G) = 1 for any restaking graph G. This graph satisfies (<ref>) as 
    
    σ_a/σ_a + σ_b·π_x/α_x = σ_a
    σ_b/σ_a + σ_b·π_x/α_x = σ_b

    whence the graph is also secure by <ref>. We now consider an initial shock D = a. As σ_a/σ_V = ϵ≤ψ, it follows that D ∈ D_ψ(G). The pair (x, b) is a valid attack on G ↘ D, since b = N_G ↘ Dx whence it is an attacking coalition, and σ_b < π_x. It follows that R_ψ(G) ≥σ_a + σ_b/σ_V = 1 as desired.


Next, we show that the bound we give in <ref> (indeed, more strongly, the condition given in <ref>) is tight for all ψ, γ > 0. 


    For any ψ, γ, ϵ > 0 such that
    
    0 ≤1 + 1/γψ - ϵ≤ 1,

    there exists a restaking graph G that satisfies the condition (<ref>) from <ref> but has R_ψ(G) ≥1 + 1/γψ - ϵ.


    We construct a restaking graph G = (S, V, E, π, σ, α) with three validators V = a, b, c and one service S = x, where an edge exists between each of the validators a and b and the service x (i.e. the edge set E := (x,a), (x,b)). Without loss of generality, suppose that ϵ≤ψ/γ. Let σ_a > 0 be any positive constant. We define
    
    σ_b    := σ_a 1/γ- ϵ/ψ
    σ_c    := σ_a1 - ψ + ϵ/ψ - 1/γ
    π_x    := 1 + 1/γψ - ϵ/1 + γ·σ_V
    α_s    := 1

    Notice first that σ_b ≥ 0 as we have taken ϵ≤ψ/γ. Next, observe that
    
    σ_c ≥ 0    1 - ψ + ϵ/ψ≥1/γ
        1 ≥1 + 1/γψ - ϵ

    whence σ_c ≥ 0 by <ref>. Finally, we must also have that π_x ≥ 0 by <ref> as well. Next, notice by construction that
    
    σ_V    = σ_a + σ_b + σ_c = σ_a1 + 1/γ- ϵ/ψ+ 1 - ψ + ϵ/ψ - 1/γ = σ_a/ψ

    This graph meets condition (<ref>) from <ref>, as 
    
    ∑_s ∈ N_Gaσ_a/σ_N_Gs (1 + γ)π_s    = σ_a/σ_a + σ_b⟨1 + 1/γψ - ϵ|σ_V
       = σ_a/σ_a1 + 1/γ- ϵ/ψ⟨1 + 1/γψ - ϵ|σ_a/ψ
       = σ_a

    A similar argument shows that 
    
    ∑_s ∈ N_Gbσ_b/σ_N_Gs (1 + γ)π_s = σ_b/σ_a⟨1 + 1/γψ - ϵ|⟨1 + 1/γψ - ϵ|σ_a = σ_b

    Finally, as the validator c has no neighbors, it also satisfies the condition, whence the graph indeed satisfies (<ref>). We now consider an initial shock D := a to the graph. Because
    
    σ_a/σ_V = σ_a/σ_a/ψ = ψ,

    the shock D ∈𝔻_ψ constitutes a ψ fraction of the total stake as desired. The attack (x, b) is a valid attack on G ↘ D. To see this, notice first that (x, b) is an attacking coalition on G ↘ D as b = N_G ↘ Dx. Furthermore, b is incentivized to attack since
    
    π_x - σ_b    = 1 + 1/γψ - ϵ/1 + γ·σ_V - σ_b
       = ⟨1 + 1/γ- ϵ/ψ/1 + γ - 1/γ- ϵ/ψ|σ_a
       = ⟨1 + 1/γψ + γϵ/(1 + γ) ψ - 1/γ|σ_a
       = γϵσ_a/(1 + γ) ψ > 0

    whence <ref> is satisfied for the pair (x, b). It follows that
    
    R_ψ(G) ≥σ_a + σ_b/σ_V = 1 + 1/γ- ϵ/ψσ_a/σ_a/ψ = 1 + 1/γψ - ϵ

    as desired.




§ LOCAL SECURITY



The bound in <ref> on the worst-possible stake loss from cascading attacks is reassuring from a global perspective, but less so from the perspective of one or a small number of services who would like an assurance that they will not be among those affected by such attacks. 
Beginning with this section, 
we focus on a specific coalition of services C ⊆ S that seeks to insulate their shared security Γ(C) against shocks and resulting cascading attacks that may come about due to the decisions of other services and validators. Formally, we denote by 

    Γ(C) := v ∈ V | N_Gv⊆ C

the set of validators that exclusively provide security for services in C. 



  
Worst-case stake loss (local version).
As before, we first suppose that an initial shock affects the restaking graph. Whereas previously, we considered shocks for which the total stake in the shock was bounded, we now consider shocks for which the total stake that impacts the exclusive security of some coalition of services C (i.e., stake that secures services from C and only services from C) is bounded. Formally, for any coalition of services C ⊆ S within some restaking graph G, we let


    𝔻_ψ(C, G) := D ⊆ V |σ_D ∩Γ(C)/σ_Γ(C)≤ψ

denote the set of all validator coalitions that provide at most ψ stake to the aggregate security of the coalition of services C. Notice that shocks D ∈𝔻_ψ(C, G) may have much more total stake σ_D than a ψ fraction of the graph. We are instead only guaranteed that the impact of the shock on stake that is being used exclusively for members in C is small. We are now interested in the potential cascading losses that can affect the stake that is exclusively utilized by the coalition C after a shock occurs that destroys at most ψ stake from the aggregate security of C. Formally, we study the quantity


    R_ψ(C, G) := ψ_Initial Shock + max_D ∈𝔻_ψ(C, G)max_(A_1, B_1), …, (A_T, B_T) ∈ C(G ↘ D)σ_⋃_t = 1^T B_t ∩Γ(C)/σ_Γ(C)_Stake lost from cascading attacks




  
Local security conditions. We seek sufficient conditions on a restaking graph G that guarantee a nontrivial upper bound on R_ψ(C, G). 
Ideally, the sufficient condition

would depend only the neighborhood/choices of the coalition C


to provide a guarantee that holds regardless of the choices made by other validators and services (i.e., the services of C can attempt to “control their own destiny” by ensuring that the locally defined sufficient condition holds). Formally, given a restaking graph G = (S, V, E, π, σ, α) and a coalition of services C ⊆ S, we call a restaking graph G' = (S', V', E', π', σ', α') a C-local variant of G if C cannot distinguish G' from G on the basis of local information: C ⊆ S', N_GC = N_G'C, and

    (π_s, α_s)    = (π'_s, α'_s)       ∀ s ∈ C
    σ_v, N_Gv   = σ'_v, N_G'v      ∀ v ∈ N_GC

We then define a local security condition f: (C, G) ↦0,1 to be a Boolean function that takes as input a restaking graph G and a coalition of services C ⊆ S such that f(C, G) must be equal to f(C, G') for all C-local variants G'. The intuition behind this definition is that the condition should only depend on service-level information (e.g. profits from corruption, security thresholds) for services in the coalition, and validator-level information for validators in N_GC. 



  
Local security impossibility. Unfortunately, without further restrictions on the attacks under consideration (like those defined later in this section), it is impossible to construct any nontrivial local security condition that yields any nontrivial upper bound on R_ψ(C, G).


    For any local security condition f, any secure restaking graph G and coalition of services C ⊆ S such that f(C, G) = 1, there exists a secure C-local variant G' of G such that R_0(C, G') = 1. 



    Take any f, C, and secure G such that f(C, G) = 1. Define 
    
    Δ := σ_N_GC - π_C

    to be the total overcollateralization of C in aggregate. Next, we define G' to be an augmented version of G, where we add a new service s^* that has a profit from corruption π_s^* = Δ + 2ϵ where ϵ > 0. We further add 2 validators a and b to the graph who are adjacent only to s^*. We let σ_a := Δ + ϵ and σ_b := ϵ. As σ_a + σ_b ≥π_s^*, the graph G' must be secure as G was secure. Next, notice that as the validator a is not path-connected to C, G' must be a C-local variant of G. However, by construction, the attack C ∪s^*, N_GC ∪b is valid on the graph G' ↘a. It follows that R_0(C, G') = 1.




  
Stable attacks. While the above impossibility appears to be quite strong, it is somewhat contrived. At the heart of the impossibility is that under the definition of a valid attack (i.e. <ref>), not every validator must be productive in carrying out the attack. There may be a subset of validators in the attack that can yield more net profit than the full coalition. In what follows, we show that if we assume that malicious validator coalitions will choose to add others to their ranks only if it is profitable for them in net to do so, then a local security condition with guarantees similar to those in <ref> does indeed exist. Formally, for A ⊆ S and B ⊆ V, we say that an attack (A, B) is stable if it is valid (i.e. <ref> hold), and for all A' ⊆ A and B' ⊆ B such that (A', B') is valid, 

    σ_B ∖ B' < π_A ∖ A'

Intuitively, if (<ref>) did not hold, then the validators of B' would be better off ditching those in B ∖ B' and attacking only the services in A'.

We say that a disjoint sequence (A_1, B_1), …, (A_T, B_T) ∈ 2^S × 2^V is a cascade of stable attacks on a restaking graph G = (S, V, E, π, σ, α)  if for each t ∈ [T], (A_t, B_t) is a stable attack on G ↘⋃_i = 1^t-1 B_i. We denote by S(G) the set of all such sequences of cascading stable attacks. In light of Theorem <ref>, we redefine our notion of worst-case stake loss using stable attacks:

    R_ψ(C, G) := ψ + max_D ∈𝔻_ψ(C, G)max_(A_1, B_1), …, (A_T, B_T) ∈ S(G ↘ D)σ_⋃_t B_t ∩Γ(C)/σ_Γ(C)

Unlike valid attacks, unions of sequences of stable cascading attacks need not be stable (which will complicate the proof of <ref> in the next section).


    There exists a restaking graph G and a sequence (A_1, B_1), (A_2, B_2) ∈ S(G) such that A_1 ∪ A_2, B_1 ∪ B_2∉ S(G).



        Consider restaking graph where there are two services S = x, y and two validators V = a,b where both validators are restaking in both services. Furthermore, π_x = π_y = 2, α_x = α_y = 1/2, and σ_a = σ_b = 1. In this case, the sequence x, a, y, b∈ S(G). However, x, y, a, b∉ S(G) because the attack x, y, a is a valid attack.




§ A LOCAL SECURITY CONDITION FOR STABLE ATTACKS


In this section, we give a family of local security conditions that yield guarantees on the local worst-case stake loss R_ψ(C, G) that resemble our result from Theorem <ref> for global security. In Theorems <ref> and <ref>, we showed that security with γ-slack was necessary and sufficient in order to obtain a 1 + 1/γψ upper-bound on R_ψ(G). In other words, it was both necessary and sufficient to ensure that all attacking coalitions (A, B) were overcollateralized multiplicatively by a factor of (1 + γ). Our condition for local security is similar: we must ensure that certain attack headers (X, Y) are overcollateralized multiplicatively by a factor of (1 + γ).



  
Attack header. Formally, we say that (X, Y) is an attack header, for X ⊆ S and Y ⊆Γ(X), if there exists a set of validators B ⊆ V satisfying

    B ∩Γ(X) = ∅

such that (X, B ∪ Y) is an attacking coalition. In other words, (X, Y) is an attack header if Y can be appended to a collection of validators B that may be slashed without attacking services in X to form an attacking coalition that attacks the services X. An attacking coalition is a special case of an attack header (in which B can be taken as ∅).


    Let G = (S, V, E, π, σ, α) be a restaking graph and C ⊆ S be a coalition of services. If, for all attack headers (X, Y) where X ⊆ C, 
    
    (1 + γ) π_X ≤σ_Y

    then R_ψ(C, G) < (1+1/γ)ψ. Furthermore, the Boolean function that checks whether <ref> holds for all attack headers is a local security condition.



    Let D ∈𝔻_ψ(C, G) and (A_1, B_1), …, (A_T, B_T) ∈ S(G ↘ D) be arbitrary. For each t ∈ [T], define
    
    L_t := B_t ∩Γ(C)

    to be the set of all validators exclusively securing C that were lost in the tth attack. Next, define
    
    A'_t := s ∈ A_t |σ_B_t ∖ L_t≥α_s ·σ_N_G s∖⋃_i = 1^t - 1 B_i ∪ D

    to be the maximal set of services such that (A'_t, B_t ∖ L_t) is an attacking coalition on G ↘⋃_i = 1^t-1 B_i ∪ D. Notice also that because (A_t, B_t) is a stable attack on G ↘⋃_i = 1^t-1 B_i ∪ D, we must have that A_t ∖ A'_t is nonempty, and furthermore must be a subset of C. 
    
     
        
    σ_⋃_t B_t ∩Γ(C) < π_⋃_t A_t ∖ A'_t

    
    
    From stability, we have that
    
    σ_L_t = σ_B_t ∖B_t ∖ L_t  < π_A_t ∖ A'_t

    To see why this holds, notice that there are two cases. If the attacking coalition (A'_t, B_t ∖ L_t) is a valid attack, then the inequality follows directly from the stability definition. If instead (A'_t, B_t ∖ L_t) is not a valid attack despite being an attacking coalition, the inequality must still hold because the original attack (A_t,B_t) is valid and therefore satisfies <ref>. Iterating over t, we find that
    
    σ_⋃_t B_t ∩Γ(C)  = ∑_t = 1^T σ_L_t < ∑_t = 1^T π_A_t ∖ A'_t = π_⋃_t A_t ∖ A'_t

    

     
        ⋃_t A_t ∖ A'_t, ⋃_t B_t ∪ D∩Γ(C) is an attack header on G, and ⋃_t A_t ∖ A'_t⊆ C.
    
    
    Because each A_t ∖ A'_t ⊆ C, we must also have that ⋃_t A_t ∖ A'_t ⊆ C as well. Next, by applying <ref> noting the disjointness of the (A_t, B_t), we have that
    
    ⋃_t A_t, ⋃_t B_t = ⋃_t A_t ∖ A'_t∪⋃_t A'_t,   ⋃_t B_t

    must be an attacking coalition on G ↘ D. In particular, we must have that ⋃_t A_t ∖ A'_t, ⋃_t B_t
    is an attacking coalition on G ↘ D, whence by Lemma <ref> we have that ⋃_t A_t ∖ A'_t, ⋃_t B_t ∪ D
    is an attacking coalition on G. Rewriting the above as 
    
    ⋃_t A_t ∖ A'_t,   ⟨⋃_t B_t ∪ D∩Γ(C)|∪⟨⋃_t B_t ∪ D∖Γ(C)|

    and noting by construction that
    
    ⟨⋃_t B_t ∪ D∖Γ(C)|∩Γ(C) = ∅,

    we find that ⋃_t A_t ∖ A'_t, ⋃_t B_t ∪ D∩Γ(C) is an attack header on G as desired.
    
    Putting these claims together yields the desired result. By Claim <ref> and <ref>, we find that
    
    (1+γ) π_⋃_t A_t ∖ A'_t≤σ_⋃_t B_t ∪ D∩Γ(C)≤σ_⋃_t B_t ∩Γ(C) + ψ·σ_Γ(C)

    Adding in Claim <ref>, we find that
    
    (1+γ)σ_⋃_t B_t ∩Γ(C)   < σ_⋃_t B_t ∩Γ(C) + ψ·σ_Γ(C)
       σ_⋃_t B_t ∩Γ(C)/σ_Γ(C) < ψ/γ
        R_ψ(C, G) < 1 + 1/γψ

    as desired. To see that the Boolean function that checks whether <ref> holds for all attack headers is a local security condition, observe that it suffices to check that for all X ⊆ C and Y ⊆ N_GC ∩Γ(C) = Γ(C) such that X, Y ∪ N_GC ∖Γ(C) is an attacking coalition, <ref> holds. Thus, for any restaking graph G and C-local variant G', this function will evaluate to the same output.


The condition in (<ref>) can be checked via enumeration, although the time required to do so grows exponentially in the number of services in C and the number of validators that contribute security exclusively to services in C. 
In the event that this is a prohibitive amount of computation,
the same guarantee holds under a stronger, easily checked local analog of the EigenLayer sufficient condition (<ref>) which, in effect, treats as malicious all validators that contribute security to any services outside of C.


    For any restaking graph G = (S,V,E,π,σ,α) and coalition of services C ⊆ S, satisfaction of the local condition
    
    ∑_s ∈ N_Gvσ_v/σ_N_Gs·(1 + γ) π_s/α'_s≤σ_v

    for all v ∈ N_GC with N_Gv⊆ C guarantees that R_ψ(C, G) < 1 + 1/γψ, where for each s ∈ C,
    
    α'_s := α_s - σ_N_Gs∖Γ(C)/σ_N_Gs



    This follows from <ref> and <ref>. Notice that to guarantee that the condition (<ref>) from <ref> holds for all attack headers (X,Y), it suffices to guarantee that no valid attacks exist, when (i) all profits from corruption for services in C are inflated by a multiplicative factor of (1+γ), and (ii) the fraction of stake required to corrupt a given service is offset by the fraction of stake in that service that is also restaking for services that do not belong to C. As <ref> provides a sufficient condition to guarantee the existence of no valid attacks, <ref> will guarantee that all attack headers are overcollateralized by a multiplicative factor of (1 + γ).




§ LOWER BOUNDS FOR LOCAL SECURITY


We next show senses in which <ref> (and more strongly, <ref>) is tight.


For any ψ, γ, ϵ > 0 such that
    
    0 ≤1 + 1/γψ - ϵ≤ 1,

    there exists a restaking graph G that satisfies the condition (<ref>) from <ref> but has R_ψ(C, G) ≥1 + 1/γψ - ϵ.


    Notice that if we take C = S, the condition (<ref>) from <ref> is identical to the condition (<ref>) from <ref>. Furthermore, R_ψ(S, G) is the same as R_ψ(G) except for the fact that R_ψ(S, G) considers only stable attacks. Repeating the argument from <ref>, and noting that the attack given in the proof of that result is stable, we obtain the desired result. 



    For any γ > 0, there exists a graph restaking graph G = (S, V, E, π, σ, α) that satisfies (<ref>) from <ref>, but there exists a C ⊆ S such that R_0(C, G) = 1.


    Let there be three services S = x, y , z, three validators V = a, b, c, and edges as follows:
    
    E := (x, a), (x, b), (y, b), (y, c), (z, c), (z, a).

    Next, let α_s = 1 for all s ∈ S, and let π_x = π_y = π_z =: π > 0 be an arbitrary positive constant. Next pick σ_a such that
    
    σ_a < 2π,

    let σ_b := 2(1 + γ)π, and let σ_c := 2(1 + γ)π. This graph satisfies (<ref>) from <ref> as
    
    σ_a/σ_a + σ_b· (1+γ) π + σ_a/σ_a + σ_c· (1 + γ) π < (1 + γ)π1/σ_b + 1/σ_cσ_a = σ_a
    σ_b/σ_a + σ_b· (1+γ) π + σ_b/σ_b + σ_c· (1 + γ) π < 3/2 (1 + γ)π < σ_b
    σ_c/σ_a + σ_c· (1+γ) π + σ_c/σ_b + σ_c· (1 + γ) π < 3/2 (1 + γ)π < σ_c

    Next, let C := x, z, and observe that the shock D = b, c is an element of 𝔻_0(C, G). However, because σ_a < π_x + π_z = 2π, we must have that (x, z, a) is a stable attack on G ↘ D. As a = Γ(C), it follows that R_0(C, G) = 1.




§ LONG CASCADES




  
Cascade Structure. Although <ref> shows that all long cascades can be made into a short, one-step attack, it is arguably more dangerous if large attacks can be made through long cascades of small attacks that each require less coordination. In this section, we show how adding γ-slack also enables us to upper-bound the length of a cascade in the worst case. Our results depend on what we call the reference depth of a cascading sequence of attacks. Formally, for a given restaking graph G and coalition of validators B_0, we say that a valid cascade of attacks (A_1, B_1), …, (A_T, B_T) on C(G↘B_0) has reference depth k if


    k = maxi ∈ [T] |∃  t ∈ [T]  s.t.  N_G(A_t) ∩ B_t - i∅


In other words, a cascading sequence has reference depth k if the services attacked in a given time step are not affected by validators that were slashed more than k steps previous. 

 
    Suppose that a restaking graph G = (S, V, E, π, σ, α) is secure with γ-slack for some γ > 0. Let ϵ = min_v ∈ Vσ_v denote the minimum stake held by a validator. Then, for any ψ > 0, B_0 ∈𝔻_ψ(G), and (A_1, B_1), …, (A_T, B_T) ∈ C(G ↘B_0) with reference depth k, 
    
    T < k1 + log_1 + γψ·σ_V/ϵγ




    Because (A_1, B_1), …, (A_T, B_T) ∈ C(G ↘B_0) has reference depth k, it follows that if we define
    
    A'_i    := ⋃_t = k(i-1) + 1^ki A_t
    
            B'_i    := ⋃_t = k(i-1) + 1^ki B_t
    
            B'_0    := B_0

    for i ∈1, …, T/k, we must have that for all j < i - 1,
    
    N_GA'_i ∩ B'_j = ∅

    Furthermore, by <ref>, we must have that the sequence (A'_i, B'_i) ∈ C(G↘ B_0). Applying <ref> again, we further have that for every i, ⋃_j = i + 1^T/k A'_i, ⋃_j = i + 1^T/k B'_i is a valid attack on G ↘⋃_j = 0^i B'_i. It follows by <ref> that for any i ∈0, …,T/k - 1,
    
    π_⋃_j = i + 1^T/k A'_j > σ_⋃_j = i + 1^T/k B'_j = ∑_j = i + 1^T/kσ_B'_j

    Next, noting that 
    
    N_G ⋃_j = i+1^T/k A_i' ∩⋃_j = 1^i - 1 B_i' = ∅

    by <ref>, it follows that indeed ⋃_j = i + 1^T/k A'_i, ⋃_j = i + 1^T/k B'_i is also a valid attack on G ↘ B_i'. Applying <ref>, we then find that ⋃_j = i + 1^T/k A'_i, B'_i ∪⋃_j = i + 1^T/k B'_i is an attacking coalition on the original graph G. Because G is secure with γ-slack, <ref> must now hold on this pair, whence for all i ∈0, …, T/k - 1,
    
    (1+γ) π_⋃_j = i + 1^T/k A'_j≤σ_⋃_j = i^T/k B'_j = σ_B'_i + ∑_j = i+1^T/kσ_B'_j

    Putting this together with <ref>, we have that for all i ∈0, …, T/k - 1,
    
    (1 + γ) ∑_j = i+1^T/kσ_B'_j <  σ_B'_i + ∑_j = i+1^T/kσ_B'_j
       σ_B'_i  > γ∑_j = i+1^T/kσ_B'_j

    It can be shown inductively from <ref> that the sequence σ_B'_i > X_i, where we define the sequence X_i by
    
    X_T/k   := ϵ
    
            X_i    := γ∑_j = i+1^T/k X_j        i ∈0, …, T/k - 1

    Solving, we find that
    
    X_0 = ϵγ (1 + γ)^T/k - 1

    It then follows as desired that
    
    ψ·σ_V = σ_B_0 > ϵγ(1+γ)^T/k - 1 T < k1 + log_1 + γψ·σ_V/ϵγ





 §.§ Acknowledgments


We thank Tarun Chitra, Soubhik Deb, Sreeram Kannan, Mike Neuder, and
Mallesh Pai for comments on earlier drafts of this paper.  We thank
Soubhik and Sreeram in particular for emphasizing the importance of
local guarantees.

This research was supported in part by
NSF awards CCF-2006737 and CNS-2212745, and research awards from the
Briger Family Digital Finance Lab and the Center for Digital Finance
and Technologies.

plain






§ LOWER BOUNDS UNDER A BOUNDED PROFIT-STAKE RATIO


The example exhibited in <ref> requires that for the whole graph to be attacked due to an arbitrarily small shock R_ϵ(G) = 1, there is a service whose profit from corruption is arbitrarily larger than the stake of some validator (i.e. π_s/σ_v for a service s and validator v). In this section, we show that equally strong lower bounds for both global and local security exist even when all of the services are “small”, in the sense that an absolute constant bounds the ratio of the profit from corruption of any service and the stake of any validator. 


    For any 0 < ϵ < 1, there exists a secure restaking graph G = (S, V, E, π, σ, α) that meets the EigenLayer condition (<ref>),
    
    max_(s, v) ∈ S × Vπ_s/σ_v = 2,

    but there exist proper subsets C ⊂ S such that R_0(C, G) = 1. Furthermore, for any ψ≥ϵ, R_ψ(S, G) = 1.


    Pick any n such that 6 | n and 1/n < ϵ. We let V consist of n validators each with stake σ_v = 1. Letting N := n/6, we let S consist of 3N services each with π_s := 2 and α_s := 1. The graph thus satisfies <ref>. We now split S into two disjoint sets S_6 and S_3, where S_6 contains N of the services, and S_3 contains the remaining 2N services. We define the edges E as follows. First, we draw edges such that the neighborhoods N_Gs_s ∈ S_6 of services in S_6 evenly partition V into groups of 6. Next, we draw edges such that the neighborhoods N_Gs_s ∈ S_3 evenly partition V into groups of 3, and make the entire graph G a single connected component. An example of such a graph for n = 30 is shown in <ref>. The graph satisfies (<ref>) as each validator is adjacent to just one service from S_6 and one service from S_3, whence for any v ∈ V,
    
    ∑_s ∈ N_G vσ_v/σ_N_Gs·π_s/α_s = 1/6· 2 + 1/3· 2 = σ_v

    Thus, G is secure by <ref>.
    
        For all attacking coalitions (A, B) on G such that (A, B)  (S, V), σ_B ≥π_A + 1.
    
    
        To see this, notice first that as α_s = 1 for all s ∈ S, for any A ⊆ S, (A, N_GA) is the unique stake-minimal attacking coalition on G that attacks A. Writing A_3 := A ∩ S_3 and A_6 := A ∩ S_6, we must further have that
        
    σ_N_GA = 3A_3 + N_GA_6 ∖ N_GA_3

        and that
        
    π_A = 2A = 2A_3 + A_6

        It follows that
        
    σ_N_GA - π_A = A_3 - 2A_6 + N_GA_6 ∖ N_GA_3

        As all validators v have σ_v = 1, it suffices to show that σ_N_GA > π_A. Suppose for contradiction that σ_N_GA≤π_A. We must then have by <ref> that A_3≤ 2 A_6 else σ_N_GA > π_A. Furthermore, 2 A_6≤A_3 since N_GA_6 ∖ N_GA_3≥ 32A_6 - A_3. It follows that we must have A_3 = 2A_6 and N_GA_6 ∖ N_GA_3 = 0. However, if both of these conditions occurred simultaneously, then (A, N_GA) must be disconnected from the rest of the graph. As G forms a single connected component, and N_GA is a proper subset of V, it follows that σ_N_GA > π_A as desired.
    
    
    Next, pick any v ∈ V, and consider the initial shock D = v. Notice that for any C such that v ∉Γ(C), D ∈𝔻_0(C, G). Further, as ϵ > 1/n, D ∈𝔻_ψ(S, G) for any ψ≥ϵ. To finish the result, it suffices to show that (S, V ∖v) is a stable attack on G ↘ D. To see this, note that it is clearly an attacking coalition, and 
    
    π_S = n > n - 1 = σ_V ∖v

    whence it is a valid attack on G ↘ D. To see that it is a stable attack, it suffices to show that there is no A ⊆ S and proper B ⊂ V∖v such that (A, B) is a valid attack. It further suffices to consider proper A ⊂ S as V ∖v is unique a subset of validators B such that (S, B) is an attacking coalition on G ↘ D. The result now follows from <ref> as for any such attacking coalitions (A, B) on G ↘ D,
    
    σ_B ∪v≥π_A + 1 σ_B≥π_A

    whence (A, B) is not a valid attack on G ↘ D as desired. ]

 Paper Title: Decentralized and Uncoordinated Learning of Stable Matchings: A Game-Theoretic Approach.





Decentralized and Uncoordinated Learning of Stable Matchings: A Game-Theoretic Approach
    S. Rasoul Etesami† and R. Srikant

†Department of Industrial and Systems Engineering, Department of Electrical and Computer Engineering, and Coordinated Science Laboratory, University of Illinois Urbana-Champaign,  Urbana, IL 61801 (Email: etesami1@illinois.edu).
Department of Electrical and Computer Engineering and Coordinated Science Laboratory, University of Illinois Urbana-Champaign,  Urbana, IL 61801 (Email: rsrikant@illinois.edu).

    August 5, 2024
=============================================================================================================================================================================================================================================================================================================================================================================================================================================================


  







           
§ ABSTRACT 
We consider the problem of learning stable matchings in a fully decentralized and uncoordinated manner, where decentralized means that players make decisions individually without the influence of a central platform, and uncoordinated means that players do not need to synchronize their decisions using pre-specified rules. In this problem, there are n men and n women, each having preference over the other side. It is assumed that women know their preferences over men, but men are not aware of their preferences over women, and they only learn them if they propose and successfully get matched to women. A matching is called stable if no man and woman prefer each other over their current matches. When all the preferences are known a priori, the celebrated Deferred-Acceptance (DA) algorithm proposed by Gale and Shapley provides a decentralized and uncoordinated algorithm to obtain a stable matching. However, when the preferences are unknown, developing such an algorithm faces major challenges due to a lack of coordination. In this work, we achieve this goal by making a connection between stable matchings and learning Nash equilibria (NE) in noncooperative games. First, we provide a complete information game formulation for the stable matching problem with known preferences such that its set of pure NE coincides with the set of stable matchings, while its mixed NE with full support can be rounded in a decentralized manner to a stable matching. Relying on such a game-theoretic formulation, we show that for hierarchical structured markets, adopting the exponential weight (EXP) learning algorithm for the stable matching game achieves logarithmic regret with polynomial dependence on the number of players, thus answering a question posed in previous literature. Moreover, we show that the same EXP learning algorithm converges locally and exponentially fast to a stable matching in general matching markets. We complement this result by introducing another decentralized and uncoordinated learning algorithm that globally converges to a stable matching with arbitrarily high probability, leveraging the weak acyclicity property of the stable matching game. Since the global convergence of this algorithm can be slow in general matching markets, we investigate conditions under which stable matchings can be learned more quickly. In particular, for general matching markets, we show that if men receive stronger feedback when their proposals are rejected, it is possible to drive the market more rapidly toward an approximate stable matching. Our proposed game-theoretic framework bridges the discrete problem of learning stable matchings with the problem of learning NE in continuous-action games. 


Learning stable matchings, learning Nash equilibrium, weakly acyclic games, monotone games, decentralized learning, uncoordinated learning, online mirror descent, regret minimization. 


Stable matchings constitute one of the fundamental problems in computer science, economics, and engineering that have received considerable attention in the past decades. Stable matchings provide a desirable notion of stability in two-sided matching markets where the agents on each side of the market have preferences over the other side. For instance, in college admission, the applicants have different preferences over colleges and vice versa, and the goal is to match the applicants to the colleges so that no applicant-college pair would prefer to break their current matches and instead get matched to each other. Similar situations arise in other applications, such as kidney exchange programs, assigning jobs to workers, matching in online dating platforms, and scheduling jobs on heterogeneous machines. 

It is known that when all the preferences are known, stable matchings always exist and a simple decentralized and uncoordinated Deferred-Acceptance (DA) algorithm, first proposed by <cit.>, can find one such stable matches in polynomial number of iterations. Here, by a decentralized algorithm we refer to an algorithm in which players make decisions individually without the influence of a central platform, and uncoordinated means that players do not need to synchronize their decisions using pre-specified rules. While the DA algorithm provides a satisfactory solution to many practical applications, there are scenarios where either the information structure of the problem hinders the implementation of the DA algorithm or the stable matching achieved by the DA algorithm is inferior in terms of social optimality, especially when there exist many feasible stable matchings in the market. For instance, there are many recent emergence of online matching markets such as online labor markets (e.g., TaskRabbit, Upwork), online dating markets (e.g., Tinder, Match.com), and online crowdsourcing platforms (e.g., Amazon mechanical turk), where the users do not know their preferences a priori, and can repeatedly interact with the market to improve their matching quality <cit.>. The more a pair on both sides of the market gets matched, the more certain they will be about their preferences. Therefore, a question of interest is how the agents should interact with the market such that in the absence of any coordination, they will learn their preferences as quickly as possible, and their collective behavior results in a stable matching.

In this work, we consider a two-sided matching market posed as a marriage problem, where we refer to the agents of one side as men and to the agents of the other side as women. We assume both men and women have preferences on the other side and that women are aware of their ordinal preferences about men. However, men do not know their preferences about women and only learn those preferences if they propose and get matched, in which case the matched men receive a noisy estimate of their preferences. We assume that men/women cannot observe any other information (e.g., who is rejected/accepted), nor can they coordinate in any sense. The goal for men is to follow a decentralized and uncoordinated proposal strategy such that the entire market converges to a stable matching over time. 

One of the major challenges for learning a stable matching in markets with unknown preferences is the collisions. More precisely, when multiple men propose to the same woman, only one of them gets matched and receives useful information; all others get rejected and receive no information regarding their preferences for that woman. Therefore, resolving such collisions without coordination is a major issue. Moreover, even if a man gets matched to a woman, he receives a noisy utility drawn from an unknown distribution that could differ from his true preference. Thus, a collision-avoidance process should be repeated many times until men can accurately estimate their true preferences. One of our goals in this work is to provide decentralized and uncoordinated algorithms that can effectively learn the true preferences while incurring a minimum number of collisions. Moreover, our work provides an alternative approach for measuring the closeness of the generated dynamics to any stable matching in general matching markets and not necessarily to a specific stable matching in structured markets, which has been the main focus in the past literature. To that end, we make a connection between learning stable matchings in general markets with unknown preferences and learning Nash equilibrium (NE) in noncooperative games and leverage this connection to devise decentralized and uncoordinated algorithms for learning stable matchings.      



 §.§ Related Work


Stable matching was first introduced by <cit.> as a model for college admissions and to study the stability of marriages. Since then, there has been a tremendous effort on generalization and extension of stable matchings to various market settings <cit.>. In their seminal work, Gale and Shapley provided a simple DA algorithm in which men propose to their most preferred women, and the women reject all the received proposals except the most preferred one. They show that such a decentralized algorithm converges to a stable matching in at most O(n^2) steps, where n is the total number of men/women in the market. Unfortunately, when the agents' preferences are unknown, the DA algorithm is no longer guaranteed to converge to a stable matching without extra coordination. More precisely, if men fail to propose to their highest preferred women (due to a lack of knowledge of their true preferences), their subsequent proposals may become unpredictable. One reason is that if a man m is rejected by a woman w at a time instance t, that does not mean that m should stop proposing to w because it could be the case that another man mistakenly proposed to w at time t, hence causing the rejection of m by w. Therefore, in order to extend the DA algorithm to the case of unknown preferences, one requires some sort of coordination among the agents to avoid uninformative acceptance/rejection patterns. To address this issue, the earlier works <cit.> have proposed various coordination-based DA algorithms, where the main idea is to develop a phase-dependent process where in some phases the agents mostly explore, and in other phases, they implement the DA algorithm based on the current estimates of their unknown preferences.  


It is known that stable matchings with known preferences can be characterized using extreme points of a fractional polytope that is totally dual integral <cit.>. Moreover, stable matchings exhibit other nice properties that allow one to characterize their geometry using so-called rotations <cit.>. It was shown in <cit.> that uncoordinated random better response dynamics converge to a stable matching with probability one. Subsequently, <cit.> provided an exponential lower bound for the worst-case convergence time of the uncoordinated random better/best response dynamics to a stable matching. However, all these results are for the case when the market preferences are fully known so that the agents can compute their best/better responses and update their decisions accordingly. 

More recently, there has been a huge interest in matching markets with unknown preferences. However, depending on the information structure and the feedback received by the agents, one would expect a wide range of performance guarantees, which also depends on the type of algorithms followed by the agents (e.g., centralized/decentralized or coordinated/uncoordinated). The work <cit.> devised a randomized polynomial-time centralized algorithm for finding a stable matching with unknown deterministic preferences, where each time, the algorithm proposes a matching and receives feedback in terms of a blocking pair.[A unmatched pair of man-woman (m,w) is called blocking if both m and w prefer each other to their matched partner.] The work <cit.> considers learning a specific stable matching with unknown stochastic preferences by adopting a suitable notion of stable regret that measures the number of time instances that men propose to women other than their stable pair in that stable matching. <cit.> considered the problem of learning stable matchings in hierarchically structured markers[In fact, the hierarchical condition (a.k.a. sequential preference condition <cit.>) that we consider in Section <ref> is weaker than the α-reducible condition considered in   <cit.>, and our results hold for broader matching markets including the ones considered in <cit.>.] in which each submarket has a fixed pair (i.e., a pair of men-women that prefer each other the most in that submarket), and developed a phase-coordinated decentralized algorithm that achieves logarithmic regret with respect to the market's unique stable matching. However, the regret bound in <cit.> depends exponentially on other parameters such as the number of men/women. Furthermore, <cit.> considered learning stable matchings in an uncoordinated and decentralized fashion. However, they require stronger assumptions on the information feedback (e.g., a player observes the actions of other players in the previous round) and use a different performance metric than the one we consider in this work. We refer to <cit.> and <cit.> for other learning algorithms in two-sided matching markets with different performance guarantees. While all these works essentially address the same problem of learning a stable matching with unknown preferences, their algorithms differ substantially due to the information/feedback structure, the type of performance guarantee, and the amount of allowable coordination among the agents. 



 §.§ Contributions and Organization


We consider the problem of learning stable matchings in matching markets, where men do not know their preferences and want to achieve a stable matching in a decentralized and uncoordinated fashion. In particular, we consider the weakest type of feedback that men can receive: a man gets to observe a noisy version of his true preference only if his proposal gets accepted and receives no information otherwise. Our main objective is to provide a novel and principled approach to designing decentralized and uncoordinated learning algorithms for stable matchings through the lens of NE learning in noncooperative games. Our contributions can be summarized as follows:

    
  * We first provide a complete information continuous-action game formulation for the stable matching problem and show that its set of pure NE coincides with the set of stable matchings while its mixed NE points can be rounded in a decentralized way to a stable matching. This connection provides an alternative approach for measuring the closeness of the market dynamics to stable matchings through the lens of NE computation in games, hence extending the conventional notion of regret with respect to a specific stable matching to any arbitrary stable matching in general markets.
    
  * Leveraging this game-theoretic formulation, we provide a simple decentralized and uncoordinated algorithm EXP that converges globally to a stable matching in hierarchical markets and achieves logarithmic regret with only polynomial dependence on the number of players, hence improving the existing bounds in the past literature.
    
  * We then show that the EXP algorithm converges locally exponentially fast to a stable matching in general matching markets. In particular, we show that if EXP converges with positive probability, it converge globally to a stable matching.[Perhaps fast local convergence is the best one can hope for general markets because, for general matching markets, there are strong exponential lower bounds for the number of iterates of the better response dynamics to converge globally to a stable matching.]     
    
  * We also provide an alternative decentralized and uncoordinated algorithm that globally converges to a stable matching in general markets with arbitrarily high probability (albeit at a low convergence rate), thanks to the weakly acyclic property of the stable matching game. 
    
  * Finally, we address an information feedback design problem by seeking conditions in general matching markets that allow players to learn a stable matching quickly. In this regard, we leverage our game-theoretic framework to provide one such information feedback by establishing a connection to monotone games. Our results offer new insights into designing payoff functions that enable faster decentralized and uncoordinated algorithms for driving matching markets to their stable points. 




The paper is organized as follows. In Section <ref>, we formally introduce the problem. In Section <ref>, we provide a complete information game-theoretic formulation for the stable matching problem and prove several characterization results for its set of NE points. In Section <ref>, we provide a decentralized and uncoordinated algorithm (EXP) that achieves logarithmic regret for hierarchical markets with polynomial dependence on the number of players. In Section <ref>, we show that EXP  converges locally exponentially fast to a stable matching in general matching markets. In Section <ref>, we provide an alternative decentralized and uncoordinated algorithm that globally converges to a stable matching in general matching markets with arbitrarily high probability. In Section <ref>, we extend our results to matching markets with stronger feedback and establish a connection between fast global learning of a stable matching and learning a NE in monotone games. We conclude the paper by identifying some future research directions in Section <ref>.




§ PROBLEM FORMULATION




 §.§ Stable Matchings with Known Preferences


Here, we first introduce the stable matching problem with known preferences <cit.>. In this problem, there are a set M of men and a set W of women such that |M|=|W|=n (we use the term “agents" to refer to either men or women).[By introducing dummy agents with appropriate preferences, without loss of generality we may assume |M|=|W| <cit.>.] Each man m∈ M has a cardinal preference for each women w∈ W, denoted by μ_mw∈ (0, 1], such if μ_mw>μ_mw', it means that m prefers w over w'. Moreover, each woman w has an ordinal preference over the men, and we write m>_w m' if woman w strictly prefers m over m'. In this work we assume that no agent has ties in their preferences and we define 

    Δ=min_m, w≠ w'|μ_mw-μ_mw'|,    μ_min=min_m,wμ_mw,    μ_max=max_m,wμ_mw.

In particular, we note that Δ>0 and μ_min>0.

Given a matching and two matched pairs (m,w) and (m',w'), we say that (m,w') forms a blocking pair if μ_mw'>μ_mw and m>_w' m'. In other words, (m,w') is a blocking pair if both m and w' prefer each other over their current matches. A perfect matching is called a stable matching if it does not contain any blocking pair. 

 



 §.§ Stable Matchings with Unknown Preferences


In this work, we consider the problem of finding a stable matching with the main difference that men do not know their true preferences μ_mw, and they only get to learn them through interactions with the market. More precisely, for any m∈ M, w∈ W, we assume that the preference of man m about woman w is in the form of a [0,1]-supported unknown distribution 𝒟_mw with unknown mean μ_mw>0. We assume men and women interact in this market through a discrete-time process, where at any time t=0,1,2,… that a man m proposes to a woman w, he receives a feedback in the following form:

    
  * If the proposal of m gets rejected at time t because woman w has received an offer from a more preferred man m', i.e., m'>_w m, then m receives no information as feedback other than the fact that he was rejected by woman w at time t. 
    
  * If the proposal of m gets accepted by w at time t, then m observes an i.i.d. realization of his sampled preference drawn from 𝒟_mw, denoted by μ̂^t_mw.    


In the stable matching problem with unknown preferences, the agents' goals are to interact with the market through the information feedback structure described above such that the emerging dynamics converge to a stable matching of the market with known preferences {μ_mw}. We note that in the discrete-time process described above, men are the decision makers on whom to propose at each time t;  women merely respond to men's decisions by accepting their most preferred proposal (if they received any) and rejecting all others. 
       





§ A GAME-THEORETIC FORMULATION AND PRELIMINARY RESULTS


In this section, we provide a complete information noncooperative game formulation for the stable matching problem with known preferences whose set of pure Nash equilibrium (NE) points coincides with the set of stable matchings. Later, we will show how to leverage this formulation to extend our results for learning stable matchings with unknown preferences. Such a formulation has three main advantages: i) it reduces the learning task in matching markets to learning NE in continuous-action concave games, ii) it captures the selfish behavior of men (players) and the feedback they receive through their payoff functions, and iii) it simplifies the combinatorial structure inherent in learning stable matchings to learning NE in continuous-action games. While analyzing the agents' decisions directly on the discrete space of matchings seems complicated, which is why some earlier works need to restrict attention to special market structures <cit.>, our game-theoretic reduction simplifies this task to some extent, thanks to the extensive literature on NE learning in concave games.

Consider a complete information noncooperative game in which the action set of each man (player) is given by the set of women W, and the set of (mixed) strategies for each player is given by the probability simplex over the set of women W, i.e., the strategy set of player m is defined by 
    𝒳_m={x_m∈ℝ_+^|W|: ∑_w∈ Wx_mw= 1 }.
 
Moreover, given a strategy profile x=(x_m,x_-m), we define the payoff of player m by [Here, x_-m refers to the strategy vector of all the players except the mth one.]

    u_m(x)=∑_w∈ W(μ_mw∏_k>_w m(1-x_kw))x_mw,

where μ_mw>0 is the utility (true preference) received by man m if he gets matched to women w.


A strategy x_m for player m is called pure if it is zero in all coordinates except one. Otherwise, it is called a mixed strategy. A pure (mixed) strategy profile x^* is called a pure (mixed) NE if for any player m and any pure (mixed) strategy x_m, we have u_m(x_m,x^*_-m)≤ u_m(x^*_m,x^*_-m). 



A strategy x can be represented as a weighted bipartite graph ℋ(x)=(M∪ W, E(x)) between men and women. In this graph, the edges E(x)={(m,w): x_mw>0} are supported over the nonzero entries of x, with the weight of edge (m,w) given by x_mw.


One can interpret u_m(x) as the expected utility that player m would receive by successfully getting matched if each man independently proposes to women according to his mixed strategy distribution x_m. The reason for defining the payoff functions as in (<ref>) is that we want our devised algorithms to be implemented in a fully decentralized and uncoordinated manner among men by relying only on their received feedback (embedded into their payoff functions). While such payoff functions are highly nonlinear, they are essential to eliminate any degrees of coordination among the players. In other words, the cost of devising a fully coordination-free algorithm comes in analyzing more complex payoff functions with higher degrees of nonlinearity. We refer to Section <ref> for a simpler form of payoff functions with less degree of nonlinearities, which in turn requires stronger information feedback. The following are three properties of the payoff functions (<ref>). 


  (i) Player m's strategy does not have any impact on the coefficient terms μ_mw∏_k>_w m(1-x_kw), w∈ W. In particular, the gradient of u_m(·) with respect x_mw is given by 
    v_mw(x):=∇_mwu_m(x)=μ_mw∏_k>_w m(1-x_kw).
 

  (ii) The payoff of each player m is linear with respect to his own strategy x_m.

  (iii) For any fixed strategy of other players x_-m, player m always has a best response among pure strategies that is obtained by setting x_mw=1 for the woman w that achieves the maximum value v_mw(x), and x_mw=0 otherwise.    


In the remainder of this paper, we will refer to the above noncooperative game as the stable matching game and simply denote it by 𝒢=(M, {u_m}_m∈ M, {𝒳_m}_m∈ M). It is worth noting that although we are interested in obtaining a NE of the stable matching game, which is a complete information one-shot game, our goal is to learn one such NE by repeatedly playing the incomplete information game, in which the true preferences μ_mw (and hence the payoff functions) are not fully known. In fact, the set of pure/mixed NE points of the stable matching game has some interesting connections with the set of stable matchings with known preferences. The following theorem establishes one such result.  
 

A pure strategy profile x^*∈{0,1}^n^2 corresponds to the characteristic vector of a stable matching if and only if it is a pure NE for the stable matching game.


Suppose x^* is a pure NE. Then, we cannot have x^*_mw=x^*_m'w=1 for some woman w and m≠ m'. Otherwise, between m and m', the one that woman w prefers less receives zero utility, while he can always deviate to propose to a woman w' that does not have any offer and strictly improve his utility. Therefore, x^* must correspond to the characteristic vector of a perfect matching. To show that x^* is a stable matching, let us, by contradiction, assume that there exists a blocking pair with respect to the matching x^* and denote it by (m,w). Moreover, let m' and w' be the corresponding matches for w and m under x^*, that is x^*_mw'=1 and x^*_m'w=1. Since (m,w) is a blocking pair, we have μ_mw>μ_mw' and m>_w m'. The payoff of player m under NE x^* equals u_m(x^*)=μ_mw'. Now if player m changes its strategy to x_m:=(0,x_mw=1), his payoff becomes 

    u_m(x_m,x^*_-m)=∑_j∈ Wμ_mj∏_k>_j m(1-x^*_kj)x_mj=μ_mw∏_k>_w m(1-x^*_kj)=μ_mw,

where the second equality holds because m'<_wm so that the term 1-x^*_m'w does not appear in the product ∏_k>_w m(1-x^*_kw), and moreover x^*_kw=0 ∀ k≠ m', which implies ∏_k>_w m(1-x^*_kw)=1. This shows that u_m(x_m,x^*_-m)=μ_mw>μ_mw'=u_m(x^*), contradicting the fact that x^* is a NE.  

Conversely, let x^* be the characteristic vector of a stable matching, and by contradiction, assume it is not a NE. Then, there exists at least one player m who can deviate and strictly improve his payoff. In particular, we can consider his pure strategy best response move, which is guaranteed by (iii), and denote his most preferred woman by w. Moreover, let m' and w' be the corresponding matches for w and m under the stable matching x^*, i.e., x^*_m'w=x^*_mw'=1. The current payoff of m under the pure strategy x^* equals u_m(x^*)=μ_mw', while his payoff after such deviation becomes

    u_m(x_m,x^*_-m)=∑_j∈ Wμ_mj∏_k>_j m(1-x^*_kj)x_mj=μ_mw∏_k>_w m(1-x^*_kw)≤μ_mw.
 
Since 0< μ_mw'=u_m(x^*)<u_m(x_m,x^*_-m), we obtain μ_mw'< μ_mw. Moreover, from relation (<ref>) and the fact that u_m(x_m,x^*_-m)>0, we must have ∏_k>_w m(1-x^*_kw)=1, which implies m'<_w m. This together with μ_mw'< μ_mw shows that (m,w) is a blocking pair, contradicting that x^* is stable.              


While the above equilibrium characterization theorem concerns pure NE points, it is possible that the stable matching game admits a mixed NE. The following is one example. 

Let us consider an instance of the stable matching game with three men M={m_1,m_2,m_3} and three women W={w_1,w_2,w_3}. We assume the preferences of men and women are given by

    μ_m_1w_1=2, μ_m_1w_2=1, μ_m_1w_3=ϵ,            m_3<_w_1 m_1<_w_1 m_2,   μ_m_2w_1=3, μ_m_2w_2=4, μ_m_2w_3=5,            m_3<_w_2 m_2<_w_2 m_1,   μ_m_3w_1=ϵ, μ_m_3w_2=12, μ_m_3w_3=6,          m_1<_w_3 m_2<_w_3 m_3,

where ϵ>0 is a small positive constant. Then, the stable matching game has a mixed NE that is supported over seven edges comprising of two cycles of length 4 that share the common edge (m_2,w_2) (see, Remark <ref>). In particular, the seven positive entries of that mixed NE are given by 

    x_m_1w_1=μ_m_2w_1/μ_m_2w_2,              x_m_1w_2=1-μ_m_2w_1/μ_m_2w_2,   x_m_2w_1=1-μ_m_1w_2/μ_m_1w_1,          x_m_2w_2=1-μ_m_2w_2μ_m_3w_3/μ_m_2w_1μ_m_3w_2,       x_m_2w_3=μ_m_1w_2/μ_m_1w_1+μ_m_2w_2μ_m_3w_3/μ_m_2w_1μ_m_3w_2-1,   x_m_3w_2=μ_m_2w_1/μ_m_2w_3,               x_m_3w_3=1-μ_m_2w_1/μ_m_2w_3.
   


As Example <ref> suggests, mixed NE points may represent unpredictable bipartite graphs (Remark <ref>), and it is difficult to make general statements about their structure. However, the following theorem establishes an interesting connection between mixed and pure NE points, implying that obtaining a mixed NE with full support on the women's side is as satisfactory as obtaining a pure NE. 


Let x be any mixed NE in which each woman receives at least one (fractional) proposal.[If this condition does not hold, the theorem statement continues to hold for the induced submarket where each woman has received at least one fractional proposal.] For each man m, let w_m be the least preferred woman among those that he fractionally proposes to them, i.e., w_m=_w∈ W{μ_mw: x_mw>0}. Then {(m,w_m), m∈ M} forms a stable matching (and hence a pure NE) for the stable matching game. 


Please see Appendix <ref> for the proof. 



Theorem <ref> suggests some strategies for obtaining decentralized solutions. We can first design a decentralized algorithm (see Algorithm <ref> in conjunction with the preference estimation oracle of Algorithm <ref>) to learn a mixed NE of the stable matching game and acquire estimated preferences. Subsequently, each man can convert his fractional mixed strategy into a pure strategy in a decentralized manner by proposing only to his least (estimated) preferred woman among those he fractionally proposes to. This process ensures that the resulting pure strategy profile will be a pure NE with high probability, and the probability of error vanishes over time. Importantly, each man has complete access to his own estimated preferences, and the rounding process outlined in Theorem <ref> does not require knowledge of others' mixed strategies. Each man only needs to be aware of his own mixed strategy and estimated preferences. This approach offers a simple decentralized method to convert mixed NE points into stable matchings. 




§ LOGARITHMIC REGRET FOR HIERARCHICAL MARKETS






In this section, we consider the stable matching game with unknown preferences and develop a decentralized and uncoordinated algorithm for learning its NE points. In particular, we focus on matching markets with specific structures because without imposing any assumption on the matching market, there are exponential lower bounds for the number of iterations to find a stable matching through better response dynamics <cit.>. One example of such a structured matching market is the hierarchical matching market, where it was shown in <cit.> that if men follow a certain decentralized (but coordinated) algorithm, the men's expected regret (see Definition <ref>) is at most logarithmic in time but exponential in terms of other parameters such as the number of men n. In particular, the authors in <cit.> raised the question of how to remove the exponential dependence of the regret on n. In this section, we show that Algorithm <ref> achieves this goal whose regret is logarithmic in time with a polynomial factor in terms of other parameters. Moreover, Algorithm <ref> is uncoordinated and, hence, removes the phase-dependent coordination required by the algorithm in <cit.>. Throughout this section, we impose the following hierarchical assumption on the matching market.    


We assume that the sets of men and women can each be ordered as M={m_1,…,m_n} and W={w_1,…,w_n} such that any man and woman with the same rank prefer each other above any other partner with a lower rank; i.e., man m_k prefers woman w_k to women w_k+1, w_k+2,…,w_n, and woman w_k
prefers man m_k to men m_k+1, m_k+2,…,m_n. It is easy to see that under this hierarchical assumption, the matching {(m_k,w_k), k=1,…,n} is a stable matching. 
 


The condition imposed in Assumption <ref> is also known as the Sequential Preference Condition (SPC) <cit.> and is weaker than the α-reducible condition considered in <cit.>, which assumes that each submarket has a fixed-pair: A pair (m,w) is called a fixed pair if both m and w prefer each other the most among anyone else in that submarket.[In fact, one can show that a matching market has a unique stable matching if and only if it satisfies α-reducible condition <cit.>.] Since Assumption <ref> is weaker than the α-reducible condition, all our results immediately applies to α-reducible markets.



Let us assume that players follow a decentralized and uncoordinated algorithm 𝒜 that results in a sequence of proposals α^t=(α^t_m, m∈ M), t=1,…,T. The expected regret of 𝒜 equals

    R(T)=∑_t=1^T 𝔼[1_{∃ k∈ [n]: α^t_m_k≠ w_k}],

which is the expected number of times that the men's proposals do not form a stable matching. 






 §.§ Algorithm Design and Preliminaries


The algorithm that we propose is an adaptation of the online dual mirror descent for adversarial multi-arm bandit problem (see, e.g., EXP3 Algorithm in <cit.>) to the stable matching game, where for simplicity we refer to it as the EXP algorithm. In particular, it uses an entropy regularizer and adaptive stepsize for updating the players' mixed strategy distributions over time.

To describe the EXP algorithm, let us denote the gradient vector of player m by v_m(x)=∇_m u_m(x) such that for any woman w, we have v_mw(x)=μ_mw∏_k>_w m(1-x_kw). In particular, we note that u_m(x)=⟨ x_m, v_m(x)⟩. Therefore, when restricting to a pure strategy (action) profile α^t=(α_1^t,…,α^t_n)∈ W×⋯× W that is played at time t, the w-th coordinate of the gradient vector is given by

    v_mw^t:=μ_mw1_{α^t_k≠ w ∀ k>_w m},

where 1_{·} denotes the indicator function. On the other hand, by playing a pure strategy profile α^t, each man m receives a feedback only for the woman α^t_m that he has proposed to her at time t. In particular, if α_m^t=w, player m observes the payoff μ^t_mw1_{α^t_k≠ w  ∀ k>_w m}1_{α_m^t=w} at time t, where we recall that μ^t_mw is the realization of a random reward drawn independently from an unknown distribution 𝒟_mw with unknown mean μ_mw and bounded variance σ^2_mw∈ [0, 1]. 


During the course of the algorithm, at each time t, each player m holds a mixed strategy X^t_m∈𝒳_m and updates it whenever he receives new information. He then computes his adjusted mixed strategy X̂^t_m=(1-γ^t)X^t_m+γ^t/n1, where γ^t>0 is a mixing parameter and 1 is the vector of all ones. Using ideas from importance sampling for bandit problems <cit.>, and to obtain an unbiased estimator of the actual gradient vector, player m constructs an estimate vector v̂^t_m by normalizing the received feedback with his adjusted mixed strategy probabilities as

    v̂^t_mw=μ^t_mw1_{α^t_k≠ w ∀ k>_w m}/X̂^t_mw·1_{α_m^t=w},     w∈ W.

The reason why we use adjusted mixed strategies X̂^t with additional mixing parameter γ^t is to ensure that the mixed strategies remain bounded away from zero by a positive quantity. That makes the estimators (<ref>) have bounded variance (see Lemma <ref>), which allows us to use martingale concentration results to bound the probability of various events.

Let us denote the (random) score vector of player m at time t-1 by L̂^t-1_m=(L̂_mw,w∈ W). Player m uses L̂^t-1_m to compute his  mixed strategy vector X_m^t using the logit update rule (<ref>). He then proposes to a woman α_m^t chosen independently at random according to his adjusted mixed strategy X̂_m^t, and receives as feedback v̂_m^t with entries given by (<ref>). Player m then updates his score vector by L̂^t_m=L̂^t-1_m+v̂^t_m and proceeds to the next round. The detailed description of the EXP algorithm is summarized in Algorithm <ref>. 





In order to analyze the convergence behavior of Algorithm <ref> to a stable matching, let {ℱ^t-1}_t=1^∞ be the increasing filtration sequence that is adapted to the history of the random processes generated by Algorithm <ref>. More precisely, 
    ℱ^t-1={X^τ,L̂^τ,α^τ,v̂^τ, τ=0,1,…,t-1}∪{X^t},
 contains all the realized events up to time t-1 except the realization of pure strategies α^t and the rewards μ^t at time t. In particular, all relevant processes at time t-1 as well as X^t and X̂^t are ℱ^t-1-measurable, but α^t and v̂^t are not ℱ^t-1-measurable. 

We begin with the following lemma, which shows that the feedback v̂^t_m received by player m at time t is conditionally an unbiased and bounded estimator of the actual gradient vector v_m^t.



The received feedback is a conditionally unbiassed and bounded estimate of the gradients 

    𝔼[v̂_m^t|ℱ^t-1]=v_m(X̂^t)  ∀ m,      𝔼[(v̂_mw^t)^2|ℱ^t-1]≤σ^2/X̂^t_mw≤n/γ^t  ∀ m,w,

where σ^2:=max_m,w𝔼[(μ̂^t_mw)^2]∈ [0, 1].


Consider any pair (m,w). Since X̂^t_mw is ℱ^t-1-measurable and μ̂^t_mw is drawn independently from the algorithm's past decisions (and hence is independent of ℱ^t-1 and α^t), we have

    𝔼[v̂_mw^t|ℱ^t-1]   =𝔼[μ^t_mw]/X̂^t_mw·𝔼[1_{α^t_k≠ w  ∀ k>_w m}·1_{α_m^t=w}|ℱ^t-1]   =μ_mw/X̂^t_mwℙ{α_m^t=w|ℱ^t-1}∏_k>_w mℙ{α^t_k≠ w|ℱ^t-1}   =μ_mw∏_k>_w m(1-X̂^t_kw)=v_mw(X̂^t),

where the second equality holds because conditioned on ℱ^t-1, the random variables {α^t_k, ∀ k} are mutually independent as they are drawn independently from the given distributions {X̂^t_k, ∀ k}. Similarly,

    𝔼[(v̂_mw^t)^2|ℱ^t-1]   =𝔼[(μ̂^t_mw)^2]/X̂^t_mw∏_k>_w mℙ{α^t_k≠ w|ℱ^t-1}≤σ^2/X̂^t_mw≤n/γ^t,   ∀ m,w.
         






 §.§ Performance of Algorithm <ref> for Hierarchical Markets

 
Here, we analyze the performance of Algorithm <ref> for hierarchical matching markets. In the following analysis, we assume that a positive lower bound for c:=1/8min_k∈ [n]{Δ,μ_m_kw_k} is known to all the players so that they can choose their mixing parameters in Algorithm <ref>. However, as we discuss in Appendix <ref>, this condition can be relaxed up to some extent by using an uncoordinated and decentralized UCB gap estimation oracle in which the players use their estimated gaps to tune their mixing parameters. 
The main result of this section is presented in Theorem <ref>, which essentially shows that for hierarchical matching markets, Algorithm <ref> achieves an expected regret that is logarithmic in T and polynomial in terms of other parameters. However, before delving into the analysis, we first provide an overview of the main steps in the proof.


    
  * First, we define a “good" event Ω, which represents the set of circumstances where the difference between the accumulated realized rewards, η^t(L̂^t_mw̃ - L̂^t_mw), for any m, w, w̃, and t, remains close to its conditional expected value. In particular, Lemma <ref> shows that the event Ω occurs with very high probability. This allows us to analyze the performance of Algorithm <ref> under its conditional mean trajectory while incurring only a small, controllable loss. 
    
  * Conditioned on the event Ω, Lemma <ref> shows that, due to the market structure and payoff functions, as time progresses, more pairs in the market are matched according to the underlying hierarchical order with very high probability. Additionally, the size of such hierarchical nested matchings grows with very high probability after a polynomial number of steps. 
    
  * Finally, in Theorem <ref>, we combine the results of Lemma <ref> and Lemma <ref> to bound the expected regret of Algorithm <ref> by conditioning on the event Ω.  






Let us consider the stochastic dynamics of Algorithm <ref> that are given by

    X^t_mw=exp(η^t L̂^t-1_mw)/∑_w'exp(η^t L̂^t-1_mw'),   X̂^t_m=(1-γ^t)X^t_m+γ^t/n1,   L̂^t_mw=L̂^t-1_mw+v̂^t_mw.




Define Y^t_mw=∑_τ=1^t v_mw(X̂^τ), where v_mw(X̂^τ)=μ_mw∏_k>_w m(1-X̂^τ_mw), and consider the event

    Ω:={|η^t(L̂^t-1_mw̃-L̂^t-1_mw)-η^t(Y^t-1_mw̃-Y^t-1_mw)|≤ 2c√(t),  ∀ t≥ 1, ∀ m, w,w̃},
 
where c=1/8min_k∈ [n]{Δ,μ_m_kw_k} is a constant parameter. The following lemma provides a high probability bound for the event Ω to occur under appropriate choice of stepsize and mixing parameter. 


Fix an arbitrary δ∈ (0, 1), and suppose each player follows Algorithm <ref> with stepsize η^t=1/√(t) and mixing parameter γ^t=Mlog t/t, where M=4n/clog1/δ. Then, ℙ{Ω}≥ 1-δ. 


Fix an arbitrary man m. For any w, w̃, as in (<ref>), we can write

    η^t(L̂^t-1_mw̃-L̂^t-1_mw)   =η^t∑_τ=1^t-1(v_mw̃(X̂^τ)-v_mw(X̂^τ))+η^t S^t-1_mw̃-η^t S^t-1_mw   =η^t(Y^t-1_mw̃-Y^t-1_mw)+η^t S^t-1_mw̃-η^t S^t-1_mw,

where S^t-1_mw:=∑_τ=1^t-1(v̂^τ_mw-v_mw(X̂^τ)) ∀ m,w. According to Lemma <ref>, {v̂^τ_mw-v_mw(X̂^τ)} is a martingale difference sequence that satisfies 

    |v̂^τ_mw-v_mw(X̂^τ)|≤n/γ^τ,    𝔼[(v̂^τ_mw-v_mw(X̂^τ))^2|ℱ^τ-1]≤n/γ^τ.

By taking u=n/γ^t-1, v^2=∑_τ=1^t-1n/γ^τ, and x=ct in Lemma <ref> (see Appendix <ref>) and noting that the event {V^t≤ v^2} holds with probability 1, we obtain 

    ℙ{η^t|S_mw^t-1|≥ c√(t)}=ℙ{|S_mw^t-1|≥ ct}≤ 2exp(-Mc^2 t^2/2n(∑_τ=1^t-1τ/logτ+ct(t-1)/log (t-1))).

Since f(τ):=τ/logτ is a concave function for τ≥ 100, using Jensen's inequality, we have

    1/t-1∑_τ=1^t-1τ/logτ=∑_τ=1^t-11/t-1f(τ)≤ f(∑_τ=1^t-11/t-1τ)=∑_τ=1^t-1τ/t-1/log(∑_τ=1^t-1τ/t-1) =t/2logt/2.

Therefore, ∑_τ=1^t-1τ/logτ≤t(t-1)/2logt/2≤t^2/log t. Substituting this relation and t(t-1)/log (t-1)≤t^2/log t into (<ref>) we get

    ℙ{η^t|S_mw^t-1|≥ c√(t)}≤ 2exp(-Mc^2 t^2/2n(1+c)t^2/log t)= 2exp(-Mc^2log t/2n(1+c))≤2δ/(nπ t)^2,

where the last inequality is obtained by choosing M sufficiently large (e.g., M=4n/clog1/δ). Thus, using the union bound, we can write

    ℙ{η^t |S_mw^t-1|< c√(t) ∀ t≥ 1, ∀ m, w}≥ 1-∑_t=1^∞2n^2δ/(nπ t)^2> 1-δ.

Finally, using (<ref>) and (<ref>), one can see that

    ℙ{Ω}   =ℙ{|η^t(L̂^t-1_mw̃-L̂^t-1_mw)-η^t(Y^t-1_mw̃-Y^t-1_mw)|≤ 2c√(t),  ∀ t≥ 1, ∀ m, w, w̃}   =ℙ{|η^t S^t-1_mw̃-η^t S^t-1_mw|≤ c√(t),  ∀ t≥ 1, ∀ m, w, w̃}   ≥ℙ{η^t |S_mw^t-1|< c√(t) ∀ t≥ 1, ∀ m, w}> 1-δ.
 



Before we present the main result of this section, we first prove the following key lemma. 


Assume that each man follows Algorithm <ref> with stepsize η^t=1/√(t) and mixing parameter γ^t=Mlog t/t, where M=4n/clog1/δ. Let a_1≥…≥ a_n be a sequence defined by a_1=Δ/2 and a_k=1/4min_i∈ [k]{Δ,μ_m_iw_i}, ∀ k=2,…,n. Conditioned on the event Ω, there exist t_1≤…≤ t_n=Õ(n^2/c^2log1/δ), such that for any t≥ t_k and any w≠ w_k, we have 

    X̂^t_m_kw_k≥1-γ^t/1+(n-1)e^-η^t a_k t+γ^t/n,    X̂^t_m_kw≤(1-γ^t)e^-η^t a_k t/1+(n-1)e^-η^t a_k t+γ^t/n.
    


We use induction on k∈ [n] to prove the lemma. For the base of the induction k=1, we note that for any X̂∈ [0,1]^n^2, we have

    v_m_1w_1(X̂)=μ_m_1w_1∏_k>_w_1 m_1(1-X̂_kw_1)=μ_m_1w_1,   v_m_1w(X̂)=μ_m_1w∏_k>_w m_1(1-X̂_kw)≤μ_m_1w.
 
Therefore, 

    Y^t-1_m_1w_1-Y^t-1_m_1w   =∑_τ=1^t-1(v_m_1w_1(X̂^τ)-v_m_1w(X̂^τ))   ≥∑_τ=1^t-1(μ_m_1w_1-μ_m_1w)≥Δ (t-1).

Define a_1=Δ/2 and let t_1=4. Conditioned on the event Ω, for any t≥ t_1 and any w≠ w_1, we have

    X^t_m_1w_1/X^t_m_1w   =e^η^t(L̂^t-1_m_1w_1-L̂^t-1_m_1w)≥ e^η^t (Y^t-1_m_1w_1-Y^t-1_m_1w)- 2c√(t)   ≥ e^η^t Δ (t-1)-2c√(t)≥ e^η^t a_1 (2t-2)-1/2η^t a_1 t   = e^η^t a_1 (3/2t-2)≥ e^η^t a_1t,

where the third inequality holds because c≤a_1/4, and the last inequality holds because t≥ t_1. Moreover, 

    1=X^t_m_1w_1+∑_w≠ w_1X^t_m_1w≤(1+(n-1)e^-η^t a_1 t)X^t_m_1w_1.

Combining relations (<ref>) and (<ref>) and using the fact that X̂^t_mw=(1-γ^t)X^t_mw+γ^t/n, for any t≥ t_1 and w≠ w_1, we have

    X̂^t_m_1w_1≥1-γ^t/1+(n-1)e^-η^t a_1 t+γ^t/n,        X̂^t_m_1w≤(1-γ^t)e^-η^t a_1 t/1+(n-1)e^-η^t a_1 t+γ^t/n.
 



Next, by induction hypothesis, suppose that the statement holds for the first k-1 pairs (m_ℓ,w_ℓ), ℓ=1,…,k-1 with corresponding sequences a_1≥ a_2…≥ a_k-1 and t_1≤ t_2≤…≤ t_k-1, and let us consider the kth pair (m_k,w_k) of the market. Notice that for any X̂∈ [0,1]^n^2, we have 

    v_m_kw_k(X̂)=μ_m_kw_k∏_m_ℓ>_w_k m_k(1-X̂_m_ℓw_k)≥μ_m_kw_k∏_ℓ=1^k-1 (1-X̂_m_ℓw_k).

We consider two cases: 

Case I: If w=w_i for some i∈{1,…,k-1}, then

    v_m_kw(X̂)=v_m_kw_i(X̂)=μ_m_kw_i∏_m_ℓ>_w_i m_k(1-X̂_m_ℓw_i)≤μ_m_kw_i(1-X̂_m_iw_i).

Therefore, using (<ref>) and (<ref>) we can write

    Y^t-1_m_kw_k   -Y^t-1_m_kw=∑_τ=1^t-1(v_m_kw_k(X̂^τ)-v_m_kw_i(X̂^τ))   ≥∑_τ=1^t-1(μ_m_kw_k∏_ℓ=1^k-1(1-X̂^τ_m_ℓw_k)-μ_m_kw_i(1-X̂^τ_m_iw_i))   ≥∑_τ=1^t-1(μ_m_kw_k(1-∑_ℓ=1^k-1(γ^t/n+(1-γ^t)e^-η^t a_ℓ t/1+(n-1)e^-η^t a_ℓ t))-μ_m_kw_i(1-γ^t/n-1-γ^t/1+(n-1)e^-η^t a_i t))    ≥∑_τ=1^t-1(μ_m_kw_k-γ^t-∑_ℓ=1^k-1(1-γ^t)e^-η^t a_ℓ t/1+(n-1)e^-η^t a_ℓ t-μ_m_kw_i(n-1)e^-η^t a_i t+γ^t/1+(n-1)e^-η^t a_i t)   ≥∑_τ=1^t-1(μ_m_kw_k-γ^t-∑_ℓ=1^k-1e^-η^t a_ℓ t-μ_m_kw_i(n-1)e^-η^t a_i t-μ_m_kw_iγ^t)   ≥∑_τ=1^t-1(μ_m_kw_k-2γ^t-2(n-1)e^-η^t a_k-1 t),

where the second inequality holds by the induction hypothesis for t≥ t_k-1, and the last inequality holds because a_1≥…≥ a_k-1. Thus, for any time step t≥ t_k-1 that satisfies e^-η^t a_k-1 t≤γ^t and t-2/∑_τ=1^t-1γ^τ≥4n/μ_m_kw_k, we have 

    Y^t-1_m_kw_k-Y^t-1_m_kw   ≥∑_τ=1^t-1(μ_m_kw_k-2nγ^t)= μ_m_kw_k(t-1)-2n∑_τ=1^t-1γ^t≥μ_m_kw_k/2t.


Case II: If w∉{w_1,…,w_k}, then μ_m_kw_k≥μ_m_kw+Δ. Using (<ref>) and the fact that μ_m_kw≥ v_m_kw(X̂) for any X̂∈ [0,1]^n^2, we can write

    Y^t-1_m_kw_k-Y^t-1_m_kw   =∑_τ=1^t-1(v_m_kw_k(X̂^τ)-v_m_kw(X̂^τ))   ≥∑_τ=1^t-1(μ_m_kw_k∏_ℓ=1^k-1(1-X̂^τ_m_ℓw_k)-μ_m_kw)   ≥∑_τ=1^t-1(μ_m_kw_k(1-∑_ℓ=1^k-1X̂^τ_m_ℓw_k)-μ_m_kw)   ≥∑_τ=1^t-1((μ_m_kw_k-μ_m_kw)-∑_ℓ=1^k-1(γ^t/n+(1-γ^t)e^-η^t a_ℓ t/1+(n-1)e^-η^t a_ℓ t))   ≥∑_τ=1^t-1(Δ-γ^t-∑_ℓ=1^k-1e^-η^t a_ℓ t)   ≥∑_τ=1^t-1(Δ-γ^t-(n-1)e^-η^t a_k-1 t).

Thus, for any integer t≥ t_k-1 that satisfies e^-η^t a_k-1 t≤γ^t and t-2/∑_τ=1^t-1γ^τ≥2n/Δ, we have 

    Y^t-1_m_kw_k-Y^t-1_m_kw≥∑_τ=1^t-1(Δ-nγ^t)=Δ(t-1)-n∑_τ=1^t-1γ^τ≥Δ/2t.

As a result, by taking

    t_k:=min{t≥ t_k-1|  tη^t/ln(1/γ^t)≥1/a_k-1,  t-2/∑_τ=1^t-1γ^τ≥max{4n/μ_m_kw_k, 2n/Δ}},   a_k:=1/4min_i∈ [k]{Δ, μ_m_iw_i},

we have a_k-1≥ a_k, t_k≥ t_k-1. In particular, for t≥ t_k both relations (<ref>) and (<ref>) hold and we have 

    Y^t-1_m_kw_k-Y^t-1_m_kw≥min{Δ/2, μ_m_kw_k/2} t≥ 2a_k t, ∀ t≥ t_k, ∀ w≠ w_k.
 
Now, using a similar argument as in the base case, for any t≥ t_k and w≠ w_k, we have

    X^t_m_kw_k/X^t_m_kw=e^η^t(L̂^t-1_m_kw_k-L̂^t-1_m_kw)≥ e^η^t (Y^t-1_m_kw_k-Y^t-1_m_kw)- 2c√(t)≥ e^η^t 2a_k t-2c√(t)≥ e^η^t 2a_k t-η^t a_k t= e^η^t a_kt,

where the second inequality uses (<ref>), and the last inequality holds because c=min_k∈ [n]{Δ/8,μ_m_kw_k/8}≤a_k/2. The above relation, together with similar arguments as in deriving (<ref>) and (<ref>) in which a_1 is replaced by a_k, gives us the desired result (<ref>). 

Finally, using the stepsize and mixing parameters η^t=1/√(t) and γ^t=Mlog t/t in relation (<ref>), one can obtain an explicit upper bound for t_n as

    t_n≤min{t∈ℤ_+|  √(t)/log(t/Mlog t)≥1/a_n-1,  t-2/Mlog^2 t≥max{4n/μ_m_nw_n, 2n/Δ}},

for which it suffices to take t_n=Õ(n^2/c^2log1/δ), where Õ(·) hides the logarithmic factors.




Assume that each man follows Algorithm <ref> with η^t=1/√(t) and γ^t=Mlog t/t, where M=4n/clog T and c=1/8min_k∈ [n]{Δ,μ_m_kw_k}. Then, the expected regret of Algorithm <ref> in hierarchical matching markets is at most R(T)=O(n^2/clog^3 T).  


Conditioned on the event Ω, which according to Lemma <ref> it occurs with probability at least 1-δ, the expected regret can be upper-bounded as 

    𝔼[R(T)|Ω]   = ∑_t=1^T 𝔼[1_{∃ k: α^t_m_k≠ w_k}|Ω]≤∑_t=1^T∑_k=1^n 𝔼[1_{α^t_m_k≠ w_k}|Ω]   ≤∑_t=1^T∑_k=1^n 𝔼[𝔼[1_{α^t_m_k≠ w_k}|X̂^t_m_k]|Ω]=∑_t=1^T∑_k=1^n 𝔼[ℙ{α^t_m_k≠ w_k|X̂^t_m_k}|Ω]   =∑_t=1^T∑_k=1^n 𝔼[1-X̂^t_m_kw_k|Ω]≤ nt_n+∑_t=t_n+1^T∑_k=1^n 𝔼[1-X̂^t_m_kw_k|Ω]   ≤Õ(n^3/c^2log1/δ)+∑_t=t_n+1^T∑_k=1^n (1-γ^t/n-1-γ^t/1+(n-1)e^-η^t a_k t)   ≤Õ(n^3/c^2log1/δ)+∑_t=t_n+1^T∑_k=1^n(n-1)e^-η^t a_k t+γ_t/1+(n-1)e^-η^t a_k t   ≤Õ(n^3/c^2log1/δ)+n^2∑_t=1^T e^-η^t a_n t+n∑_t=1^Tγ_t   ≤Õ(n^3/c^2log1/δ)+2n^2/a_n^2+nMlog^2 T   =O(n^3/c^2log1/δ+n^2/clog1/δlog^2 T),

where the third inequality uses Lemma <ref>, the fifth inequality uses the fact that a_n:=min_k∈ [n]{a_k}, and the last inequality is obtained by choosing γ^t=Mlog t/t, η^t=1/√(t), M=4n/clog1/δ, and noting that c≤ a_n.

Finally, since δ∈ (0,1) can be chosen arbitrary, by choosing δ=1/T, we can write

    𝔼[R(T)]   =ℙ{Ω}𝔼[R(T)|Ω]+ℙ{Ω̅}𝔼[R(T)|Ω̅]   ≤ (1-δ)O(n^3/c^2log1/δ+n^2/clog1/δlog^2 T)+δ n^2T    =O(n^2/clog^3 T).



It is worth noting that the regret of Algorithm <ref> is logarithmic in T, with a polynomial factor n^2/c in terms of the problem parameters, thereby addressing a question raised in <cit.>. Furthermore, Algorithm <ref> is both decentralized and uncoordinated, overcoming the limitations of the algorithm proposed in <cit.>, which is notably more complex and requires phase-dependent coordination among men according to the Deferred-Acceptance algorithm by Gale and Shapley. 






§ EXPONENTIAL LOCAL CONVERGENCE FOR GENERAL MATCHING MARKETS
 

In this section, we consider learning a stable matching in general matching markets (i.e., without any hierarchical assumption) and show that the same decentralized and uncoordinated algorithm EXP (Algorithm <ref>) converges locally at an exponential rate to a stable matching when players' strategies get sufficiently close to one of the stable matchings. To that end, we first consider the following technical lemma, which shows that if a strategy profile is sufficiently close to a stable matching, then the reward of choosing an action according to that stable matching is strictly the best decision.    

Let X^* be a pure NE of the stable matching game and set c=1/8min{Δ,μ_min}. For any strategy profile x that satisfies x-X^*_1≤c/μ_max, we have v_mw^*(x)-v_mw(x)>c ∀ w≠ w^*, where w^* is the woman that m is matched to her under the pure NE X^*. 


Please see Appendix <ref> for the proof.


Next, we consider the following lemma that will be used to capture the local behavior of the dynamics of Algorithm <ref> around a pure NE.  


Suppose X^* is a pure NE and {X^t} be the sequence of iterates generated by Algorithm <ref>. Moreover, let A^t-1= ∑_τ=1^t-1(v_mw^*(X^τ)-v_mw(X^τ)), where w^* is the woman that man m is matched to her under the pure NE X^*. Then, the following statements hold:


  * i) If X^t-X^*_1≤c/250n^2, then η^tL̂^t-1_mw^*-η^tL̂^t-1_mw≥ln (n^2/2c)+6  ∀ m,w≠ w^*.

  * ii) If η^tL̂^t-1_mw^*-η^tL̂^t-1_mw≥ln(n^2/2c) ∀ m,w≠ w^*, then X^t-X^*_1≤ c.

  * iii) Let t_0 be the first time such that 1/η^t_0+1-1/η^t_0≤ c/(ln(n^2/2c)+3). If for some t≥ t_0 we have η^t A^t-1≥ln(n^2/2c)+3 and v_mw^*(X^t)-v_mw(X^t)≥ c ∀ m,w≠ w^*, then η^t+1 A^t≥ln(n^2/2c)+3.
 


Please see Appendix <ref> for the proof. 


Now we are ready to state the main result of this section. 

Fix an arbitrary δ∈ (0, 1), and suppose each player follows Algorithm <ref> with some mixing sequence {γ^t} and decreasing stepsize sequence {η^t} that satisfie the following conditions:

    (∑_τ=1^t-1γ^t-1/γ^τ)^-1≤η^t≤min{(2n^2∑_τ=1^t-1γ^τ)^-1, (8nln(nπ/√(δ)t)∑_τ=1^t-11/γ^τ)^-1/2}.

Let t_0 be the first time such that 1/η^t_0+1-1/η^t_0≤ c/(ln(n^2/2c)+3) and X^t_0-X^*≤c/250n^2. Then with probability at least 1-δ, the sequence {X^t} converges to the pure NE X^* exponentially fast, i.e.,

    ℙ{X^t+1-X^*_1≤ 41 nexp(-ctη^t+1) ∀ t≥ t_0 | X^t_0-X^*_1≤c/250n^2}≥ 1-δ.
  


Fix an arbitrary man m, and assume X^*_mw^*=1. For any w≠ w^*, we have

    η^tL̂^t-1_mw^*   -η^tL̂^t-1_mw=η^t∑_τ=1^t-1(v̂^τ_mw^*-v̂^τ_mw)   =η^t∑_τ=1^t-1(v_mw^*(X̂^τ)-v_mw(X̂^τ))+η^t∑_τ=1^t-1(v̂^τ_mw^*-v_mw^*(X̂^τ))-η^t∑_τ=1^t-1(v̂^τ_mw-v_mw(X̂^τ))   =η^t∑_τ=1^t-1(v_mw^*(X̂^τ)-v_mw(X̂^τ))+η^t S^t-1_mw^*-η^t S^t-1_mw,
 
where S^t-1_mw:=∑_τ=1^t-1(v̂^τ_mw-v_mw(X̂^τ)). Moreover, for any m and w, using the mean-value theorem, there exists a θ=λX̂^τ+(1-λ)X^τ such that 

    |v_mw(X̂^τ)-v_mw(X^τ)|   =|∇ v_mw(θ)· (X̂^τ-X^τ)|   ≤∇ v_mw(θ)_∞X̂^τ-X^τ_1   ≤ n X̂^τ-X^τ_1   =nγ^τ∑_mX_m^τ-1/n1_1 ≤ n^2γ^τ.

Substituting this relation into (<ref>) and using the triangle inequality, we can write

    |η^tL̂^t-1_mw^*-η^tL̂^t-1_mw-η^t∑_τ=1^t-1(v_mw^*(X^τ)-v_mw(X^τ))|   ≤η^t |S^t-1_mw^*|+η^t |S^t-1_mw|+2n^2η^t∑_τ=1^t-1γ^τ   ≤η^t |S^t-1_mw^*|+η^t |S^t-1_mw|+1,

where the last inequality follows from the upper bound on the choice of stepsize η^t. 

Next, notice that according to Lemma <ref>, {v̂^τ_mw-v_mw(X̂^τ)} is a martingale difference sequence that almost surely satisfies 

    |v̂^τ_mw-v_mw(X̂^τ)|≤n/γ^τ,   𝔼[(v̂^τ_mw-v_mw(X̂^τ))^2|ℱ^τ-1]≤𝔼[(v̂_mw^τ)^2|ℱ^τ-1]≤n/γ^τ.

Therefore, by taking u=n/γ^t-1, v^2=∑_τ=1^t-1n/γ^τ, and x=1/η^t in Lemma <ref>, we obtain

    ℙ{η^t |S_mw^t-1|≥ 1}≤ 2exp(-(1/η^t)^2/2n(∑_τ=1^t-11/γ^τ+1/η^tγ^t-1))≤ 2exp(-(1/η^t)^2)/4n∑_τ=1^t-11/γ^τ),
 
where the second inequality holds because using the lower bound on η^t we have 1/η^tγ^t-1≤∑_τ=1^t-11/γ^τ. Since by the choice of stepsize (1/η^t)^2/∑_τ=1^t-1 1/γ^τ≥ 8nln(nπ/√(δ)t), we get ℙ{η^t |S_mw^t-1|≥ 1}≤2δ/(nπ t)^2. In particular, using the union bound, the probability that η^t |S_mw^t-1|< 1 for any t≥ 1 and any m, w, is at least

    ℙ{η^t |S_mw^t-1|< 1 ∀ t≥ 1, ∀ m, w}≥ 1-∑_t=1^∞2n^2δ/(nπ t)^2> 1-δ.

Let us define A^t-1= ∑_τ=1^t-1(v_mw^*(X^τ)-v_mw(X^τ)). Using (<ref>) and (<ref>), with probability at least 1-δ, we have

    |η^tL̂^t-1_mw^*-η^tL̂^t-1_mw-η^tA^t-1|≤ 3,  ∀ t≥ 1, ∀ m, w.
    

Next, let us assume X^t-X^*_1∈c/250n^2. Then, for any m, w≠ w^*, we have v_mw^*(X^t)-v_mw(X^t)≥ c and η^tL̂^t-1_mw^*-η^tL̂^t-1_mw≥ln(n^2/2c)+6, by Lemma <ref> and Lemma <ref> (part i), respectively. Using relation (<ref>), it means that η^tA^t-1≥ln(n^2/2c)+3, and thus by Lemma <ref> (part iii), we have η^t+1 A^t≥ln(n^2/2c)+3. This in view of Lemma <ref> (part ii) shows that X^t+1-X^*_1≤ c. Similarly, Lemma <ref> implies v_mw^*(X^t+1)-v_mw(X^t+1)≥ c ∀ m,w≠ w^*, which in view of Lemma <ref> (part iii) shows that η^t+2 A^t+1≥ln(n^2/2c)+3. Hence, using relation (<ref>), η^t+2L̂^t+1_mw^*-η^t+2L̂^t+1_mw≥ln(n^2/2c) ∀ m,w≠ w^*, and thus X^t+2-X^*_1≤ c by Lemma <ref> (part ii). 

The above inductive argument shows that if X^t_0-X^*≤2c/n^2 for some t_0≥ 1 such that 1/η^t_0+1-1/η^t_0≤ c/(ln(n^2/2c)+3), then with probability at least 1-δ, we have X^t-X^*≤ c ∀ t≥ t_0, and in particular, using Lemma <ref>, v_mw^*(X^t)-v_mw(X^t)≥ c ∀ m,w, t≥ t_0 . Therefore, with probability at least 1-δ, for any m, w≠ w^*, t≥ t_0, we have 

    X^t+1_mw^*/X^t+1_mw=exp(η^t+1L̂^t_mw^*-η^t+1L̂^t_mw)≥exp(η^t+1A^t-3)≥exp(ctη^t+1-3).

Therefore, X^t+1_m-X^*_m_1=2(1-X^t+1_mw^*)≤ 2nexp(-ctη^t+1+3), which shows that

    ℙ{X^t+1-X^*_1≤ 2nexp(-ctη^t+1+3) ∀ t≥ t_0| X^t_0-X^*_1≤c/250n^2}≥ 1-δ.




    In the statement of Theorem <ref>, the condition that t≥ t_0  is not restrictive because if Algorithm <ref> converges to a pure NE, then eventually for some t_0, we have 1/η^t_0+1-1/η^t_0≤ c/(ln (n^2/2c)+3) and  X^t_0-X^*≤c/250n^2. In particular, t_0 is a constant that scales polynomially in terms of other parameters (see, e.g., Remark <ref>). We note that it is possible to further relax this condition by a constant scaling of η^t, i.e., by setting η^t←1/c(ln (n^2/2c)+3)η^t and rescaling all other parameters such as the mixing sequence γ^t accordingly. However, that requires the players to know c=1/8min{Δ,μ_min} to choose their stepsize η^t, while knowing c is not required in the stepsize condition given in (<ref>).    



In fact, there exist feasible choices for η^t and γ^t that satisfy the stepsize condition (<ref>) in Theorem <ref>. For instance, if for some α,β∈ (0,1),  α> max{1-β, 1+β/2}, we choose 

    η^t=Θ(1/ln( t/√(δ))t^α),      γ^t=Θ(1/t^β),

then condition (<ref>) is satisfied for any t_0≥ (α b/c)^1/(1-α), where b=ln(n^2/2c)+3. More specifically, by choosing γ^t=1/t^1/3 and η^t=1/t^3/4, the exponential convergence rate of Theorem <ref> would be in the order of exp(-ct^1/4) for any t_0≥ (b/c)^4. We note that condition (<ref>) provides only a sufficient condition to achieve local exponential convergence and we did not attempt to fully characterize all feasible choices of stepsize/mixing parameters. However, there could be other choices of stepsize that do not satisfy condition (<ref>) while still achieving local exponential convergence.


Although Theorem <ref> shows that Algorithm <ref> achieves an exponential convergence rate, its convergence guarantee holds in a local sense, i.e., when the dynamics get sufficiently close to a neighborhood of a stable matching.
In the following theorem, we complement this result by showing that if the dynamics of Algorithm <ref> converge with positive probability, it must converge to a mixed NE of the stable matching game. 


Under the same stepsize assumptions of Theorem <ref>, if Algorithm <ref> converges to a limit point X^* with positive probability, then X^* must be a (possibly mixed) NE of the matching game. 


Please see Appendix <ref> for the proof.    







§ GLOBAL LEARNING OF STABLE MATCHINGS IN GENERAL MARKETS


In the previous section, we showed that the decentralized and uncoordinated EXP algorithm converges locally to a stable matching at an exponential rate in general matching markets. This prompts the question of whether there is an uncoordinated and decentralized algorithm capable of globally learning a stable matching in general markets, regardless of the convergence rate. In this section, we answer this question affirmatively by introducing an alternative algorithm that leverages the weak acyclic property of the stable matching game. To that end, we first present several useful properties of the stable matching game, which will be instrumental in establishing our main convergence results. 



 
A pure strategy profile x is a good state if all the men matched under x are at their best responses.[In a pure strategy profile x=(x_m, m∈ M), a matched man is the one whose pure strategy x_m has exactly one coordinate equal to 1 (say wth coordinate), and all other coordinates are zero, and no other man of higher preference proposes to w, i.e., x_kw=0 ∀ k>_w m.] In other words, a good state x refers to a partial matching in which the matched men are at their best responses.
 


The following theorem shows that if the stable matching game with known preferences starts from a good state, then any sequence of best response dynamics by the players reaches a pure NE. Unless stated otherwise, we assume that each player m is aware of both his own true preferences and those of all the women, enabling him to compute his payoff function. This assumption will be relaxed in Theorem <ref>, where players seek to learn these unknown preferences through interactions with others. 


Consider the stable matching game that starts from a good initial state x^0. Consider any sequence of updates where at each time t, an arbitrary subset of players that includes at least one unsatisfied player updates their actions by playing their best responses. Then, the sequence of updates converges to a pure NE in no more than |M||W| iterations.


Let us encode the ordinal preferences of each woman w into cardinal values λ_wm∈ℝ^+ such that m_1>_w m_2 if and only if λ_wm_1>λ_wm_2. Given a  pure strategy profile x, let r_w(x)∈{λ_wm, m∈ M} be woman w's reward for her most preferred man who is matched to her under x. By convention, we set r_w(x)=0 if w is not matched to any man under the pure-strategy profile x. We show that r_w(x) is nondecreasing for each woman w (and for at least one w it is strictly increasing) such that the potential function Φ(x)=∑_wr_w(x) strictly increases after each iteration. Since each r_w(x) can increase at most |M| times, the sequence must terminate at a pure NE after no more than |M||W| iterations. We show this by arguing that during the sequence of updates, no matched man m wants to deviate unless he is rejected by his partner w because she has received a better offer, implying that r_w(x) must have increased. Note that this statement holds initially because x^0 is a good state where no matched man wants to deviate.    

More precisely, let us by contradiction assume that t is the first iteration at which some woman w loses her more preferred man during the sequence of updates, i.e., r_w(x^t)<r_w(x^t-1). This can happen only if man m that is matched to w at time t-1 (i.e., x^t-1_mw=1) is selected to update at time t whose best response is to leave w for another women w' (i.e., x^t_mw'=1). Otherwise, r_w(x^t-1) can only increase or remain the same because women reject any proposal other than their most preferred one. Note that since the game starts from a good state x^0, we must have t≥ 2.  

According to the payoff structures, the best response of player m at time t is to propose to his most preferred achievable woman w', i.e., among all women who rank m higher or equal to their match under x^t-1, m proposes to the woman w' which brings him the highest utility μ_mw'. Note that in particular μ_mw'>μ_mw. Thus, the first time τ≤ t-1 when man m proposed and got matched to woman w, the woman w' was his more preferred choice and the reason why m did not propose to w' at time τ was because w' was not achievable, i.e., x^τ-1_m'w'=1 for some man m'>_w' m. Since w' was not achievable for m in x^τ-1 but she is achievable in x^t-1 implies that r_w'(x^t-1)<r_w'(x^τ-1). Therefore, r_w'(x^τ')<r_w'(x^τ'-1) for some τ≤τ'≤ t-1, contradicting the choice of t. 



In fact, one can view the DA algorithm of <cit.> as a special version of the above lemma in which all the players are selected to best respond at each round, and the initial good state is the empty matching. Lemma <ref> shows that the stable matching game admits an ordinal potential function if the game starts from a good state (we refer to Appendix <ref> for another interpretation of this potential function). However, that does not imply that the stable matching game is an ordinal potential game <cit.>, which requires the existence of a potential function whose changes are aligned with players' payoff changes from any initial state. Nonetheless, the existence of such a “weak” potential function allows us to show the weak acyclicity of the stable matching game as defined next.


A better response path in a finite action noncoopertaive game is a sequence of pure strategy profiles x^1,x^2,…,x^L such that for each ℓ=1,…,L-1, x^ℓ+1 is obtained from x^ℓ by letting some player i_ℓ to play a better response. A game is called weakly acyclic if from any pure strategy profile x^0, there is a finite length better response path to a pure NE.



A pure NE x^*=(x^*_m,x^*_-m) of the stable matching game is called strict if each player has a unique best response at the equilibrium x^*, i.e., for every m and any pure strategy x_m≠ x^*_m, we have u_m(x_m,x^*_-m)<u_m(x^*_m,x^*_-m).   


Using the above definitions, we are now ready to state the main result of this section, which provides a decentralized and uncoordinated algorithm for learning stable matchings in general markets with unknown preferences. The algorithm is an adaptation of the sample experimentation algorithm for weakly acyclic games <cit.> to the stable matching game. Intuitively, each player m goes through a sequence of phase-dependent exploration/exploitation. At the end of each episode, player m evaluates his payoff for each action he has taken during the last episode. He then updates his baseline action (an action that brings him the highest reward in the past episode up to some tolerance) and sticks to that baseline action most of the time in the next episode while still exploring new actions with a small probability. The detailed description of the algorithm is summarized in Algorithm <ref>.   






Let p,ω∈ (0,1) be arbitrary probabilities and ϵ≤min{1-p/n, δ/4n, Δ-δ/4n}, where δ∈ (0, Δ). For each man m, there exists a sufficiently large episode length τ_m=τ_m(p,ϵ), such that if each player m follows Algorithm <ref> with episode length τ_m, for all sufficiently large times t>0, the pure strategy profile α^t will be a (fixed) stable matching with probability at least p.
   

First, we show that the stable matching game is a weakly acyclic game. If the game starts from a good state, by Lemma <ref>, any sequence of best responses by the players is a better response path that leads to a pure NE in |M||W| iterations. Thus, we only need to show that from any initial state, there is a better response path to a good state. This is also true by sequentially letting a matched man who wants to deviate (if there is such a man) to play his best response. In this way, either the number of matched men decreases due to collisions (which can happen at most |M| times), or between every two consecutive collisions, there could be at most |M| best response updates. Therefore, |M|^2 steps are enough to bring us from any arbitrary state to a good state, and the overall number of best response iterates to obtain a pure NE starting from any initial state is at most |M||N|+|M|^2.



Next, we note that any pure NE in the stable matching game is a strict NE.[In a weakly acyclic game, an arbitrarily high probability convergence can be guaranteed to a strict NE <cit.>. If we do not have strictness property, the dynamics may oscillate between two nonstrict NE points with a positive probability.] This follows directly by the fact that there are no ties between men's preferences in the stable matching game. More precisely, let α^* be a pure NE (stable matching) and assume u_m(α^*)=μ_mw. For any α_m≠α^*_m, either u_m(α_m,α^*_-m) equals 0 or μ_mw' for some w'≠ w. Because α^*_m is the best response of player m at the NE and there are no ties between man m's preferences, we must have μ_mw'<μ_mw. Since 0<μ_mw, in either case we have u_m(α_m,α^*_-m)<u_m(α^*_m,α^*_-m).

Finally, it was shown in <cit.> that the class of sample experimentation dynamics in weakly acyclic games converges with arbitrarily high probability to a strict NE if for any joint baseline action b^s at the start of an episode s, u^s_mw is an arbitrarily close estimation of u_m(w,b^s_-m) for sufficiently large episode length τ_m=τ_m(p,ϵ). Since Algorithm <ref> is an adaptation of the sample experimentation dynamics to the stable matching game, the convergence result follows from <cit.> if we can show that for any δ^*=min{δ/4, Δ-δ/4}>0, p<1, and 1-(1-ϵ)^n-1<δ^*/2, for sufficiently large τ_m we have[This is an equivalent version of Claim 3.4 in <cit.> to the stable matching game.]  

    ℙ{|u_mw^s-u_m(w,b^s_-m)|≥δ^*}≤ 1-p.

This statement is also true because during each episode, players sample their baseline actions (or explore a new action) independently of others, and moreover, μ^t_mw are i.i.d. realizations of a random variable with mean μ_mw. Thus, relation (<ref>) holds using the law of large numbers, which completes the proof.   


While Algorithm <ref> provides a decentralized and uncoordinated algorithm for learning stable matchings in general markets with arbitrarily high probability, the convergence time to a NE may take a long time. This seems inevitable to some extent in view of the exponential lower bound in terms of n such that the best response dynamics in general matching markets converge to a stable matching <cit.>. 






§ MATCHING MARKETS WITH STRONGER INFORMATION FEEDBACK


So far, we have assumed that men receive the minimum amount of information feedback as they interact with the market; namely, men only get to observe whether their proposal gets rejected or, if matched, they observe a random realization of their preference. In this final section, we consider the learning problem from a new perspective by addressing the following question: Is there stronger information feedback that will allow us to derive the market faster globally to a stable matching in a decentralized and uncoordinated fashion?  Therefore, we address the problem of learning 
a stable matching from an information feedback design perspective by showing that if the players are allowed to receive a bit more information as feedback, then the stable matching game will become a monotone game, hence allowing a wide range of decentralized and uncoordinated learning algorithms for obtaining its NE points. 


Let us consider a simplified version of the stable matching game with the same set of players M and strategy spaces 𝒳_m, m∈ M, except that the payoff of player m is now given by   

    u_m(x)=∑_w∈ Wμ_mw(1-∑_k>_w mx_kw)x_mw.




The simplified payoff function (<ref>) can be viewed as the expected perceived reward for player m if he randomly and independently of others proposes to women according to his mixed strategy x_m∈𝒳_m but under slightly stronger feedback: if player m's proposal gets rejected from a woman w, he also gets to know how many men higher ranked than him proposed to w. In this way, each man m learns how many more preferred men compete with him to get matched to woman w. Such information feedback can be implemented by using a waiting list order for each woman, which shows the ranking of a rejected man m in that list. Henceforth, we refer to such information feedback as the waiting list feedback. 

Let α_k∈ W denote the random woman that player k proposes to her, which is drawn independently from his mixed strategy distribution x_k. Under the waiting list feedback information, player m has access to the information ∑_k>_w m1_{α_k=w} if he proposes to w, and hence can estimate his perceived payoff by proposing to a woman w as μ_mw(1-∑_k>_w m1_{α_k=w}),  Therefore, the expected perceived payoff of player m under the waiting list feedback equals

    𝔼[∑_w∈ Wμ_mw(1-∑_k>_w m1_{α_k=w})1_{α_m=w}]   =∑_w∈ W𝔼[μ_mw(1-∑_k>_w m1_{α_k=w})|α_m=w]ℙ(α_m=w)   =∑_w∈ Wμ_mw(1-∑_k>_w mx_kw)x_mw=u_m(x),
    
where the second equality holds by independency of α_k ∀ k∈ M. 

The motivation for waiting list feedback and the payoff function (<ref>) is that if we view 1-∑_k>_w mx_kw as the remaining fractional capacity of women w for being matched, using the approximation 1-∑_k>_w mx_kw∼ e^-∑_k>_w mx_kw, one can think that man m loses his interest to offer women w exponentially in terms of the occupied capacity of woman w by higher preferred men. One immediate consequence of such simplified payoff functions is the following proposition whose proof follows nearly verbatim arguments as those in Theorems <ref> and <ref>, and is omitted from here for the sake of brevity.


Consider the stable matching game with payoff functions (<ref>). Then, x^* is a pure NE if and only if it is the characteristic vector of a stable matching. Moreover, any mixed NE of this game with full support can be rounded in a decentralized manner to a pure NE.  



Next, we consider the following definition of monotone games <cit.>. 

A continuous-action game (M,{U_m}_m∈ M, {𝒳}_m∈ M) is called monotone if for any two action profiles x,x'∈𝒳_1×⋯×𝒳_|M|, we have (x'-x)^T(F(x')-F(x))≤ 0, where 
    F(x)=(∇_m U_m(x), m∈ M)^T.
  


An important property of monotone games is that they admit a variety of decentralized learning algorithms that converge globally to a NE, which, under further assumptions, it can be shown that the converge rate could be exponential or polynomial <cit.>. The following theorem shows that the regularized stable matching game with waiting list feedback is indeed a monotone game under a suitable assumption on the preferences. 
 


Consider the regularized stable matching game with list feedback, where each player aims to maximize its regularized payoff function given by U_m(x_m,x_-m):=u_m(x_m,x_-m)-β/2x_m^2. Then, for any two strategy profiles x',x, we have

    (x'-x)^T(F(x')-F(x))=-1/2∑_w∈ W(x_w'-x_w)^TQ^w(x_w'-x_w),
 
where x_w is the vector x_w=(x_mw, m∈ M)^T, and Q^w is a symmetric matrix whose diagonal entries are 2β and all the entries in row m up to the diagonal entry in that row equal μ_mw.  In particular, if β>nμ_max/2, the regularized stable matching game with waiting list feedback is a monotone game.  


Note that due to the linearity of U_m(x) with respect to player m's strategy x_m, we have 

    ∇_mw U_m(x)=μ_mw(1-∑_k>_w mx_kw)-β x_mw, ∀ m,w,

where ∇_mw U_m(x) denotes the gradient of U_m(x) with respect to x_mw. Thus, we can write

    (x'   -x)^T(F(x')-F(x))=∑_m (x_m'-x_m)^T (∇_m U_m(x')-∇_m U_m(x))   =∑_m∑_w (x_mw'-x_mw)(∇_mw U_m(x')-∇_mw U_m(x))   =∑_m∑_w -μ_mw(x_mw'-x_mw)(∑_k>_w mx'_kw-∑_k>_w mx_kw+β(x'_mw-x_mw))   =-∑_m∑_w∑_k>_w mμ_mw(x_mw'-x_mw)(x'_kw-x_kw)-β∑_m∑_w (x'_mw-x_mw)^2   =-∑_w(∑_m∑_k>_w mμ_mw(x_mw'-x_mw)(x'_kw-x_kw))-β∑_m∑_w (x'_mw-x_mw)^2   =-∑_w(∑_m∑_kμ_mw1_{k>_w m}(x_mw'-x_mw)(x'_kw-x_kw))-β∑_w∑_m (x'_mw-x_mw)^2.

For each woman w, let us define a vector x_w=(x_mw, m∈ [M])^T, and an n× n incidence preference matrix A^w whose entries are given by A^w_mk=μ_mw1_{k>_w m}, ∀ m,k∈ [n]. Then, using (<ref>) we have

    (x'-x)^T(F(x')-F(x))   =-∑_w(∑_m∑_k A^w_mk(x_mw'-x_mw)(x'_kw-x_kw))       -∑_w (x'_w-x_w)^T(β I)(x'_w-x_w)   =-∑_w(x_w'-x_w)^T(A^w+β I) (x_w'-x_w)   =-1/2∑_w(x_w'-x_w)^TQ^w(x_w'-x_w),

where Q^w=(A^w)^T+A^w+2β I. Next, we note that permuting the rows and columns of a matrix by the same permutation matrix preserves the semi-definiteness property as it only changes the basis of its eigenvectors. Thus, without loss of generality, we may assume 1>_w 2>_w …>_w n. However, in that case, we may assume that for any w, the matrix A^w will be a lower triangular matrix whose diagonal entries are all zero. In particular, Q^w=(A^w)^T+A^w+2β I would be a symmetric matrix whose elements in each row m up to the diagonal entry are identical and equal to μ_mw. Finally, for sufficiently large β>nμ_max/2, the matrix Q^w is positive semidefinite for all w, and we have -(x_w'-x_w)^TQ^w(x_w'-x_w)≤ 0  ∀ w. This shows that (x'-x)^T(F(x')-F(x))≤ 0, and completes the proof.  



The monotonicity property in Theorem <ref> holds when the regularizer parameter β is sufficiently large. The regularizer implies that players tend to maximize their approximate payoff functions. Thus, instead of learning a NE for the payoff functions (<ref>), they learn an ϵ-NE where the approximation error ϵ grows as β increases. As a result, applying decentralized learning algorithms to learn a NE of the regularized game only guarantees an ϵ-stable matching in the sense that the players will achieve a solution in which each player would be ϵ worse off than what he could achieve in a stable matching. Therefore, the regularizer parameter trades off between the market's convergence speed and the accuracy of the learned stable matching.  



Finally, we note that without imposing any assumption on the agents' preferences, one cannot expect the stable matching game to be a monotone game. The reason is that monotone games imply the global uniqueness of a NE <cit.>. However, by Proposition <ref>, any stable matching is a pure NE, and for general matching markets, there could be many stable matchings under arbitrary preferences. Moreover, the assumption of waiting list feedback is not necessary for faster convergence in any general market; it is merely one choice that we consider in this work, and there could be other types of information feedback that allow faster convergence. Nevertheless, our game-theoretic approach to the stable matching problem provides valuable insights on how to properly design the utilities and the information feedback to devise faster decentralized and uncoordinated algorithms for learning stable matchings.   









§ CONCLUSIONS


In this work, we considered the problem of learning stable matchings with unknown preferences in a fully decentralized and uncoordinated manner. Using a game-theoretic formulation, we showed how to leverage the rich literature for learning NE in noncooperative games to derive the matching market to its stable points in a principled way. In particular, we established several global and local convergence results with performance guarantees for learning stable matchings in hierarchical markets, general markets, and under weak/strong information feedback. Our results provide new insights for learning stable matchings through NE learning in noncooperative games and bridge the discrete problem of learning a stable matching with learning NE in continuous-action games.  

This work opens several avenues for future research. One interesting direction is to characterize an information-theoretic limit below and above which efficient learning of a stable matching in a fully decentralized and uncoordinated fashion may or may not be possible. Moreover, our waiting list feedback is merely an initial attempt to design faster decentralized and uncoordinated algorithms under richer information feedback. Extending our results to include other types of information feedback and studying their effect on the convergence speed of the market to its stable points would be another interesting research direction.      





IEEEtran








§ SUPPLEMENTARY MATERIALS





 §.§ Freedman's Concentration 
Inequality for Martingales
 

Let {ζ_τ}_τ=1^t be a martingale difference sequence with max_τ∈ [t]|ζ_τ|≤ u, and consider the corresponding martingale S^t=∑_τ=1^tζ_τ with S^1=0. Let V^t=∑_τ=1^t𝔼[ζ^2_τ|ℱ^t-1] be the sum of the conditional variances. Then, for any x>0 and v∈ℝ, we have 
    ℙ{|S^t|≥ x, V^t≤ v^2}≤ 2exp(-x^2/2(v^2+ux)).



The proof can be found in <cit.>.   




 §.§ Proof of Theorem <ref>



Given a mixed NE x, let ℋ(x)=(M∪ W, E(x)) be the induced bipartite graph by x, where E(x)={(m,w): x_mw>0} denotes all the edges with fractionally positive proposals. Moreover, let us denote the neighboring nodes of m and w in ℋ(x) by N_m(x) and N_w(x), respectively. Since x is a NE, the value v_mw(x)=μ_mw∏_k>_w m(1-x_kw) should be the same for all w∈ N_m(x) and equals to u_m(x). Otherwise, using properties (i) and (ii), player m could strictly increase his payoff by putting his entire strategy mass on the woman w^*=_w∈ N_m(x)v_mw(x). The fact that u_m(x)=v_mw(x) ∀ w∈ N_m(x) follows because x_m is a probability distribution that is supported over N_m(x).

Next, we note that a man m cannot be the most preferred choice for more than one woman in ℋ(x). Otherwise, let w_1,w_2∈ N_m(x) be two different women who rank m the highest among their fractional matches in N_w_1(x) and N_w_2(x), respectively. Then, we have 
    ∏_k>_w_1 m(1-x_kw_1)= ∏_k>_w_2 m(1-x_kw_2)=1,
 which together with the above argument implies that

    μ_mw_1=μ_mw_1∏_k>_w_1 m(1-x_kw_1)=v_mw_1(x)=u_m(x)=v_mw_2(x)=μ_mw_2∏_k>_w_2 m(1-x_kw_2)=μ_mw_2.
  
This contradicts the fact that there are no ties between the preferences of man m.    

Since each woman receives at least one fractional proposal, N_w(x)≠∅ ∀ w. Let m_w be the unique most favorite man of w in N_w(x). Using the above argument, the elements of {m_w, w∈ W} must be distinct such that ℳ_1={(m_w,w), w∈ W} forms a matching. Viewing this matching from men's side, one can represent ℳ_1 in the form of ℳ_2={(m,w_m), m∈ M}, where w_m=_w∈ N_m(x)μ_mw is the least preferred woman among those that man m fractionally proposed to. To see this equivalence, fix a man m and let w_0 be his match in M_1, i.e., m_w_0=m. Observe that each w'∈ N_m(x)∖{w_0} has a unique distinct most favorite man m_w'≠ m. That means that 

    v_mw'(x)=μ_mw'∏_k>_w' m(1-x_kw')≤μ_mw'(1-x_m_w'w')<μ_mw'.

On the other hand, since w_0 ranks m the highest among those who propose to her, we have v_mw_0(x)=μ_mw_0∏_k>_w_0 m(1-x_kw')=μ_mw_0. Since at a NE v_mw_0(x)=v_mw'(x), we must have μ_mw_0<μ_mw', ∀ w'∈ N_m(x)∖{w_0}, which implies w_0=w_m. Thus, for any pair (m_w_0,w_0)∈ℳ_1 there is a pair (m,w_m)∈ M_2 such that (m_w_0,w_0)=(m,w_m), which shows that ℳ_1=ℳ_2. 

Finally, we will show that the binary characteristic vector of the matching {(m,w_m), m∈ M}, denoted by x̂, is indeed a pure NE. We will show that by arguing that any strictly better deviation in x̂ is also a strictly better deviation in x, contradicting the fact that x is a NE. On the contrary, let m_w be a man who wants to deviate from his pair w to a woman w'≠ w. Such a deviation is beneficial to player m_w only if m_w'<_w' m_w. Otherwise, his payoff will reduce from u_m_w(x̂)=μ_m_ww to 0. Henceforth, we may assume m_w'<_w' m_w. The fact that the payoff of player m_w is currently u_m_w(x̂)=μ_m_ww and he wants to deviate to w' implies that μ_m_ww<μ_m_ww'. As m_w is the most preferred man for woman w in ℋ(x), we also have u_m_w(x)=μ_m_ww. Since m_w'<_w' m_w, the woman w' would prefer m_w over all the men in N_w'(x). Thus, in the mixed NE x, man m_w could also deviate by solely proposing to w', in which case his payoff would strictly increase from u_m_w(x)=μ_m_w,w to μ_m_ww', which is a contradiction.    






 §.§ Proof of Lemma <ref>



Fix a player m and assume X^*_mw^*=1. Since X^* is a pure NE, for any w≠ w^* we have

    v_mw^*(X^*)=μ_mw^*> v_mw(X^*)=μ_mw∏_k>_w m(1-X^*_kw)∈{0, μ_mw}.

On the other hand, we have 

    v_mw^*(x)-v_mw^*(X^*)   =μ_mw^*∏_k>_w^* m(1-x_kw^*)-μ_mw^*   ≥ -μ_mw^*∑_k>_w^* mx_kw^*   ≥ -μ_mw^*/2x-X^*_1,

where the second inequality holds because X^*_mw^*=1 implies X^*_kw^*=0 ∀ k≠ m, and hence 1/2x-X^*_1≥∑_k>_w^* mx_kw^*. Now we consider two cases: 

i) If v_mw(X^*)=μ_mw, then v_mw(x)≤μ_mw< μ_mw^* ∀ x, which implies v_mw^*(X^*)-v_mw(x)≥Δ. Combining this relation with (<ref>) and because x-X^*_1≤Δ/μ_max, we get v_mw^*(x)-v_mw(x)>Δ/2>c. 

ii) If v_mw(X^*)=0, there exists k>_w m such that X^*_kw=1. Then, v_mw(x)≤μ_mw(1-x_kw)= μ_mw(X^*_kw-x_kw) ≤1/2μ_mwX^*-x_1, and we have v_mw^*(X^*)-v_mw(x)≥μ_mw^*-1/2μ_mwX^*-x_1. Combining this relation with (<ref>) and because x-X^*_1≤μ_min/μ_max, we get v_mw^*(x)-v_mw(x)>μ_min/2> c.    




 §.§ Proof of Lemma <ref>


i) Suppose X^t-X^*_1≤c/250n^2, then X^t_m-X^*_m_1=2(1-X^t_mw^*)≤c/250n^2 ∀ m. Since X^t_m is a probability distribution, we can write 
    X^t_mw^*/X^t_mw≥1-c/500n^2/c/500n^2= 500n^2/c-1, ∀ m,w≠ w^*.
 
Therefore, for any m,w≠ w^*, we have η^tL̂^t-1_mw^*-η^tL̂^t-1_mw≥ln (500n^2/c-1)≥ln (n^2/2c)+6.

ii) Suppose η^tL̂^t-1_mw^*-η^tL̂^t-1_mw≥ln(n^2/2c) ∀ m,w. Then, we can write

    X^t_mw^*/X^t_mw=exp(η^tL̂^t-1_mw^*-η^tL̂^t-1_mw)≥n^2/2c ∀ m,w≠ w^*,

which implies X^t_mw^*≥1/1+2c/n≥ 1-2c/n. Thus, X^t_m-X^*_m_1=2(1-X^t_mw^*)≤4c/n ∀ m, and hence X^t-X^*_1≤ c.

iii) Suppose η^tA^t-1=b+β, where b=ln(n^2/2c)+3 and β≥ 0. Then, we have

    η^t+1A^t   =η^t+1A^t-1+η^t+1(v_mw^*(X^t)-v_mw(X^t))   ≥η^t+1A^t-1+η^t+1 c   = η^t+1/η^t(b+β)+η^t+1 c   =b+η^t+1/η^tβ+(η^t+1 c-(1-η^t+1/η^t)b)   ≥ b+η^t+1(c-(1/η^t+1-1/η^t)b)   ≥ b,
 
where the last inequality holds by the fact that 1/η^t+1-1/η^t≤c/b ∀ t≥ t_0.




 §.§ Proof of Theorem <ref>



Using identical arguments as in the proof of Theorem <ref>, with probability at least 1-δ,

    |η^tL̂^t-1_mw'-η^tL̂^t-1_mw-η^tA^t-1|≤ 3,  ∀ t≥ 1, ∀ m, w w',

where we recall that A^t-1= ∑_τ=1^t-1(v_mw'(X^τ)-v_mw(X^τ)). Now let Ω be the event such that X^τ→ X^*, and assume X^*_mw>0 for some m,w. Conditioned on the event Ω, we have

    η^tA^t-1≤η^tL̂^t-1_mw'-η^tL̂^t-1_mw+3=ln(X^t_mw'/X^t_mw)+3,  ∀ t≥ 0.

For any w'≠ w, by taking limit from the above relation as t→∞, we get
lim sup_t→∞η^tA^t-1≤ln(X^*_mw/X^*_mw)+3≤ O(1).[Here, if X^*_mw'=0, by convention we define ln(X^*_mw'/X^*_mw)=-∞.] Now let κ:=v_mw'(X^*)-v_mw(X^*). We claim κ≤ 0. Otherwise, if κ>0, then for any sufficiently large t>t̅ we must have A^t-1>tκ/2. Given the choice of stepsize, we obtain lim_t→∞η^t A^t-1=lim_t→∞tη^tκ/2=∞, which contradicts the fact that lim sup_t→∞η^tA^t-1≤ O(1). Thus, κ≤ 0, which shows that if X^*_mw>0, then v_mw(X^*)=max_w'v_mw'(X^*), i.e., X^* is a NE.          




 §.§ An Alternative Form for the Potential Function in Lemma <ref>

One can represent the potential function in Lemma <ref> directly for womens' ordinal preferences. Given a mixed strategy profile x, let us associate a vector p_w(x)∈ℝ_+^|M| to each woman w whose coordinates are sorted in the order of w's preferences, such that if m_1>_wm_2>_w…>_w m_n, we let the jth coordinate of p_w(x) be given by 
    p_jw(x)=x_m_jw∏_k>_w m_j(1-x_kw),
 which is the probability that woman w is matched to her jth favorite man, assuming that the players propose to women independently and according to their mixed strategies x. We note that for a pure-strategy profile x∈{0,1}^|M||W|, p_w(x) would be a binary vector with all entries being zero except the jth one being one, where j is the index of the highest ranked man proposing to w. Therefore, Lemma <ref> asserts that after each best response move, the vectors {p_w(x), w∈ W} increase lexicographically with at least one of the vectors increases strictly. Now, if we again encode woman w's preferences in a cardinal vector λ_w=(λ_wm_1,…,λ_wm_n), then for a pure strategy profile x∈{0,1}^|M||W|, we have 
    r_w(x)=⟨ p_w(x),λ_w ⟩=∑_mλ_wmx_mw∏_k>_w m(1-x_kw),
 which coincides with the definition of r_w(x) in Theorem <ref> that picks the reward of the highest-ranked man proposing to w under x. Therefore, in the space of cardinal preferences and for any pure-strategy profile x∈{0,1}^|M||W|, one can rewrite the potential function in Theorem <ref> in a compact as 

    Φ(x)=∑_w r_w(x)=∑_m,wλ_wmx_mw∏_k>_w m(1-x_kw),

whose value increases after every best response update by a player. We note the similarity between the potential function (<ref>) and sum of players' payoffs ∑_mu_m(x)=∑_m,wμ_mwx_mw∏_k>_w m(1-x_kw), where one is written using mens' preferences {μ_mw} and the other using the womens' preferences {λ_wm}.



 §.§ A Gap Estimation Oracle for Unknown Δ.


Motivated by the Upper Confidence Bound (UCB) for stochastic bandits <cit.>, here we provide a gap estimation oracle, which, in combination with Algorithm <ref>, allows each player to adaptively adjust his mixing parameter based on the estimated gap he has obtained so far. The integration of this oracle with Algorithm <ref> allows the algorithm to be more practical to the situation when a lower bound for the gap Δ is unknown. In particular, using standard concentration bounds for the UCB algorithm, one can show that the event that the estimated gap is larger than the true gap decays fast as time increases, hence only introducing a small error term to the overall performance of the algorithm.     

 








 Paper Title: Learning in Multi-Objective Public Goods Games with Non-Linear Utilities.










1987 
















A]Nicole Orzan0000-0002-9204-0688Corresponding Author. Email: n.orzan@rug.nl
B]Erman Acar0000-0001-7541-2999
A,B]Davide Grossi0000-0002-9709-030X 
C]Patrick Mannion0000-0002-7951-878X
D,E]Roxana Rădulescu0000-0003-1446-5514 

[A]University of Groningen
[B]University of Amsterdam
[C]University of Galway
[D]Utrecht University
[E]Vrije Universiteit Brussels



§ ABSTRACT 
Addressing the question of how to achieve optimal decision-making under risk and uncertainty is crucial for enhancing the capabilities of artificial agents that collaborate with or support humans. In this work, we address this question in the context of Public Goods Games. We study learning in a novel multi-objective version of the Public Goods Game where agents have different risk preferences, by means of multi-objective reinforcement learning. We introduce a parametric non-linear utility function to model risk preferences at the level of individual agents, over the collective and individual reward components of the game. We study the interplay between such preference modelling and environmental uncertainty on the incentive alignment level in the game. We demonstrate how different combinations of individual preferences and environmental uncertainties sustain the emergence of cooperative patterns in non-cooperative environments (i.e., where competitive strategies are dominant), while others sustain competitive patterns in cooperative environments (i.e., where cooperative strategies are dominant).








§ INTRODUCTION


How can cooperation emerge and sustain itself in situations where agents do not necessarily have a direct motive for cooperation? This is one of the fundamental questions in various research areas, such as evolutionary biology <cit.>, political sciences <cit.>, cognitive sciences <cit.> and physics <cit.>. To answer this question, researchers developed and studied models of real-world scenarios involving tension between the collective and personal motives, called social dilemmas <cit.>. The main characteristic of these social dilemmas is that players are better off defecting at the individual level, while, at the group level, the best outcome is mutual cooperation.

In this work, we focus on a specific class of social dilemmas known as Public Goods Games (PGG), extensively studied in literature <cit.>. A PGG describes situations where cooperation by all agents is Pareto optimal, but because of the profitability of free-riding <cit.>, rational agents fail to cooperate: 
defection by all agents is a Nash equilibrium <cit.>. We refer to this kind of game as mixed-motives, since the incentives of the agents are partially misaligned. 

In addition to incentive misalignment, other factors influencing the emergence of cooperation in many real-world scenarios include uncertainty and different individual attitudes towards risk <cit.>. 



Uncertainty can have different sources: we refer to environmental uncertainty when actors are unsure about the amount of goods they can receive from the environment <cit.>, and to social uncertainty when it comes to ambiguity about the opponents' possible actions <cit.>. Individual preferences express a personal inclination towards one choice over another. In the specific context of PGGs, we are interested in modelling two main types of individuals in the presence of uncertainty: 
those that are biased towards taking risks in the presence of uncertainty, also called risk-seeking agents, and others which are inclined not to take risks, also called risk-averse agents. 
We model these attitudes using a parametric nonlinear utility function of the reward received by individuals as the result of their investment in the collective good. 

Since we are working with non-linear utility functions in the PGG, we need to decouple our perspective from the literature on the PGGs addressing non-linear public good productions. This branch focuses on settings where the public good results from a non-linear production process <cit.>. These are called non-linear public good games and allow one to model certain real-world situations (populations of bacteria, viruses, or cooperative hunting <cit.>). In contrast, we shift our focus to an individual level and capture settings where potentially different attitudes towards risk can occur within a population.


To model risk attitudes, in our work, we explicitly decouple the collective versus the individual incentives experienced by the agents, and parameterize the collective incentive at the individual level. This choice allows us to model settings where individuals in a population can have different perceptions regarding these incentives. Furthermore, we take a multi-objective approach to the optimization of these two levels of rewards, drawing on multi-objective reinforcement learning (MORL) methods. This way we can investigate learned behaviours that emerge from individually preferred trade-offs between the cooperative and competitive objectives.

Contributions. 
We investigate learning in PGGs where agents have different risk preferences, modelled as non-linearities over the game payoffs. We study the interplay between this mechanism and environmental uncertainty from a multi-objective perspective. More specifically, our contributions are as follows:


  * We propose a novel multi-objective multi-agent (MOMA) environment based on the extended Public Goods Game (EPGG) <cit.>, called the Multi-Objective EPGG (MO-EPGG). 
This environment, next to facilitating training agents on games with different levels of incentive alignment, also allows one to explicitly model the trade-off between the individual and the cooperative components of PGGs. Moreover, it enables the decoupling of environmental and social uncertainties, allowing for the analysis of their impact, both when occurring concurrently or in isolation.


  * We propose a non-linear utility function that allows one to combine the collective and individual rewards, parameterized at the agent level. The selected shape of the utility function allows us to model risk-averse and risk-seeking agents, by operating a convex or concave transformation over the collective game reward, respectively. Moreover, it allows us to model a population of agents with varying attitudes towards risk.


  * We perform an analysis of the MO-EPGG under different multi-objective optimisation criteria. In particular, we look at the joint impact of different risk preferences and incentive alignment levels on the best responses, the Nash equilibria and the price of anarchy. 


  * We perform experiments on 
a population of independent multi-objective reinforcement learning agents trained on the MO-EPGG. We show that risk-averse utility functions strongly diminish cooperation in cases with and without uncertainty. In the presence of environmental uncertainty, risk-seeking utilities improve cooperation in environments where defection is the dominant strategy. 







§ RELATED WORK


The related literature can be grouped under two main categories: non-linear utilities in public good games and multi-objective reinforcement learning. In this section, we describe them respectively. 



 §.§ Non-Linear Utilities in Public Goods Games



Although the PGG with linear utility functions is the most known and commonly used, various models of non-linear PGGs have been proposed in the literature as well. In the threshold public goods game, the resulting public good is given by a step function of the number of cooperators: the resource is created only if a minimum fraction of actors participate in the production of the public good <cit.>. When the minimum number of participants is 1, this is called the Volunteer's Dilemma <cit.>. A sigmoid public goods function closely models many biological systems where the output production is small for low input levels and bigger for intermediate inputs, decreasing again for even bigger ones <cit.>. In other paradigms, public good production is modelled by applying a concave (convex) function over agents' contributions, where the produced good is lesser (greater) than the good provided by a linear function of the contributions. 


Several papers focused on analyzing non-linear public good games, by different means. In <cit.> authors employ non-linear PGG with different incentive structures to analyze behavioural subtyping, i.e., if cooperative behaviour in one task can predict cooperative behaviour in another. In <cit.>, evolutionary dynamics techniques are employed to study the role of different non-linear production functions on the evolution of cooperation in finite populations, while in <cit.>, 
the evolutionary dynamics of two different populations collaborating for the production of a non-linear public good is investigated. In <cit.> authors explore the effects of different non-linear PGGs on the evolution of cooperation using Darwinian dynamics.

In the aforementioned literature, non-linearities in PGGs are typically functions that influence the production of the public good. In our work, however, we take a different perspective by introducing non-linearities at the level of the individual utilities extracted from 
rewards. More specifically, our goal is to model individuals' attitudes towards risk. 
In doing so, we follow decision theory which seeks to understand human decision processes and derive optimal decision-making strategies <cit.>, therefore the study of risk and uncertainty has been a central focus. Some studies have shown that people make decisions based on some subjective function of the investment they made <cit.>. For instance, an individual's risk attitude is often described as a function of the investment made (x) by means of a utility function shaped as u(x) = x^β. Here, the parameter β governs the risk preference of the individual: if 0<β<1, the function is concave, signifying risk-aversion; if β>1, the function is convex, indicating a risk-seeking attitude <cit.>. In our work, we draw on this idea to formulate a utility function that allows us to model individual preferences for actors participating in the PGG.



 §.§ Multi-Objective Reinforcement Learning


In the field of reinforcement learning, the main focus is often to solve single-objective problems, by determining the agent's best policy to reach a specific goal. However, real-world challenges are of a multi-objective nature most of the time <cit.>. Autonomous agents, whether human or artificial, need to optimize for multiple goals simultaneously, or find a trade-off between them. This is the central concept of multi-objective reinforcement learning (MORL) <cit.>, a field that
has developed rapidly in recent years <cit.>. In MORL, the core idea is to receive vector rewards from the environment instead of scalar rewards. Under the utility-based perspective <cit.>, rewards can be combined by means of a scalarization function to determine the final optimisation goal.

Often, a linear scalarization function is employed, which allows the employment of single-objective RL methods. Alternatively, other choices include monotonically increasing non-linear scalarization functions <cit.>. These are of particular interest for our work since non-linear functions are often used to model utilities under uncertainty and risk, especially in the economics literature, which aims at modelling human behaviour 
<cit.>.


Another part of this field of research focuses on fairness, i.e., how to optimize the trade-off among the objectives of different individuals under particular fairness constraints <cit.>. For example, in <cit.>, authors employ deep RL techniques to learn a policy that treats users equitably. We build on the framework developed in <cit.>, but rather than focusing on the fair treatment of a set of users, we investigate the effect of uncertainty and
individuals' attitudes towards risk. To this end, we extend their approach to work with a different scalarisation function customized for our scenario, which allows us to model individual preferences, and train independent reinforcement learning agents in a multi-objective setting. We thus adopt a multi-objective multi-agent reinforcement learning (MOMARL) <cit.> perspective, which extends MORL to multi-agent scenarios. 



§ PRELIMINARIES


In this section, we present the formal definitions and the background knowledge. These include the Extended Public Goods Game, multi-objective stochastic games and the multi-objective optimization criteria.


 §.§ The Extended Public Goods Game




The Extended Public Goods Games <cit.> is a tuple ⟨ N, c, A, f, u⟩, where N is the set of players whose size is denoted as |N| = n ∈ℕ. Every player i is endowed with some amount of wealth (or coins) c_i ∈ℝ_≥ 0, and c = (c_1, …, c_n) denotes the tuple containing all agents' coins. Each agent can decide whether to invest in the public good (cooperate) or keep the endowment for themselves (defect); therefore, the set A of allowed actions consists of cooperate (C) and defect (D) i.e., A = {C, D}. The vector a = (a_1, … , a_n) ∈ A^n represents the action profile of the agents. The quantity f is called multiplication factor, and specifies the scalar by which the total investment is multiplied in order to produce the public good. The resulting quantity is then evenly distributed among all agents. The difference with respect to the original PGG lies in the interval of allowed values for f. While in the PGG f ∈ (1,n), in the EPGG we take f ∈ℝ_≥ 0. The reward function for each agent i is defined as r_i: A^n×ℝ_≥ 0×ℝ_≥ 0^n 
→ℝ, with:

    r_i(a, f, c) = 1/n∑_j=1^n c_j I(a_j) · f + c_i (1 - I(a_i)),

where a_j is the j-th entry of the action profile a and I(a_j) is the indicator function, equal to 1 if the action of the agent j is cooperative, and 0 otherwise, and c_j denotes the j-the entry of c. For the sake of simplicity, in the following, we assume all endowments to be equal, namely c_i = c,  ∀ i ∈ N. 


Depending on the value of f, the EPGG can model three types of scenarios. When 1 < f < n, like in the classic PGG, we model mixed-motives scenarios, in which all agents playing defect is a dominant strategy equilibrium. Yet, this profile is Pareto dominated by the profile in which all agents cooperate. 

When 0 ≤ f ≤ 1, playing defect is a Pareto optimal dominant strategy (and therefore Nash) equilibrium.
In addition, the EPGG can also model fully cooperative scenarios (i.e., when f ≥ n) in which the cooperation profile is a Pareto optimal dominant strategy (and therefore Nash) equilibrium.



 §.§ Multi-Objective Stochastic Games



We model the multi-objective multi-agent interactions using the multi-objective stochastic game (MOSG) framework, defined as


the tuple ℳ = (S, 𝒜, T, γ, ℛ), with n≥2 agents and d≥2 objectives, where:


  * S is the state space,

  * 𝒜 = A_1 ×…× A_n is the set of joint actions, with A_i being the action set of agent i,

  * T  S ×𝒜× S →[ 0, 1 ] is the probabilistic transition function,

  * γ is the discount factor,

  * ℛ=𝐑_1 ×…×𝐑_n are the reward functions, 
where 𝐑_i  S ×𝒜× S →ℝ^d 
is the vectorial reward function of agent i for each of the d objectives.[We note that in this article the terms reward and payoff are synonyms. For the sake of clarity and consistency, we stick to the former term which aligns with the reinforcement learning terminology.]
 

We take a utility-based perspective <cit.> for multi-objective decision making, assuming that each agent i has a utility function u_i: ℝ^d →ℝ that maps the received reward vector to a scalar value, determining the desired trade-off between the objectives.



 §.§ Optimization Criteria



In reinforcement learning, the goal of an agent is to find a policy π that maximizes the expected scalar return V^π = 𝔼_π [∑_t=0^∞γ^tr_t]. 
In MORL, depending on how agents derive their utility, there are two optimisation criteria one can employ in the scalarisation process when maximising the expected discounted long-term reward vector: 

    
  * The Scalarised Expected Return (SER) criterion:

    V^π_u = u  ( 𝔼_π [∑_t=0^∞γ^tr_t   ]  ),

where π: S × A → [0,1] is the agent's policy, and r_t = 𝐑(s_t, a_t, s_t+1) is the vectorial reward at timestep t.

  * The Expected Scalarised Return (ESR) criterion:

    V^π_u = 𝔼_π [ u ( ∑_t=0^∞γ^tr_t  )    ]


Which one of these criteria to choose depends on the problem at hand <cit.>. If we care about the goodness of a single policy execution, ESR is the correct criterion. 
If instead, we are interested in the quality of average policy executions, we should use SER. 

In this work, we opt for the SER criterion in the learning, modelling agents that are interested in optimising their behaviour in repeated interaction settings. 



§ MULTI-OBJECTIVE EPGG



We formulate a multi-objective version of the EPGG, called Multi-Objective Extended Public Goods Game (MO-EPGG), by employing the framework of multi-objective stochastic games, outlined in Section <ref>. In our framework, the state space consists of the value of the multiplication factor f of the game currently being played, and the action space coincides with that of the single-objective EPGG (Section <ref>). We notice that the transition function for this framework is simply a random sampling from the set of possible multiplication factors at the beginning of each episode and deterministically returns that same f value at all the subsequent steps of the episode.

To complete our multi-objective formulation of the EPGG, we need to vectorize the scalar reward signal obtained by agents in the EPGG. This process is called multi-objectivization of single-objective problems <cit.>.
By observing the form of the reward function in Equation <ref>, we can easily distinguish between the part that defines the collective (r^C) and the individual payoff (r^I):

    r^C_i(a, f, c)    = 1/n∑_j=1^n c_j I(a_j) · f 
    
       r^I_i(a, c)    =c_i (1 - I(a_i)).


Then, in the proposed MO-EPGG, the vectorial reward received by agent i, given action profile a, current multiplication factor f, and a tuple of endowments c, is as follows:


    r_i(a, f, c) = ( r^C_i(a, f, c), r^I_i(a, c) ).


This completes our description of the MO-EPGG as a MOSG with d=2 objectives. In Figure <ref> we display an example of the vectorial rewards received by N=2 agents playing the MO-EPGG for three different values of the multiplication factor f.

To define the agents' utility functions, we follow a similar approach to the incentive structure proposed by <cit.>, but note that we employ the non-linear function at an individual level, to model agents' risk attitudes as preferences over the received vectorial payoff. 

In particular, in our model, the gain in utility obtained from the collective reward behaves non-linearly by means of an exponential function where β_i serves as an exponent. Therefore, we define the following non-linear utility function that specifies the final scalarised utility for the MO-EPGG agents:



    u_i(a, f, c) = w^C_i 𝔼_a[ r^C_i(a, f, c) ^β_i] + w^I_i  𝔼_a[r^I_i(a, c)],









    u_i(_i) = (^C_i )^β_i + ^I_i,

with β_i being a hyperparameter. In our setting, ^C and ^I represent expected returns (or expected discounted sums of rewards) i.e., = ∑_tγ^t r_t. Note that we are employing expected returns rather than rewards since we are working under the SER criterion. 
In this equation, the parameter β_i governs the risk-seeking/averse behaviour of agent i towards the collective expected return ^C_i, namely a value β=1 returns a linear utility function, while β<1 generates a concave function (with beta>0) which models a risk-avoiding agent, while β>1 generates a convex function which models a risk-seeking agent <cit.>. 

In Equation <ref>, the exponent is only applied over the collective reward component. This choice is motivated by our conceptualization of the collective reward as the result of a risky investment. The result depends on the value of the multiplication factor f, which might not be known with certainty; and the actions of the other players.






 §.§ Game Analysis



We analyse the MO-EPGG with the proposed utility function. We first analyse the MO-EPGG under ESR and SER, to examine 
the dynamics of best responses for different values of the game and utility function parameters. Second, we investigate the impact of different 
values of β and f on the set of Nash equilibria under SER.



  
ESR. From Equation <ref>, we can observe that the preference between the cooperative or defective behaviour in the MO-EPGG depends on the relationship between three values, namely, f, c and β. In particular, assuming a uniform value of β among the whole population of agents (β_i = β for all i ∈ N),

the collective cooperative action 
(a_C = (C, …, C))

is preferred over the collective defective action 
(a_D = (D, …, D)) 

by all the agents when r^C(a_C, f, c)^β > r^I_i(a_D, f, c), which is the case when (cf)^β > c.
This relationship between the variables induces a shared preference over collective cooperative behaviour in otherwise defective scenarios (the cases when f<1). In general, collective cooperation is preferred over collective defection whenever either of the following conditions holds:

    β   < log(c)/log(cf)if 0<cf<1 
    β    > log(c)/log(cf)if cf>1.



In the same way, collective defection is preferred over collective cooperation whenever (cf)^β < c.



  
SER.
From this point onwards, we focus on analysing a 2-player MO-EPGG, under the SER criterion. We determine the minimum cooperation level of the opponent for which the player's best response is to cooperate. We compute this value for different values of f and β. 
The results are displayed in Table <ref>. We note that the minimum cooperation level of the opponent should be strictly greater than the value presented in the table, for the best response to be cooperation. 

We can observe that, for the game with competitive incentive alignment (f=0.5), the best response for each player is to defect every time the value of β≤ 2 (i.e., the opponent's probability to cooperate cannot be >1, hence the condition is unattainable). If β>2 the best response is cooperation whenever the strategy of the opponent is to cooperate with a probability bigger than the value presented in the table. For example, for β=3, the best response is cooperation whenever the strategy of the opponent is to cooperate with a probability bigger than 0.6. For β=4, the threshold moves to 0.4, and to 0.3 for for β=5. For both the games with f=1.0 and the game with a mixed-motive incentive alignment, f=1.5, the best response is to defect every time β≤ 2, and to cooperate otherwise. For a game with cooperative incentive alignment, the best response is to cooperate every time β≥ 1. 






  
NE under SER. To complement the above results, we perform an analysis of the Nash equilibria for the MO-EPGG played by two agents under SER. Similar to <cit.>, we sweep through the space of possible joint strategies and identify the Nash equilibria[To check whether a joint strategy is an equilibrium under SER, we use the iterated_best_response and verify_nash methods of the Ramo library <cit.>.] for a set of games. We select values of f in {0.5, 1.0, 1.5, 2.0, 2.5, 3.0}, to cover a spectrum of competitive, mixed-motive, and cooperative games. Additionally, we let the value of β span over the interval [0., 3.], to capture risk-averse, risk-neutral and risk-seeking tendencies. The results are presented in Figure <ref>, where we display, for one agent, the value of the probability of cooperation in the Nash equilibria strategy as a function of β. We provide the plot for one agent only, since the corresponding plot for the other agent is identical.

From Figure <ref> we can observe, for all the games, the coexistence of two Nash equilibria whenever β<1. At each of these equilibria, the joint strategy of the two agents displays the following symmetry: if one agent defects, they both are better off when the other agent slightly moves away from full defection. For f<2 and β>1 we observe the existence of an interval of β values in which both mutual defection and cooperation coexist as Nash equilibria. Based on the results in Table <ref>, we notice that for a high enough value of β, even for f=0.5, the mutual defection Nash equilibrium will eventually disappear.
Whenever β>1 and f>2, as expected, the only Nash equilibrium is mutual cooperation. We present additional results on the SER in the Supplementary Material, in Section <ref>.



 §.§ Price of Anarchy


To evaluate the 
goodness of the possible outcomes of the system 
against the Nash equilibria of each game in the MO-EPGG (defined by a specific f value), we adapt the metric known as Price of Anarchy (PoA) <cit.>. The PoA is, the ratio between the welfare of the system in its “best solution”, i.e. social optimum, and the welfare of the system at its worst NE. Thus, it expresses the potential degradation factor of the social optimum. We highlight that higher values of the PoA indicate the existence of possible worst outcomes for the system due to agents' selfishness, in comparison to the social optimum.

On the other hand, a PoA with a value of 1 indicates an overlap between the social optimum and the worst-case selfish action. To define the social optimum in our setting, we employ the utilitarian welfare function, summing the outcomes of the utility functions of the players[In the context of multi-objective games under SER, we cannot apply the welfare function directly on the payoffs of the matrix game.]: W(π) = ∑_i=0^n u_i(V_i^π), where V_i^π is the expected vectorial return of player i, under the joint strategy π. Then, the PoA is defined as follows:

    PoA = max_s W(π) /min_π∈Nash W(π) .








In Figure <ref> we display the values of the PoA for a 2-player MO-EPGG, varying the values of f and β. 
From the figure, we can observe that for every f, the value of the PoA is 1 for β<0.5. This is due to mutual defection being the only Nash equilibrium and best welfare point. 
We can also observe that every f displays a range of β for which the value of the PoA is greater than 1. This is due to the presence of a Nash equilibria in mutual defection when the welfare is instead maximized in mutual cooperation.
We can also observe that the PoA value stabilizes to 1 for every game when β overcomes a certain threshold, which is dependent on the value of f. This is due to mutual cooperation being both the strategy that maximizes the welfare and the only Nash equilibrium of the game. 



§ EXPERIMENTAL SETUP
 





  
MO-DQN We train independent RL agents by adapting the multi-objective version of the Deep Q-network (DQN) algorithm <cit.> described in <cit.>, which allows us to optimize policies under the SER criterion
[We stress that the need for function approximation is due to the continuous input values provided by f.]<cit.> train a DQN to predict a Q-function for every objective. Therefore, the dimension of the output is |A| × d. We adjust their approach to work with our scalarization function (Equation <ref>). The loss function for MO-DQN can be expressed as follows:

    L(θ) = 𝔼_s, a, s^', r∼ D[ (r + γ Q̂_θ^'(s^', a^*) - Q̂_θ(s, a) )^2 ],


where θ and θ^' represent the DQN weights at two different timesteps of the training. D represents the buffer of stored transitions, and r is the vector reward. We find the best action a^* by applying the SER optimization criterion, namely, by applying our custom scalarization function u to update the MO-DQN function:[Similar to <cit.>, we compute the expectation of the scalarisation, which is a lower bound for the SER criterion.]

    a^* = argmax_a ∈ A u ( 𝔼[ r + γ Q̂_θ^'(s^', a^')]  ).




  
MO-REINFORCE As an alternative, we can optimize the policy gradient objective under the SER optimization criteria. Here as Roxana was commenting, since we are expanding from the SER optimization criteria based on the value function, I am uncertain if what we need to do it to apply the utility function after the expectation over the probability of trajectories. I would claim that this is the case. We can directly use the result from <cit.> (Appendix A.1) or his thesis <cit.> on deriving the SER in PG (MO-AC algo) and not waste space deriving all this.

The objective function for the scalar reward objective used in policy gradient methods is:


    J(θ)    = 𝔼_τ∼ p_θ(τ) [ G(τ) ],


where G(τ) = ∑_t=0^T γ^t r_t(s_t, a_t), where p_θ(τ) is the probability of encountering trajectory τ when employing the policy π(a|s,θ). From the policy gradient theorem, gradient of the objective function, employed to update the network weights is as follows:


    ∇_θ  J(θ)    = 𝔼_τ∼ p_θ(τ) [ ∇_θlog p_θ(τ) G(τ) ]


In the multi-objective case, under the SER optimization criterion, this becomes:


    ∇_θ  J(θ)    = u  ( 𝔼_τ∼ p_θ(τ) [ ∇_θlog p_θ(τ) G(τ) ]  )


which, for any differentiable policy becomes:


    ∇_θ  J(θ)    = 𝔼_τ∼ p_θ(τ) [  ( ∑_t=1^T∇_θlogπ(a_t|s_t;θ)  )  ( ∑_t=1^Tr(s_t, a_t)  )  ]




 §.§ Experiments


The experiments are run over a pool of N=20 agents. At each iteration t of the learning process, a multiplication factor f_t is sampled uniformly from the interval [f_min, f_max], where f_min and f_max are chosen such as to include cooperative, competitive, and mixed-motive games. 
Afterwards, a subset with M=4 active agents is randomly sampled from the pool of N agents, to participate in the game for 10 consecutive rounds. After these interactions, the MO-DQN networks are updated. Given M=4, we picked f_min = 0.5 and f_max = 6.5, to enable agents to engage in competitive, mixed-motive and cooperative games. This enables sampling from a set that contains competitive (f<1), mixed-motive (1<f<M), and cooperative games (f>M). 
Each agent receives as observation the current value of the multiplication factor—which can be observed with uncertainty—together with the previous actions taken by each opponent at the previous time step: o^i_t = (f^i_obs, a^-i_t-1), where a^-i = (a^j)_j ∈ M, j ≠ i. Therefore, each agent learns a policy π_i: 𝒪_i × A_i → [0,1], where O_i is the set of all possible observations of agent i.

We model uncertainty over the observation of the multiplication factor as Gaussian noise over the value of f, received from the environment: f^i_obs = f + 𝒩(0,σ_i^2), where σ_i is the uncertainty experienced by agent i. To ensure that the value of the observed multiplication factor coheres with the set of allowed values of f in the MO-EPGG, we round up every negative sampled value to 0.

All the experiments are run for 20000 epochs, and results are averaged over 20 runs for every condition. The RMSprop learning rate is set to λ=0.001, and γ=0.99. The action selection mechanism is ϵ-greedy, with ϵ=0.01. The values of the weights are w^C = w^I = 1 for all the agents. All the plots show the values of the average cooperation of the active agents at every evaluation step of the learning process. The DQN networks are composed of 2 hidden layers, and ReLU nonlinearities are employed between layers.[All the parameters employed to perform the experiments are described in the Supplementary Material, in Table <ref>.]





§ RESULTS



We group our empirical results in two categories, namely homogeneous preferences, when the value of β is identical for every agent, and heterogeneous preferences, where each agent i is characterised by a different β_i value. Both categories include experiments with and without uncertainty on the observation of the multiplication factor f.






 §.§ Learning with homogeneous preferences

We first explore the impact of different values of β on the scenarios with and without uncertainty on the observations. We performed experiments for different values of β, that define a linear (β=1), a convex (β>1) and a concave (β<1) utility function. In each of these experiments, β values are identical for every agent. The results for these experiments are depicted in Figure <ref>.

The experiments with β=1 represent the baseline where agents are playing the linear version of the MO-EPGG. Therefore, in the games without uncertainty, we observe as expected convergence to cooperation whenever f>M, convergence to defection whenever f<1, and a certain percentage of cooperation when 1<f<M, depending on whether the value of f is closer to a cooperative or a competitive one. When uncertainty is introduced, cooperation is increased in the competitive and mixed-motive scenarios. This results from the concurrent learning on a set of games with different levels of incentive alignment, as previously observed in <cit.>.

The experiments with β = 0.5 represent a system of risk-avoiding agents playing the MO-EPGG. This risk preference strongly pushes the system's behaviour toward competition across all games. The result stays consistent across the scenarios with and without uncertainty. This result is consistent with the analytical findings under SER outlined in Section <ref>: for all the games with different f values, (ϵ-)collective defection strategies are the only the Nash equilibria. 

The experiments with β≥ 2 induce a system of risk-seeking agents playing in the MO-EPGG. 
Here, we observe that the cooperation of the system is drastically increased in all games with respect to the baseline β=1. We note that for β=2 and f=0.5 agents do not converge to either cooperation or defection. This outcome aligns with the existence of both collective cooperation and collective defection as Nash equilibria.






 §.§ Learning with heterogeneous preferences

Secondly, we investigate the impact of learning in the MO-EPGG when the agents' preferences β_i are heterogeneous, i.e., the value of β_i for every agent i is sampled from a normal distribution centred in 1, i.e. β_i ∼𝒩(μ_β,σ_β^2)  ∀  i ∈ N, with μ_β = 1. This allows us to get more values centred around risk neutrality, and few extreme risk-averse or risk-seeking tendencies. We performed experiments with different values of σ_β, i.e., 0.5, 2 and 3. The resulting system represents a population where every individual has a different risk preference and is centred on risk-neutrality (β=1).

Figure <ref> reports the results for the scenarios without (top row) and with (bottom row) uncertainty on the observations. We can observe that when the σ_β of the distribution is small (i.e. σ_β=0.5) and no uncertainty is introduced, the competitive equilibria for f ∈{0.5, 1.5} is maintained, while the cooperative equilibria for f ∈{3.5, 6.5} is lost. We also notice that the higher the value of σ_β, the higher the average cooperation of the system in all the games. This result signals the importance of the magnitude of beta on the risk attitude: the higher β, the higher the risk appetite, which in our case translates to more cooperative behaviour. 

When uncertainty is introduced, we observe that cooperation is increased in the competitive and mixed games with respect to the cases without uncertainty. This result is consistent with previous findings on the presence of uncertainty in non-cooperative environments <cit.>. Interestingly, only the non-cooperative games are affected by the presence of uncertainty: in the cooperative games, the average cooperation of the system is equal to the one observed in the scenario without uncertainty. 



 §.§ Equilibria and Learning


We compare now our experimental results with the analysis from Section <ref>. We underline that, while the computation of equilibria is game-specific, the outcomes of the experiments result from concurrent learning on the set of environments modeled by the MO-EPGG.

Comparing the plots in Figure <ref> and the analytical results of the Nash equilibria from Figure <ref>, we can observe that, in the case without uncertainty, in all the games for which f ≠ 0.5, namely f ∈{1.5, 3.5, 6.5}, the pool of agents learns the best response, which means that the system converges to the Nash equilibrium. Only for β=1 the convergence is not perfect. In the case f=0.5, the agents converge to defection when β<2, while for β≥ 2 they cooperate with a certain probability, which is higher for higher β values. This result is not surprising given the presence of more than one Nash equilibrium. When uncertainty is introduced, as previously mentioned, the average cooperation of the system increases.



§ CONCLUSIONS AND FUTURE WORK



In this work, we introduced and analyzed a novel multi-objective variant of the Extended Public Goods Game, the MO-EPGG. This allowed us to model individual risk attitudes, by decoupling the collective and the individual payoffs. We investigated the role of different factors such as misalignment of incentives, uncertainty and individual risk preferences on cooperation, utilizing multi-objective reinforcement learning. In particular, we observed how risk-averse attitudes can increase defection in cooperative environments, and, inversely, risk-seeking ones, can increase cooperation in competitive and mixed-motive games, especially when uncertainty is introduced. Moreover, we observed how a population with heterogeneous risk attitudes, centred on risk neutrality, can fail to reach cooperation in cooperative settings. 

As future work, we plan to explore additional MORL approaches, such as policy-gradient based methods, in the MO-EPGG. Additionally, we will investigate the interplay between risk preferences, uncertainty and additional mechanisms, such as reputation mechanisms and social norms. Last but not least, we plan to apply other forms of non-linear utility functions, including exploring different dynamics between the collective and individual payoffs.


This research has been supported by the https://hybrid-intelligence-centre.nlHybrid Intelligence Center, a 10-year program funded by the Dutch Ministry of Education, Culture and Science through the Netherlands
Organisation for Scientific Research (NWO). RR was partly supported by the Research Foundation – Flanders (FWO), grant number 1286223N.










§ LEARNING IN MULTI-OBJECTIVE PUBLIC GOODS GAMES WITH NON-LINEAR UTILITIES:



§ SUPPLEMENTARY MATERIAL






§ ADDITIONAL RESULTS










In Figures <ref> and <ref> we present a heatmap of the SER values, displayed in the space of joint strategies for Player 0 and Player 1, in a MO-EPGG with N=2 players. Values of f and β are fixed for each plot. In the same figure, we also display the Nash equilibria and the joint strategies that belong to the Pareto Coverage Set (PCS), emphasizing which Nash points belong to the PCS. In multi-objective theory, the PCS is defined as the set of strategies for which the corresponding vectorial expected payoffs are not Pareto-dominated. According to the Pareto dominance relationship, one vector is preferred over another when it is at least equal on all objectives and strictly better on at least one. To obtain the PCS, we follow the same procedure as proposed by <cit.>: we step through the strategy space, using a resolution of 0.01, calculate for each joint-strategy the corresponding expected payoff vector, and then retain only the ones that are not Pareto dominated.

From the figures, we can observe how the landscape of the SER value changes when increasing the value of β. In particular, we can notice that for β=0.5 (Figures <ref> and <ref>) the SER for both players is maximized when their opponent acts cooperatively and the player acts defectively. We also note, as already observed in Figure <ref>, the presence of the two ϵ-defective Nash points. The Pareto Coverage Set, for both players, consists of all the joint strategies where the opponent cooperates. This last remark holds for all values of f and β. The same observations made for β=0.5 hold for β=1.5 (Figures <ref> and <ref>), where the only difference is the presence of a single Nash on defection.

When β>2 (Figure <ref>), we can observe that the SER landscape changes for both players and its value gets maximized in mutual cooperation. In this point, as we identified in Section <ref>, the social welfare is also maximized. Moreover, a new NE appears on the Pareto front, on mutual cooperation, for β>2.



§ HYPERPARAMETERS



Table <ref> displays the parameters employed to run the experiments illustrated in Figures <ref> and <ref>.





 Paper Title: Escape Sensing Games: Detection-vs-Evasion in Security Applications.


Securing Proof of Stake Blockchains: Leveraging Multi-Agent Reinforcement Learning for Detecting and Mitigating Malicious Nodes
    Faisal Haque Bappy^1, Kamrul Hasan^2, Md Sajidul Islam Sajid^3, Mir Mehedi Ahsan Pritom^4, and Tariqul Islam^5

^1, 5 Syracuse University, Syracuse, NY, USA

^2 Tennessee State University, Nashville, TN, USA

^3 Computer and Information Sciences, Towson University, Towson, MD, USA

^4 Computer Science, Tennessee Tech University, Cookeville, TN, USA

Email: {fbappy@syr, mhasan1@tnstate, msajid@towson, mpritom@tntech, mtislam@syr}.edu 

    August 5, 2024
==========================================================================================================================================================================================================================================================================================================================================================================================================================================================



§ ABSTRACT 
Traditional game-theoretic research for security applications primarily focuses on the allocation of external protection resources to defend targets. 
    This work puts forward the study of a new class of games centered around strategically arranging  targets to protect them against a constrained adversary, with motivations from varied domains such as peacekeeping resource transit and cybersecurity.
    Specifically, we introduce Escape Sensing Games (ESGs). 
    In ESGs, a blue player manages the order in which targets pass through a channel, while her opponent tries to capture the targets using a set of sensors that need some time to recharge after each activation. 
    We present a thorough computational study of ESGs.
    Among others, we show that it is NP-hard to compute best responses and equilibria. 
    Nevertheless, we propose a variety of effective (heuristic) algorithms whose quality we demonstrate in extensive computational experiments.




§ INTRODUCTION

The past decade has witnessed an influential line of    research in AI, particularly multiagent systems (MAS),  that  employs computational game theory to tackle critical challenges in security and public safety applications, ranging from protecting national ports <cit.> to combating  smuggling <cit.> and illegal poaching <cit.> to defending our cyber systems <cit.>. At the core of almost all of these problems is  to optimize the allocation of (often limited) external forces  to protect critical targets. 
In this work, we adopt a similar computational game theory approach but address a fundamentally different type of security challenge that looks to improve security via  optimizing the arrangement of targets  in the face of adversaries.  
 This research contributes a novel perspective to game-theoretic security strategies, emphasizing target arrangement as a  defense mechanism against adversaries. 

Specifically, we introduce and study 
Escape Sensing Games (ESGs). 
In these games, a blue player aims to securely navigate a set of  targets through a channel, whereas her opponent, the red player, controls a set of sensors along the channel and tries to sense (and therefore “steal”) as many  targets as possible.
This model captures strategic interactions   arising in various domains.   One example is the transportation of peacekeeping resources using a convoy of ships or cars over a fixed route with malicious actors (e.g., pirates  or hostile forces) trying to intercept them <cit.> (see <Ref> for details).  
In cybersecurity, the blue player could model a network administrator routing sensitive data packets through a network with an attacker trying to intercept them. 
Our model captures strategic interactions in these settings arising when security measures are either unavailable or have already been allocated and the blue player is only left with scheduling the targets to avoid detection by the attacker.

We study the optimal sequential play in  ESGs, where the blue player first commits to an ordering of targets followed by the red player devising an optimal sensing plan. Herein, sensors' capabilities are limited in two ways. First, each sensor is only capable of sensing certain targets, modeling that detection and interception technologies are not uniformly effective across different targets, due to differing characteristics such as size, speed, or defense mechanisms. Second,  sensors need a certain time to recharge after sensing a target, modeling limits inherent in detection and interception systems, where permanent action is not feasible.

There are certain challenges integral to our model that make the computation of equilibria highly non-trivial. First, the action space of both players has an exponential size, rendering standard solution approaches such as support enumeration computationally infeasible. 
Second, also after the strategies of both players have been fixed, the game evolves in a complex, sequential fashion with targets moving one after each other through the channel.  
Connected to this, third, it turns out that the red player's best response problem of coming up with an optimal sensing plan given a target ordering is already NP-hard. 
Consequently, this paper also contributes to the algorithmic research on computationally challenging games, a fairly unexplored topic outside of combinatorial game theory   <cit.>.



 §.§ Our Contribution

We contribute a new perspective to the rich literature on computational game theory for security applications through our study of the previously overlooked problem of target arrangement. 
Specifically, we introduce and analyze Escape Sensing Games with a focus on the target-controlling blue player. 
We demonstrate that solving this game is highly complex, as we prove that it is NP-hard for both players to compute their optimal strategies. 
To nevertheless be able to solve ESGs in practice, we devise algorithms for computing the red player's strategy, which turn out to scale well in our experiments. 
Computing the blue player's strategy and thereby the game's Stackelberg equilibrium turns out to be a much more intricate task. 
Our experiments show that our formulation of the problem as a bilevel program is only capable of solving small instances of the game exactly. 
Motivated by this, we present a heuristic that effectively combines simulated annealing with a greedy heuristic and an Integer Linear Program (ILP) for computing the red player's strategy. 
We demonstrate the quality of our heuristic through extensive experiments.

We further this investigation in <Ref> by 
studying a different variant where sensors are decentralized hence each sensor acts independently according to a simple greedy strategy. We show that it remains NP-hard for the blue player to compute its optimal strategy. While in this setting blue's problem admits an ILP formulation, we demonstrate in experiments that it can only solve up to medium-sized instances. 
Addressing this, we present heuristics that perform well in our experiments. We also demonstrate that while sensors usually have some gain from coordination, this gain depends decisively on the instance structure and is oftentimes rather small (below 20%). 

Full proofs of all results and descriptions of additional experiments can be found in our full version <cit.>.



  
High-level description of the game
We study a novel zero-sum game where an 



  
Motivating examples
Milind has this description. Let's maybe find some background citations for this. Would be nice if we can get data about this actually. That way we can even run experiments on it. 

We have discussed the idea of a peacekeeping resource supply convoy of ships going towards refugees. We assume there are pirates that are trying to intercept the convoy. The ships can engage in deception by masking themselves in different ways, so a medical supply ship may appear to be carrying agriculture equipment.  The pirates have different types of sensors, e.g., infrared equipped drones, ground based sensors, to sense the actual ship type. Each sensor can sense the true type of a specific type of ship. For example, the drone equipped with infrared can sense the presence of medical personnel on a medical ship and thus realize the true type of that ship.





  
Why do previous approaches fall short
Discuss why this renders classic approaches insufficient. 



  
Questions for Experiments
How much does sensor/target ordering matter?



 §.§ Our Contributions




 §.§ Related Work





§ ESCAPE SENSING GAMES (ESGS): THE MODEL



In an Escape Sensing Game (ESG)  a blue player (henceforth ) tries to route a set of targets through a channel.
They compete against a red player (henceforth ) who controls a set of sensors and tries to sense (and therefore “steal”) as many of 's targets as possible.[Note that the terms   “sensor” and “sensing” are only part of our terminology and do not limit the applications of our model. For instance, instead of “sensing” the targets,  might also aim to intercept them.]
Formally, an Escape Sensing Game is defined by

    
  * a set of targets T={t_1,…, t_n}, each target t_i∈ T equipped with some utility value v_i∈ℝ^+,  
    
  * a set of sensors S={s_1,…, s_k} and a recharging time τ∈ℕ∪{∞} which is the same for all sensors and, 
    
  * a sensing matrix D∈{0,1}^n× k where D_i,j=1 means that sensor s_j is capable of sensing target t_i.

We assume that all of these parts are known to both players at any point in time. Note that in this paper, we consider the constant-sum utility structure and leave the general-sum version for future work.
That is,  
seeks to maximize the summed value of not-sensed targets, i.e., targets not sensed by any sensor. In contrast,  seeks to minimize this value, or equivalently, maximize the summed value of targets that are sensed by some sensor.

The strategy of  is an ordering of the targets σ: T→ [n] that assigns each target t∈ T a unique position σ(t), i.e., σ is a bijection. 
The targets move through the channel according to σ, i.e., the target on position 1 moves first, on position 2 second, and so on. In each time step each target moves to the next sensor, leaves the channel (in case it passed all sensors), or enters the channel at the first sensor (in case it is the next target in the ordering σ).

The strategy of  is a sensing plan ψ: S → 2^T that maps each sensor to a subset of targets T'⊆ T sensed by the sensor, where each sensor senses different targets, i.e., ψ(s)∩ψ(s')=∅ for each s≠ s'∈ S. 
 cannot play arbitrary sensing plans but only those which are valid.
A sensing plan is valid (with respect to a senor ordering σ) 
if 
[label=(*)]
    
  * a sensor only senses targets it has the capabilities to sense, i.e., for each s_j∈ S and t_i∈ψ(s_j) we have D_i,j=1, and
    
  * a sensor pauses for at least τ time steps after sensing a target, i.e., for each s∈ S and t,t'∈ψ(s) we have |σ(t)-σ(t')|>τ.

Given a sensing plan, we can immediately calculate the value of not-sensed targets as v(ψ):=∑_t_i∈ T∖∪_s∈ Sψ(s) v_i,  which quantifies 's utility.



  
Objectives and equilibrium Due to the motivating applications of our interest,  this work adopts 's perspective and analyzes sequential play in this game by assuming  moves first.[Note that this is already reflected in our game definition, since the validity of 's sensing plan depends on the strategy of .  Thus,  cannot move before or simultaneous to .]
Our analysis consists of two parts. 
First, we will analyze the best response problem for  called Best Red Response: Given an ordering σ of the targets, output the sensing plan ψ that is valid with respect to σ and minimizes v(ψ) among such plans.
Second, to compute the optimal strategy of , we analyze the game's Stackelberg equilibrium[We assume that ties in the strategies are broken according to some predefined lexicographic ordering of the strategies.], which can be written as the following bilevel optimization problem: max_σmin_ψ: ψ is valid wrt. σ v(ψ).
We term the corresponding computational problem Blue Leader Stackelberg Equilibrium.

We analyze sequential play in Escape Sensing Games.
Depending on who moves first, i.e., which player is the follower and which one is the leader in the resulting Stackelberg game, two different variants emerge. 

We start by discussing the variant where  moves first. 



  
 Leads. Maybe directly move this to Section 4? This preliminary section only introduces the game parameters of the Escape Sensing game? Just a thought – I am not sure whether this is a good idea or not.  The strategy of  is an ordering of the targets σ: P→ [n] that assigns each target p∈ T a unique position σ(p), i.e., σ is a bijection. 
The targets move through the channel according to σ, i.e., the target on position 1 moves first, on position 2 second, and so on. 

The strategy of  is a sensing plan ψ: S → 2^P that maps each sensor to a subset of targets P'⊆ P sensed by the sensor— where each sensor senses different targets, i.e., ψ(s)∩ψ(s')=∅ for each s≠ s'∈ S. 
 cannot play arbitrary sensing plans but only those which are valid, i.e., respect sensors' sensing capabilities and recharging times.
A sensing plan is valid (with respect to a senor ordering σ)[We omit this part if the sensor ordering is clear from context.] if 

    
  * a sensor only senses targets it has the capabilities to sense, i.e., for each s_j∈ S and t_i∈ψ(s_j) we have D_i,j=1, and
    
  * a sensor pauses for at least τ time steps after sensing a target, i.e., for each s∈ S and p,p'∈ψ(s) we have |σ(p)-σ(p')|>τ.

Given a valid sensing plan played by , we can immediately calculate the value of not-sensed targets as v(ψ):=∑_t_i∈ T∖∪_s∈ Sψ(s) v_i.
Using this, we can write the game as: 

    max_σmin_ψ:
    ψ is valid wrt. σ v(ψ).


Notably, solving this equation will give us the Stackelberg equilibrium in the induced game; we term the corresponding computational problem Blue Leader Stackelberg Equilibrium.[Not even clear whether it is in NP. Conjecture to lay beyond.]
As part of our study of this game, we will analyze the best response problem for  called Best Red Response: Given an ordering σ of the targets, output the sensing plan ψ that is valid with respect σ and minimizes v(ψ) among such plans.

Maybe directly move this to Section 3?  

  
Red Player Leads. 
In case  moves first, it cannot commit to a valid sensing plan because its validity depends on the undecided strategy of . 
We study a game variant where we circumvent this problem by making  symmetric to : 
 has to commit to an ordering π: S → [k] of sensors that assign each sensor s∈ S to a unique position π(s) along the channel, i.e., π is a bijection. 
After  fixes the ordering of targets σ and when the game starts to unfold, 's sensors act greedily always sensing a target if possible. Formally, 's sensing plan is computed as follows.
targets move according to σ through the channel with sensors being arranged in the channel according to π, i.e., for i∈ [n] and j∈ [k] target σ^-1(i) passes sensor π^-1(j) in step i+j-1. 
We construct ψ sequentially in a greedy fashion: For each step ℓ∈ [n+k-1], if target t_i passes sensor s_j in step ℓ, then we add t_i to ψ(s_j) if the resulting sensing plan ψ remains valid with respect to σ.[Note that it would also be possible to consider more complicated algorithms to move from an ordering of sensors and targets to a valid sensing plan, but the greedy version described above feels to be the most natural one.]
For an ordering of sensors π and an ordering of targets σ, we let ϕ(π,σ) be the valid sensing plan computed according to the above-described procedure. 
Accordingly, the game can be written as: 

    min_πmax_σ v(ϕ(π,σ)).

Again, we will be interested in solving this game to analyze the respective Stackelberg equilibria; we term the corresponding computational problem Red Leader Stackelberg Equilibrium.
Moreover, we will study the best response problem for  called Best Blue Response: Given an ordering π of the sensor, output the ordering of targets σ that maximizes  v(ϕ(π,σ)).
 

An alternative view on the two games presented above is that in both of them,  commits to an ordering of the sensors. However, in the first setting, the sensors are allowed to coordinate with each other once the targets start moving through the channel. In contrast, in the second scenario, no coordination between the sensors is allowed, which leads to the adoption of a greedy strategy for each individual sensor. This view also motivates the study of other game variants. For instance, it is possible to flip the leader and follower roles, i.e.,  commits to an ordering of targets first followed by  committing to an ordering of sensors who then act greedily on the targets. We discuss some results for this variant in xxx. For this game variant, it would also be possible to consider simulations play; however, we show in yyy that a Nash equilibrium in pure strategies may fail to exist. 



We do not consider mixed strategies here because ...





  
A motivating application One major motivation of our work is the secure transit of peacekeeping resources in the presence of adversarial actors such as pirates, which has critical importance due to past incidents, e.g., to the United Nations <cit.>. Citing the UN's peacekeeping mission manual <cit.>, “protecting shipping in transit ensures the safety and security of vessels as they pass through waters threatened by piracy on the high seas...” In these applications, UN plays 's role whereas pirates correspond to , who can observe the ordering of targets and then act second.  The UN commands a fleet of ships (i.e., targets in our model) that often carry resources of different importance and that can be arranged strategically. Protecting shipping is overall a complex, multi-facet, task and our model captures one of the phases after potential (often scarce) security measures have already been allocated to the ships and the pirates look to identify targets to attack. According to  <cit.>,  pirates often use a set of boats (i.e., sensors in our model) to probe different passing targets, usually by following them to observe their speed, crew amount, firearm, etc. to judge based on this whether they are capable of capturing the ship. Such probing takes time, which is modeled by the recharging time τ.  



  
Sensor and target types
We develop some customized algorithms for instances with only a few different target or sensor models:
We say that two sensors s_i,s_j∈ S are of the same type if they are capable of sensing the same targets, i.e., the i-th and j-th column of the sensing matrix are identical. 
We say that two targets t_i,t_j∈ T are of the same type if they have the same utility value and can be sensed by the same sensors, i.e., v_i=v_j and the i-th and j-th row of the sensing matrix are identical. 
We denote as Γ={γ_1,…,γ_n_χ} and Θ={θ_1,…,θ_k_χ} the set of target and sensor types, respectively. 
It is easy to see that k_χ≤ 2^n_χ, as a sensor's sensing capabilities are defined by the set of target types it can sense. Similarly, assuming that all targets have the same value, it holds that n_χ≤ 2^k_χ.


§ RELATED WORK

While the escape sensing game model is new, it is closely related to a few lines of AI research, as detailed below. 



  
Computational game theory for security Conceptually, our work subscribes to the extensive MAS literature on computational game theory for tackling security challenges. The Stackelberg security game <cit.> is one widely studied example. Other game-theoretic models  include the hide-and-seek game <cit.>, blotto games <cit.>, auditting games <cit.> and catcher-evader games <cit.>. Most of these games study the optimal usage of security forces under different game structures. In contrast, our ESG model is motivated by detection-vs-evasion situations in which security forces have already been allocated.





  
Multi-agent planning In terms of modeling, our problem is related to the rich MAS literature on multiagent planning since both players' strategies are sequential and thus require optimal planning. Most of previous multi-agent planning research has focused on cooperative environments <cit.> and addresses for instance challenges on coordination <cit.>
and unknown states <cit.>.
In contrast, our problem is about multi-agent planning in non-cooperative settings, which has been significantly less studied. 



  
Scheduling On a formal level,  our problem is to schedule/order targets in an adversarial environment, which shares similarities with the classic problem of scheduling that looks to assign tasks to different machines to optimize certain criteria  <cit.>. There is a rich body of AI research on scheduling, ranging from solving varied problems using AI techniques such as satisfiability <cit.> and distributed constraint optimization <cit.>, to developing new models of scheduling problems under uncertainty <cit.> or in multi-agent setups <cit.>.

In fact, the Best Red Response problem can be formulated as the following slightly non-standard scheduling problem: There are k machines (modeling sensors). In each step, a job (modeling a target) arrives. The job can be processed (modeling sensing) by a given subset of machines and if executed successfully generates a given reward value. The job has a processing time of τ and needs to be processed within the next τ steps. This implies that the job needs to be processed (i.e., sensed) either now or its reward is lost.



§ THE ALGORITHMICS  OF ESCAPE SENSING GAMES
  
We analyze the computational complexity of ESGs starting with 's best response problem, followed by computing equilibria.



 §.§ Computing 's Best Response Strategy
 
We analyze 's best response problem that  needs to solve in each game after  has committed to a target ordering. 
This problem turns out to be NP-hard, even if  is only interested in determining whether it can sense all targets. This intractability result is the first strong indicator of the intricate game dynamics in ESGs. 
theoremredresponse
Best Red Response is NP-complete, even when asked to decide whether  can sense all targets or not.


We reduce from Hitting Set where we are given a universe U, a collection of sets 𝒵={Z_1,…, Z_m} and an integer t, and the question is whether there a size-t subset U'⊆ U containing at least one element from each set in 𝒵 (we assume that t≥ 2 and |U|>t). 
 
In the construction, all targets have a value of 1 and the question is whether  can sense all targets.
As the core of the construction we add element sensors {a_u| u∈ U}, set targets {α_Z| Z∈𝒵}, and selection targets {β_i,j| i∈ [|U|-t], j∈ [m]}. 
Each element sensor can sense all selection targets and all set targets corresponding to sets in which the element appears. 
Regarding the ordering of targets, it is easiest to think of the targets as being arranged in “rounds”. In each round j∈ [m], first the selection targets {β_i,j| i∈ [|U|-t]} move through the channel followed by the the set target α_Z_j. 
The idea is that the same |U|-t element sensors sense the selection targets in every round, which correspond to the elements that are not part of the hitting set (we extend the construction in the following paragraph to ensure that this holds). 
Then, the remaining t element sensors need to form a hitting set to be able to sense the set target in each round.

We extend the construction as follows. 
We add filling targets γ_i,j for all i∈ [t-1] and j∈ [m], which all element sensors can sense. 
Moreover, we add dummy sensors d_i,j for each i∈ [2|U|] and j∈ [m] and dummy targets δ_i,j for each i∈ [2|U|] and j∈ [m]. 
For each i∈ [2|U|] and j∈ [m], dummy sensor d_i,j can sense dummy target δ_i,j. 
We set τ:=2|U|+1. 
Formally, the target ordering σ is constructed—in multiple “rounds”—as follows. In each round j∈ [m], we first move the selection targets {β_i,j| i∈ [|U|-t]} through the channel, then the dummy targets {δ_i,j| i∈ [|U|]}, then the set target α_Z_j, then the filling targets {γ_i,j| i∈ [t-1]} and then the dummy targets {δ_i,j| i∈ [|U|+1,2|U|]} (the ordering of targets in each of the groups is arbitrary). 



  
Proof of correctness: forward direction
Assume that U'⊆ U is a size-t hitting set of 𝒵. 
For each i∈ [2|U|] and j∈ [m], we let d_i,j sense δ_i,j. 
We construct the sensing plan for the element sensors iteratively as follows. 
In each round j∈ [m], we let each of the |U|-t element sensors {a_u| u∈ U∖ U'} sense exactly one of the selection targets {β_i,j| i∈ [|U|-t]}. 
Now, let u^* be an element from U' that is contained in Z_j (such an element needs to exist because U' is a hitting set). 
We let a_u^* sense α_Z_j and we let each of the t-1 element sensors  {a_u| u∈ U' ∖{u^*}} sense exactly one of the filling targets {γ_i,j| i∈ [t-1]}. 

The constructed sensing plan senses all targets and clearly respects the sensing matrix. 
It remains to be argued that the recharging times of all element sensors are respected (dummy sensors only sense one target).
For each u∈ U ∖ U', we have that a_u senses one selection target in each round. Between two selection targets in two different rounds there are at least 2|U| dummy targets and one set target, so recharging times are respected. 
For each u∈ U', the sensor a_u senses either a set or filling target in each round. There are 2|U| dummy targets and |U|-t≥ 1 selection targets between each two sets and filling targets from different rounds, so recharging times are respected. 



  
Proof of correctness: backward direction
Assume that ψ is a valid sensing plan that senses all targets. 
Consequently, in each round, the |U| element sensors need to sense |U|-t selection, t-1 filling, and one set target. 
As ψ is valid and there are only |U|-t-1+|U|+1+t-2=2|U|-2 targets between the first selection and last filling target in each round, this means that each element sensor needs to sense exactly one of these targets in each round.
Note that an element sensor that senses a non-selection (i.e., either a set or filling) target in round j∈ [m] cannot sense a selection target in round j+1, as there are only t-1+|U|+|U|-t-1=2|U|-2 targets between the first non-selection target in round j and the last selection target in round j+1. 
Consequently, as each element sensor needs to sense one target in each round, it follows   
that there is a set U”⊆ U of |U|-t elements so that the corresponding element sensors sense a selection target in every round. 
Consequently, the remaining t element sensors need to sense all set targets. 
As an element sensor is only capable of sensing a set target if the element appears in the set, it follows that U∖ U” is a size-t hitting set of 𝒵.
We reduce from Hitting Set where we are given a universe U, a collection of sets 𝒵={Z_1,…, Z_m} and an integer t, and the question is whether there a size-t subset U'⊆ U containing at least one element from each set in 𝒵. 
We ask whether  can sense all targets.
The core of our construction consists of element sensors {a_u| u∈ U}, set targets {α_Z| Z∈𝒵}, and selection targets {β_i,j| i∈ [|U|-t], j∈ [m]}. 
Each element sensor can sense all selection targets and all set targets corresponding to sets in which the element appears. 
Regarding the ordering of targets, it is easiest to think of the targets as being arranged in multiple "rounds". In each round j∈ [m], first the selection targets {β_i,j| i∈ [|U|-t]} move through the channel followed by the the set target α_Z_j. 
The idea is that the |U|-t element sensors that sense the selection targets are the same in every round and correspond to the elements that are not part of the hitting set (we extend the construction to ensure that this holds). 
Then, the remaining t element sensors need to form a hitting set to be able to sense the set target in each round. 

Despite this intractability result, it is still possible to construct exact combinatorial algorithms for  Best Red Response. 
In particular, we present a dynamic programming-based algorithm empowered by some structural observations on ESGs that runs in 
𝒪( n· (k_χ +1)^τ+2) (recall that k_χ≤ k). 
This algorithm in particular implies that the problem becomes polynomial-time solvable if the recharging time, which we expect to be rather small in comparison to the number of targets, is a constant. 
propositionredresponseDP
There is a   𝒪( n· (k_χ +1)^τ+2)-time algorithm for Best Red Response.  


Our idea is to construct a valid sensing plan iteratively by going through the arriving targets one by one (we assume that the ordering of targets is t_1,…, t_n). For each target, we either decide that it will not be sensed or assign it to one of the sensors so that the resulting plan is still valid.  
Our key observation to bring down the time and space complexity of the dynamic program is that we do not need to store the full sensing plan to ensure the validity of the plan after updating it. Instead, it is sufficient to know for each sensor whether it has sensed a target in the last τ steps. 
More formally, given a valid sensing plan ψ that has been constructed by iterating over the first i∈ [n] targets, we only store the following information: 
[label=(*)]

  * the value of all targets {t_1,…, t_i} that have not been assigned to a sensor in ψ, i.e., ∑_t_j∈{t_1,…, t_i}∖∪_s∈ Sψ(s) v_j,

  * the sensors the last τ+1 targets have been assigned to.

It is possible to store this information in a table of size 𝒪(n· (k +1)^τ+1) where each cell can be computed in 𝒪(k)-time.  To extend the algorithm to sensor types, we prove that we can collapse sensors of one type into a “meta” sensor, making it sufficient to bookmark the types of sensors that have sensed the last τ+1 targets.

We conclude by giving a clean ILP formulation of Best Red Response, which turns out to scale very favorably in our experiments allowing us to solve instances with up to 10000 targets within one minute.

    Best Red Response  admits  an ILP formulation with  𝒪(n· k) binary variables and 𝒪(n· k) constraints. 

 
    We model an instance ℐ of Best Red Response as an ILP as follows. 
    We assume that the targets are ordered as t_1,…, t_n. 
    We create a binary variable x_i,j for each i∈ [n] and j∈ [k+1]. 
    Setting x_i,j to one corresponds to letting sensor s_j sense target t_i if j∈ [k], and letting  t_i not be sensed by any sensor if j=k+1. 

    To ensure that  minimizes the value of not-sensed targets, the optimization criterion becomes: 
    min∑_i∈ [n] v_i· x_i,k+1.
    To ensure the validity of the sensing plan ψ, for each i∈ [n], we enforce that: 
    ∑_j∈ [k+1] x_i,j=1.
    Moreover, to ensure that sensor capabilities are respected, we impose for each i∈ [n] and j∈ [k] that: 
    x_i,j≤ D_i,j.
    Lastly, to enforce that recharging times are respected, for each j∈ [k] and i∈ [n-τ] we add the constraint:
    ∑_ℓ=i^i+τ x_ℓ,j≤ 1.




  §.§.§ Greedy Heuristics for Sensing Plans
 
as a sub-algorithm in section 3.2
While we will show in our experiments that the ILP can solve medium-sized instances of the Best Red Response problem in a reasonable time, we present and test three quadratic-time greedy heuristics. 
The idea behind these greedy heuristics is to build the sensing plan ψ iteratively by trying to sense the most valuable targets first. 
We consider the targets in decreasing order of their value. Let T' be the already processed targets and t_ℓ the target to consider. Moreover, let  S'⊆ S the set of sensors s so that ψ remains a valid sensing plan after adding t_ℓ to ψ(s), i.e., the sensors that are currently free to sense t_ℓ. 
If S' is empty, then we do not assign t_ℓ to any sensor, implying that it will be won by . 
Otherwise, we apply three different methods to decide which sensor from S' to pick: 

    
  Random Randomly select a sensor from S'.
    
  Remaining Value Pick the sensor s from S' that has the lowest summed value of remaining targets that s is capable of sensing, i.e., _s_j∈ S'∑_t_i∈ T ∖ T': D_i,j=1 v_i.
    
  Harm Pick the sensor s from S' that where assigning t_ℓ does the least harm: The harm that t_ℓ does to s in ψ summed value of remaining targets that s is capable of sensing that it can no longer sense when t_ℓ is assigned to s, i.e., _s_j∈ S'∑_t_i∈ T ∖ T': D_i,j=1  and  |i-ℓ|≤τ v_i.






 §.§ Solving for the Stackelberg Equilibrium

We now study the problem of computing 's optimal strategy, i.e., to solve Blue Leader Stackelberg Equilibrium. <Ref> already shows the NP-hardness of Best Red Response. While this does not imply the hardness of computing Stackelberg equilibria[Note that the fact that it is NP-hard for  to best respond to certain  strategies (as constructed in the reduction of <Ref>) does not imply that is also hard for  to best respond to the particular Stackelberg equilibrium strategy of  (as these strategies might admit some structure that makes it easier to best respond).], a convincing intractability result for 's optimal strategy shall ideally “disentangle” its complexity from 's best response problem. With this in mind, we prove the NP-hardness of Blue Leader Stackelberg Equilibrium  even in situations where  's best response problem is linear-time solvable. 
This demonstrates that the complexity in our reduction does not come from finding 's strategy but from the problem of whether  can arrange the targets in an optimal way.  

theoremstackcomp
Blue Leader Stackelberg Equilibrium is NP-hard, even on instances where  Best Red Response is linear-time computable and the recharging time is 3. 

Note that the NP-hardness upholds even if sensors' recharging time is constant, a case in which 's best response problem is polynomial-time solvable (see <Ref>).
Our hardness result indicates that computing 's optimal strategy is a generally much harder problem than computing  's optimal strategy.
In fact, it remains open whether Blue Leader Stackelberg Equilibrium is contained in NP or whether it is complete for complexity classes beyond NP. We suspect the latter to hold.



  §.§.§ Bilevel Optimization

In light of this, it is unclear (and from our perspective rather unlikely) that Blue Leader Stackelberg Equilibrium admits an ILP formulation. 
Naive brute-force approaches are also computationally infeasible, as we would need to enumerate all n! possible target orderings and solve the NP-hard Best Red Response problem as a subroutine for each of them. 

Thus, we turn to a formulation as a bilevel optimization problem <cit.> as one way to solve the problem exactly. 
In such formulations, constraints are still linear,
but there exist two connected levels of the problem, i.e., an outer and an inner level. The inner level controls certain variables that it sets to minimize an objective subject to linear constraints that also involve variables controlled by the outer level, while the outer level sets these variables to maximize the objective. 
In our problem, we can model 's best response problem as the inner level loosely following the ILP from <Ref>. 
The outer-level models 's problem. 
The key parts of the outer level are variables for each target that encode the position in which the target appears in the final ordering and that are used in the inner level to ensure the validity of the sensing plan. 
propositionbilevel
Blue Leader Stackelberg Equilibrium admits a bilevel optimization formulation with 𝒪(n^2+n· k) binary variables, 𝒪(n) integer variables, and 𝒪(n^2· k) constraints.  

Note that standard techniques to convert this bilevel program into an (integer) linear program, e.g., by exploiting KTT-optimality conditions <cit.>, are not applicable in our setting, as we are solving an integer bilevel program within which the inner-level program is already non-convex.



  
Types


Dynamic programming 
Target types α_1,…,α_γ_T
Sensor types β_1,…,β_γ_S

Let J[i_1,…, i_γ_T, a_1,… ,a_τ+1,b_1,… ,b_τ+1] be a table entry. 
We say that a target ordering σ respects the entry if it

    
  * contains i_ℓ targets of type α_ℓ for each ℓ∈ [γ_T]
    
  * the ith last target in σ is of type a_i. 


Similarly we say that a sensing plan σ is respecting an entry and target ordering σ if  

    
  * the ith last target in σ is sensed by a sensor of type b_i. 


Each table entry J[i_1,…, i_γ_T, a_1,… ,a_τ+1,b_1,…,b_τ+1] contains a target ordering respecting the entry, a sensor plan ψ respecting the entry and σ.
We refer to these entries as  ψ(J[…]) and σ(J[…]), respectively.

A table entry is filled correctly if:

    
  * ψ is a valid sensing plan with respect to σ,
    
  * ψ is a best response to σ among all valid sensing plans respecting the entry and σ,  
    
  * σ is the target ordering respecting the entry that maximizes the minimum value of a valid sensing plan respecting the entry and σ.


The final value of the game is: 

    max_a_1,… ,a_τ+1min_b_1,… ,b_τ+1
        v(ψ(J[n_1,…, n_γ_T,a_1,… ,a_τ+1, b_1,… ,b_τ+1]))


Assuming b_1≠∅ and let a_1 be of type α_j. 
Let 𝒳 be the entry maximizing:

    max_p∈ [γ_T]min_q∈ [γ_S]   v(ψ(J[i_1,…, i_j-1, i_γ_T, a_2,… ,a_τ+1,α_p,b_2,… ,b_τ+1,β_q]))

Then we construct σ(J[𝒴]) by appending a target t of type a_1 at the end of σ(J[𝒳]). 
Moreover, we extend the sensing plan  
ψ(J[𝒴]) by letting b_1 sense t.

We prove by induction over ∑_ℓ=1^γ_T i_ℓ and assume that all entries with these sums contain the right entries. 

valid clear

Now assume that correctness condition (b) is violated, i.e., there is a sensing plan ψ valid with respect to σ(J[𝒴]) that respects 𝒴 and for which v(ψ)<v(ψ(J[𝒴])). 
Either we have used β_q here which then results in a better plan for 𝒴 or if we used something else then 
we claim that the minimum was not taken correctly. 
Consider cell 𝒴' that results from 𝒴 by replacing β_q with what is used in v(ψ). 
Then it needs to hold that 






  §.§.§ Heuristic

We will see later that the running time for the bilevel formulation of the problem becomes already infeasible on small-sized instances.
Therefore, we experimented with different heuristics to solve the problem.[Note that the heuristic double-oracle approach that has been successfully employed for other large combinatorial games <cit.> is not applicable to ESGs. Traditionally, the approach successively expands the strategy spaces of both players by letting them best respond to each other. However, in ESGs,  we face a bilevel problem in which there is no best response of the leader to the follower. The approach also fails here because the valid strategies of the follower heavily depend on the strategy picked by the leader.]
In the following, we present two variants of 
simulated annealing-based heuristics that performed best. 
For a target ordering σ, we denote as N(σ) its neighbors, i.e., all  n  2 orderings that arise from σ by swapping the position of any two different targets. 
The relaxed version of our simulated annealing (SA_Relax) is presented in <Ref>. 
The idea is to find an optimal ordering through repeated local rearrangements.
We store the current ordering as σ and compute its value for  by solving 's best response problem using <Ref>. 
Then, we pick a random neighbor of σ, compute its value, and update the ordering based on this according to standard simulated annealing rules. 





In the full version of our simulated annealing (SA), instead of picking a random neighbor σ̂ from N(σ) in Line <ref> of Algorithm <ref>, we first run a heuristic for Best Red Response
on all orderings from N(σ).[In our (greedy) heuristic, we consider the targets in decreasing order of their value and construct the sensing plan ψ iteratively. Let S'⊆ S be the sensors s so that ψ remains a valid sensing plan after adding the current target to ψ(s). We let the target be sensed by a randomly selected sensor from S (or by no sensor if S is empty). For a formal description, see our full version <cit.>.]
Then, on the μ fraction of neighbors with the highest returned value, we execute the ILP from <Ref> to compute the optimal sensing plan. Of the examined neighbors, we pick the one with the highest returned value as σ'. As a hyper-parameter tuning process, we tested the performance of our heuristic algorithm with respect to the choice of μ (see our full version <cit.> for results). 
It turns out that μ = 0.1 provides a good trade-off between the algorithm's running time and 's utility. Thus, we fix μ=0.1 throughout the paper.
For both heuristics, we always run the heuristic three times with three different initial randomly generated target orderings and return the best computed ordering.  



§ EXPERIMENTAL EVALUATIONS


We analyze the quality and performance of our algorithms to compute the Stackelberg equilibrium.[We use Gurobi <cit.> to solve the ILP from <Ref> and MIBS <cit.> to solve the bilevel program from <Ref>. Both are among the most popular off-the-shelf tools for solving the respective problem.] 
We consider three simulated game settings for generating ESGs. For each setting, we
determine the value of a target by drawing a number uniformly within [0, 1][In our full version <cit.>, we analyze  supplementary scenarios, reinforcing similar conclusions to those presented here.]: 


    
  * Default (Def): For each i∈ [n] and j∈ [k], we set D_i,j =1 with probability 0.2.
    
  * Euclidean (Euc):  
    Each target t_i ∈ T and each sensor s_j ∈ S are uniformly sampled points in [0,1] × [0,1]. A sensor can sense a target (i.e., D_i,j = 1) if the Euclidean distance between their points is below 0.3.
    
  * RandomLevel (Rand):  
    Each target t_i∈ T has a difficulty level d_i uniformly sampled from [0,1], and each sensor s_j∈ S has a skill level s_j uniformly sampled from [0,1].
    For each i∈ [n] and j∈ [k], we set D_i,j =1 with probability (1-d_i)· s_j.

In all our experiments, if not stated otherwise, we average over 50 instances generated according to one of the models. 
We present our experimental results as tables where each entry contains  's average utility (i.e., the summed value of not-sensed targets) from the computed target ordering assuming  best responds and the average running time in seconds in italics, both followed by their respective standard deviations.
Note that standard deviations are calculated across the different sampled instances, implying that independent of the solution method some non-trivial standard deviation is to be expected, as certain instances are more favorable for  than others. 

We analyze the maximum size of instances that we can solve exactly using the bilevel program, which we denote as OPT. 
We present results for the Default game setting in <Ref> (results for other simulated game settings are similar).
It turns out that while instances with 5 targets can be solved within a second by OPT, instances with 9 targets take already around 9 hours to solve. This demonstrates that the bilevel program is only usable for quite small instances.
Moreover, we observe the to-be-expected trend that 's utility increases when  has more targets or  has less sensors. However, we do not find any consistent trend regarding whether it is more advantageous for : more targets or fewer sensors. 

Motivated by the high computational cost of the bilevel program, we now turn to analyzing the quality of our heuristics. We also include the Random method here as a baseline where  simply picks an arbitrary ordering of targets (and  best responds to it). In addition, we compare our heuristics against a naive random strategy of comparable computational cost. For this, we include the Random2 method which generates 3000 random orderings for Table <ref> and 3· 10^7 random orderings for Table <ref>. The sampled ordering that achieves the highest utility for  assuming that  best responds is returned.


	



In <Ref>,  we show the algorithms' performance for small instances where we can still compute 's maximum utility (OPT) via the bilevel program. 
In <Ref>, we consider larger instances where the optimum value is unknown. 
Note that higher values correspond to a better performance of the algorithm, as we always report 's  utility for 's best response.

From the results in <Ref>, we can see that all heuristics perform well on small instances. In particular, SA_Relax, SA, and Random2 find the optimal solution in all (but one) cases. However, SA_Relax proves advantageous because it only needs a sixth of the running time of the other two methods.

While our two heuristics SA and SA_Relax show a similar approximation quality for small instances, for larger instances (<Ref>) SA clearly outperforms SA_Relax. For the Default game setting, using SA compared to SA_Relax even regularly leads to a doubled utility for . 
While this is a strong argument for using SA, SA's downside is its higher computational cost, needing over 7 hours to solve instances with 75 targets. 

Finally, we observe that both methods clearly outperform the Random baseline, with SA consistently preserving an average of approximately 20 more targets for the larger instances.
This highlights that the solution quality of the target ordering clearly increases throughout the simulated annealing. 
Considering Random2, we find that repeatedly sampling orders (instead of only once) leads to a noticeable utility increase. However, on the larger instances, Random2 performs even worse than SA_Relax while running as long as SA, thereby combining the disadvantages of SA and SA_Relax.
Overall, our experiments highlight that  benefits from ordering the targets strategically instead of randomly. 


We also examine the averaged Stackelberg equilibrium state across 50 random instances by analyzing the experimental results. When the number of sensors is 2, Our findings reveal that nearly half of the targets typically remain not-sensed, with half of the not-sensed targets being valuable targets. The maximum value among the survived targets is typically associated with medium-high value targets rather than the highest value target. For instance, in the scenario where n=7, s=2, τ=2, an average of 4 targets survive, with approximately 1.7/4 ≈ 42.5% zero-valued targets. The average maximal value of the surviving targets is 0.6, compared to the overall maximal target value of 0.78. 
Additionally, in the setting of n=8, s=2, τ=2, the observation is similar. An average of 4.67 targets survive, with around 2.56/4.67≈ 54.8% zero-valued targets. The average maximal value of the surviving targets is 0.57, while the maximal value of all targets averages is 0.79.

What is more, when we increase the number of sensors s, the sensors would favor capturing more valuable targets, leaving the zero-valued platforms not-sensed. For example, in the setting of n=8, s=3, τ=2, an average of 4.32 targets survive, with around 3.2/4.32≈ 74% zero-valued targets. The average maximal value of the surviving targets is 0.34. With the same n=8, τ=2 but s=5,  an average of 3.53 targets survive, with around 3.28/3.53≈ 92% zero-valued targets. In other words, among the survived targets, only one of them is valuable and the average value of survived targets is 0.13 compared to the maximal value of all targets averages is 0.79.












§ ESCAPE FROM NON-COORDINATED SENSING


ESGs assume that the different sensors are controlled by a central authority that computes the sensing plan. We now investigate the situation where these sensors are non-coordinated and each one acts independently based on a natural greedy algorithm. This happens when sensors cannot easily exchange information and coordinate with each other. Another motivation is when  
 sensors are controlled by different adversaries, each serving only their own interests and being unlikely to coordinate their actions and share their reward.  
 Both of these scenarios can occur in our motivating domain of piracy at large open seas, as coordination between different groups is likely to be challenging. Different pirate groups might even refuse to coordinate at all and instead directly compete with each other.
 
We model these situations by assuming that sensors have a predefined ordering as s_1,…,s_k (as induced by fixed locations of the sensors); for each sensor s∈ S, as soon as a not previously sensed target that s can sense passes s (i.e., s has the capabilities and is currently not recharging), s senses it, thereby greedily maximizing its number of sensed targets.

Formally, given a target ordering σ, we construct a sensing plan ψ_σ sequentially as follows. For each step ℓ∈ [n+k-1], if target t_i passes sensor s_j in step ℓ, then we add t_i to ψ_σ(s_j) if the resulting sensing plan remains valid with respect to σ (formally, for i∈ [n] and j∈ [k] target σ^-1(i) passes sensor s_j in step i+j-1). 
As the strategy of   is fixed, the problem Best Blue Response  faces is to pick a target ordering σ so that v(ψ_σ) gets maximized.
In the following, we study the computational complexity of this problem and solve it in computational experiments. 
By comparing the answer of Best Blue Response to the value of the Stackelberg equilibrium in the corresponding ESG we can ultimately answer how much  gains from being able to centrally control its sensors. 



 §.§ Algorithmic Analysis


Unfortunately, it turns out that computing 's strategy is NP-hard, even in restricted cases where each sensor can only sense one target. 
Due to the sequential construction of 's sensing plan, this reduction is our most intricate one: 


theorembestblue
Best Blue Response is NP-complete, even if the recharging time is ∞, i.e., each sensor can sense only one target, each target has value 1, and the sum of each row and column in the sensing matrix is at most four. 



We focus on the variant where each sensor can only sense one target. 
Interestingly, as discussed in more detail in our full version <cit.> this problem shares some similarities with the NP-hard Minimum Maximal Matching problem, as we can view the sensors and targets as two sides of a bipartite graph with sensor-target pairs where the sensor senses the target corresponding to maximal matchings in this graph. 
However, the ordering of the sensors makes only certain maximal matchings in these graphs realizable, which is why we instead show NP-hardness by reducing from a variant of 3-SAT where each variable appears only twice positive and once negative. 
The core idea of our construction is the following: We add a literal target for each literal. Moreover, for each clause, we add a clause sensor and a clause target. The clause sensor is capable of sensing the corresponding clause target as well as targets corresponding to the three literals appearing in the clause. 
We add further targets and sensors to the instance so that all clause targets need to make it unsensed through the channel.  
This implies that each clause sensor needs to sense a literal target as it will otherwise sense the corresponding clause target in passing, i.e., we need to “cover” each clause with a literal appearing in the clause. 
Now for each variable, we add a slightly intricate gadget that ensures that we can either use the targets corresponding to positive literals to cover clause sensors (which corresponds to setting the variable to true) or the one target corresponding to a negative literal (which corresponds to setting the variable to false). Because we need to “cover” each clause, the induced assignment is satisfying. 


We can adopt a similar view as in <Ref> to solve the problem via dynamic programming. 
However, this time the dynamic programming iteratively constructs the optimal target ordering and we need to keep track of the previously used targets together with the sensors used in the last τ+1 timesteps. This results in a naive running time of 𝒪(n· 2^n· (k+1)^τ+2), which can be improved to  𝒪(n_χ·(∏_i=1^n_χ (ℓ_i+1))· (k+1)^τ+2) if we incorporate types:
propositionDBGreedy
Best Blue Response is solvable in 𝒪(n_χ·(∏_i=1^n_χ (ℓ_i+1))· (k+1)^τ+2), where ℓ_i is the number of targets of type γ_i.




  §.§.§ ILP Formulation

Constructing an ILP for Best Blue Response turns out to be slightly more challenging, as we need to encode 's greedy sequential behavior:
propositionbestblueILP
    There is an ILP formulation for Best Blue Response with  𝒪(n^2· k) binary variables, 𝒪(n) integer variables, and 𝒪(n^2· k) constraints.


    We introduce for each target i∈ [n] an integer variable z_i encoding the position in which the target appears. 
    Moreover, similar to  <Ref>, for each i∈ [n] and j∈ [k+1], we add a binary variable x_i,j, which encodes whether t_i is sensed by sensor s_j or whether the target makes it unsensed through the channel (for j=k+1).
    We can add mostly straightforward constraints to ensure that x_i,j respects recharging times. 
    The main challenge is to encode the greedy behavior of the sensors (i.e., the ILP cannot have the freedom to pick the x_i,j values arbitrarily to optimize 's utility but they are set according to sensors' greedy behavior). 
    For this, for each i,i'∈ [n] and j∈ [k], we add a binary variable y_i,i',j and add constraints so that y_i,i',j  is equal to one if target i is sensed by sensor j and because of this j recharges when i' is passing, i.e., i “covers” i'. 

To encode sensors' greedy behavior, we want to add a constraint that makes sure that in case x_i,j=1, the target needs to be covered by other targets for all sensors that are capable of sensing it placed before j. Note that this together with another constraint (∑_j∈ [k+1] x_i,j=1) in particular implies that each target is sensed by the first sensor it passes which is not recharging, thereby encoding the greedy behavior of sensors. 
Specifically, for each i∈ [n] and j∈ [k+1], we add: 


    ∑_t∈ [j-1]: D_i,t=0 1+    ∑_t∈ [j-1]:D_i,t=1∑_i'∈ [n] y_i',i,t -(j-1)
       ≥ -n(1-∑_t=j^k+1 x_i,t).








  §.§.§ Heuristic


Since it will turn out that the ILP formulation cannot quickly solve medium-to-large instances, we explore various simulated annealing-based heuristics, similar to the approach discussed in Section <ref>. 
We present the variant SA_Relax where a random neighbor is picked in <Ref>.
The other variant SA computes 's utility v(ψ_σ̂) for all neighbors and picks the one with the highest utility.  

 


 §.§ Experiments


We reuse the general setup described in <Ref>, but naturally now report 's computed utility assuming that sensors act greedily. Here, we let the Random2 method generate 1000 random orderings in Table <ref> and 5· 10^5 random orderings in Table <ref>.

First of all, we evaluate the scalability of our ILP for Best Blue Response (OPT) in Table <ref>. The ILP can solve the problem for medium-sized instances with up to 25 targets in a few minutes. However, due to the complexity of the ILP modeling, already for 25 targets as soon as the number of sensors reaches 5, instances can take more than 5 hours to solve. This is why the last line of the table only reports the running time for one instance. 



	








Next, we analyze the solution quality of our heuristic approaches. On small instances presented in <Ref>, our best heuristic algorithm approximates the optimal solution quite well and the error is typically below 10% with the SA_Relax method consistently outperforming SA. 
Both heuristics outperform Random, while Random2 performs better than SA (yet still worse than SA_Relax, while having a much longer running time).
When moving to larger instances in <Ref>, the picture flips, as SA is now substantially outperforming SA_Relax. This shows a general trend that the solution quality of SA scales more favorably than that of SA_Relax (while the opposite is naturally true for the running time). 
The heuristics again clearly outperform Random, with SA sensing approximately 15 more targets. Random 2 performs similarly to the suboptimal heuristic SA_Relax, while being slower by a factor of more than 100.


















Finally, we are interested in exploring the power of coordination for , i.e., the difference between the optimal utility  gets in the non-coordinated setting explored in this section compared to its utility in the Stackelberg equilibria from <Ref>. 
We find that for the small instances where we can compute the Stackelberg equilibrium exactly  can reduce 's utility by 10% to 20% through coordination. 
For larger instance sizes, we no longer know the optimal solutions, which is why we resort to comparing the results of the respective SA heuristics. 
We find that for larger instances, the gap decreases with  being only able to decrease 's utility by 5% through coordination in the instances from the Default setting underlying  <Ref>. 
In our full version <cit.>, we show that when 's sensors are capable of sensing more targets, coordination is more important sometimes leading to halving 's utility.





Moreover, in scenarios with large instance sizes, such as n=75, k=10, τ=5, where the optimal (bilevel) ILP is unsolvable, we can compare 's approximately optimal utility under the best heuristic algorithms. Specifically, as shown in Table <ref>, under the Default setting, the average 's utility under the SA algorithm is 2.26 ± 1.13 for 9 instances. Given the same 9 instances, the average 's utility when escaping from non-coordinated sensing is 4.43 ± 1.41, which is approximately twice the value observed in the coordinated sensing setting. Once again, due to the constant-sum nature of this game, the utility loss for  in transitioning from non-coordinated sensing to coordinated sensing essentially represents the utility gain for , highlighting the effectiveness of coordination.







§ CONCLUSION

By introducing Espace Sensing Games, we initiated the study of a new class of games concerned with target arrangement and motivated by security applications.  We showed that while the worst-case computational complexity of ESGs is prohibitive, our presented algorithms still have a good performance in experiments. 

There are multiple directions for future work emanating from our work.
First, pinpointing the precise complexity of computing Stackelberg equilibria remains a concrete open question. 
Second, there are other variants of ESGs beyond those studied by us. For instance, it would be possible to merge the settings studied in <Ref> into a game where sensors act greedily but  can control the ordering of the sensors. In this game variant where both  and  need to pick orders, it would also be possible to study simultaneous play or Stackelberg equilibria where  moves first. 
Lastly, there are various other target arrangement problems to be studied. One example could be a game where  needs to place targets on a grid and  cannot sense any two targets placed close to each other.


This work was supported by the Office of Naval Research (ONR) under Grant Number N00014-23-1-2802. The views and conclusions contained in this document are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied, of the Office of Naval Research or the U.S. Government.


plain
10

DBLP:conf/aaai/AdamHKK21
Lukás Adam, Rostislav Horcík, Tomás Kasl, and Tomás
  Kroupa.
Double oracle algorithm for computing equilibria in continuous games.
In AAAI, pages 5070–5077, 2021.

allende2013solving
Gemayqzel Bouza Allende and Georg Still.
Solving bilevel programs with the kkt-approach.
Math. Program, 138:309–332, 2013.

bampis2022scheduling
Evripidis Bampis, Konstantinos Dogeas, Alexander V Kononov, Giorgio Lucarelli,
  and Fanny Pascual.
Scheduling with untrusted predictions.
In IJCAI, pages 4581–4587, 2022.

behnezhad2018battlefields
Soheil Behnezhad, Avrim Blum, Mahsa Derakhshan, MohammadTaghi HajiAghayi,
  Mohammad Mahdian, Christos H Papadimitriou, Ronald L Rivest, Saeed Seddighin,
  and Philip B Stark.
From battlefields to elections: Winning strategies of blotto and
  auditing games.
In SODA, pages 2291–2310, 2018.

blocki2015audit
Jeremiah Blocki, Nicolas Christin, Anupam Datta, Ariel Procaccia, and Arunesh
  Sinha.
Audit games with multiple defender resources.
In AAAI, pages 791–797, 2015.

us
Niclas Boehmer, Minbiao Han, Haifeng Xu, and Milind Tambe.
Escape sensing games: Detection-vs-evasion in security applications.
CoRR, abs/XXXX, 2024.

bucarey2017building
Victor Bucarey, Carlos Casorrán, Óscar Figueroa, Karla Rosas, Hugo
  Navarrete, and Fernando Ordóñez.
Building real Stackelberg security games for border patrols.
In GameSec, pages 193–212, 2017.

chapman2014playing
Martin Chapman, Gareth Tyson, Peter McBurney, Michael Luck, and Simon Parsons.
Playing hide-and-seek: an abstract game for cyber security.
In ACySE, pages 1–8, 2014.

colson2007overview
Benoît Colson, Patrice Marcotte, and Gilles Savard.
An overview of bilevel optimization.
Annals of operations research, 153:235–256, 2007.

DBLP:books/daglib/0005753
John H. Conway.
On numbers and games, Second Edition.
Academic Press, 2001.

crawford1994experimental
James M Crawford and Andrew B Baker.
Experimental results on the application of satisfiability algorithms
  to scheduling problems.
In AAAI, pages 1092–1097, 1994.

DBLP:conf/mfcs/Demaine01
Erik D. Demaine.
Playing games with algorithms: Algorithmic combinatorial game theory.
In MFCS, pages 18–32, 2001.

dempe2020bilevel
Stephan Dempe and Alain Zemkoho.
Bilevel optimization.
In Springer optimization and its applications, volume 161.
  Springer, 2020.

fang2015introduction
Fei Fang, Thanh Nguyen, Benjamin Ford, Nicole Sintov, and Milind Tambe.
Introduction to green security games.
In IJCAI, 2015.

gurobi
Gurobi Optimization, LLC.
Gurobi Optimizer Reference Manual, 2022.

DBLP:conf/wg/GuruswamiRCCW98
Venkatesan Guruswami, C. Pandu Rangan, Maw-Shang Chang, Gerard J. Chang, and
  C. K. Wong.
The vertex-disjoint triangles problem.
In WG, pages 26–37, 1998.

DBLP:conf/atal/JainKVCPT11
Manish Jain, Dmytro Korzhyk, Ondrej Vanek, Vincent Conitzer, Michal Pechoucek,
  and Milind Tambe.
A double oracle algorithm for zero-sum security games on graphs.
In AAMAS, pages 327–334, 2011.

lenstra1977complexity
Jan Karel Lenstra, AHG Rinnooy Kan, and Peter Brucker.
Complexity of machine scheduling problems.
In Annals of discrete mathematics, volume 1, pages 343–362.
  Elsevier, 1977.

li2016catcher
Yuqian Li, Vincent Conitzer, and Dmytro Korzhyk.
Catcher-evader games.
In IJCAI, pages 329–337, 2016.

DBLP:conf/mfcs/LodingR03
Christof Löding and Philipp Rohde.
Solving the sabotage game is PSPACE-hard.
In MFCS, pages 531–540, 2003.

shieh2012protect
Eric Shieh, Bo An, Rong Yang, Milind Tambe, Craig Baldwin, Joseph DiRenzo, Ben
  Maule, and Garrett Meyer.
Protect: A deployed game theoretic system to protect the ports of the
  united states.
In AAMAS, pages 13–20, 2012.

sultanik2007modeling
Evan Sultanik, Pragnesh Jay Modi, and William C Regli.
On modeling multiagent task scheduling as a distributed constraint
  optimization problem.
In IJCAI, pages 1531–1536, 2007.

tahernejad2020branch
Sahar Tahernejad, Ted K Ralphs, and Scott T DeNegre.
A branch-and-cut algorithm for mixed integer bilevel linear
  optimization problems and its implementation.
Math. Program. Comput., 12:529–568, 2020.

tambe2011security
Milind Tambe.
Security and game theory: algorithms, deployed systems, lessons
  learned.
Cambridge university press, 2011.

pirate-un
UN-News.
Somalia: Pirates attack UN aid ship, prompting call for action,
  June 2007.
<https://reliefweb.int/report/somalia/somalia-pirates-attack-un-aid-ship-prompting-call-action>.

UNmarine
United-Nation.
United nations peacekeeping missions military maritime task force
  manual, Sept 2015.

vaneky2012game
Ondřej Vaněk, Zhengyu Yin, Manish Jain, Branislav
  Bošanskỳ, Milind Tambe, and Michal Pěchouček.
Game-theoretic resource allocation for malicious packet detection in
  computer networks.
In AAMAS, pages 905–912, 2012.

winn2008maritime
John I Winn and Kevin H Govern.
Maritime pirates, sea robbers, and terrorists: New approaches to
  emerging threats.
Homeland Security Rev., 2:131, 2008.

DBLP:journals/siamcomp/Yannakakis81
Mihalis Yannakakis.
Edge-deletion problems.
SIAM J. Comput., 10(2):297–309, 1981.

yannakakis1980edge
Mihalis Yannakakis and Fanica Gavril.
Edge dominating sets in graphs.
SIAM J. Appl. Math., 38(3):364–372, 1980.

zhang2016co
Chongjie Zhang and Julie A Shah.
Co-optimizating multi-agent placement with task assignment and
  scheduling.
In IJCAI, pages 3308–3314, 2016.










§ ADDITIONAL MATERIAL FOR <REF>


*

For our dynamic program, we create a table J[i,b_1,…,b_τ+1]∈ℕ for each i∈ [n] and b_1,…,b_τ+1∈ S∪{∅}. 
J[i,b_1,…,b_τ+1] stores the minimum value of surviving target from t_1,…, t_i induced by a valid sensing plan ψ that for each j∈ [max(1,i-τ),i] assigns target t_j to sensor b_i-j+1  if b_j≠∅ and to no sensor if b_j=∅. 
In this case, we say that ψ witnesses the table entry.
If no such sensing plan exists, J[i,b_1,…,b_τ+1] is ∞. 
Clearly, 
the answer to the problem is min_b_1,…,b_τ+1∈ S∪{∅}J[n,b_1,…,b_τ+1]. 

For the initialization, we set an entry J[i,b_1,…,b_τ+1]  to 0 if i=0 and to ∞ otherwise. 
Now, we update the table for increasing i=0,1,…,n by filling J[i,b_1,…,b_τ+1] as follows. We start by assuming that b_1≠∅ implying that t_i is to be sensed by b_1. In this case, we set the entry to ∞ if  ∃ℓ∈ [2,τ+1]: b_1=b_ℓ (i.e., the recharging period of b_1 would be violated) or if b_1 is not capable of sensing t_i. 
In both cases, there is no sensing plan to witness the entry.
Otherwise, we update the entry as: 

    J[i,b_1,…,b_τ+1]=min_s∈ S∪{∅} J[i-1,b_2,…,b_τ+1,s].

We claim that in case an update of the entry happens here, it is correct.
For this, let s^*∈ S∪{∅} be the minimizer of the right hand side and ψ^* the plan witnessing  J[i-1,b_2,…,b_τ+1,s^*]. Then, we can extend ψ^* to a sensing plan ψ' by adding t_i to ψ^*(b_1). Plan ψ' is valid and a witness for the updated entry, since ψ^* is valid and we ruled out above that b_1 is either not ready or capable of sensing t_i.

Lastly, it remains to consider the case b_1=∅, where t_i is not assigned to any sensor. In this case,  we update the entry as: 

    J[i,b_1,…,b_τ+1]=min_s_∈ S∪{∅} J[i,b_2,…,b_τ+1,s]+v_i.

Analogous to above, in case an update happens assume that  s^*∈ S∪{∅} is the minimizer of the right-hand side and ψ^* the plan witnessing  J[i-1,b_2,…,b_τ+1,s^*]. 
Then ψ^* is also a witness for the updated entry. 

The algorithm runs in 𝒪(n· (k+1)^τ+2), as the table contains 𝒪(n· (k+1)^τ+1) entries and for each entry we need to take the minimum over k+1 values. 

For i∈ [k_χ], let ℓ_i be the number of sensors of type θ_i.
To extend the algorithm to sensor types, we need to prove the following lemma that allows us to collapse all sensors of the same type θ_i into a “meta”-sensor, which can sense ℓ_i many targets in each τ-time window:

There is a valid sensing plan of value v if and only if there is a sensing plan of value v where sensor capabilities are respected and for each j∈ [1,n-τ] at most ℓ_i targets from {t_j,…, t_j+τ} are assigned to a target of type θ_i for i∈ [k_χ].


     Let ψ be a valid sensing plan of value v. We claim that ψ also fulfills the second condition. Assume for the sake of contradiction that there is a θ_i and some j∈ [1,n-τ] so that more than ℓ_i targets from {t_j,…, t_j+τ} are assigned to a target of type θ_i. Then, by the pigeonhole principle, at least one sensor of type θ_i needs to sense two targets within τ steps, rendering the plan invalid. 

    For the reverse direction, assume ψ is the sensing plan of value v respecting the condition. For each sensor type θ_i, let T_ψ(θ_i) be the targets sensed by sensors of type θ_i in ψ. 
    We construct a sensing plan ψ' as follows. 
    For each sensor type θ_i, we iterate over the targets in T_ψ(θ_i) according to their position in the target ordering and always assign a target to the sensor of type θ_i who has not sensed a target for the longest time. 
    ψ' has the same value as ψ and respects sensor capabilities, so it remains to argue that the recharging periods are respected. 
    For the sake of contradiction assume that there is a sensor of type θ_i and some j so that the sensor is assigned two targets from {t_j,…, t_j+τ} in ψ'. 
    Then, by the construction of ψ' all other sensors of type θ_i are also assigned a target from {t_j,…, t_j+τ}. It follows that sensors of θ_i are assigned at least ℓ_i+1 many targets from {t_j,…, t_j+τ} in ψ' and thereby also in ψ, which contradicts our initial assumptions on ψ.     

Using this lemma we can easily adjust the dynamic programming formation: Instead of bookmarking the sensors b_1,…,b_τ+1∈ S that have sensed the last τ+1 targets, we instead bookmark the types θ'_1,…,θ'_τ+1∈Θ of these sensors. Now, we can set a table entry to ∞ (due to violated recharging time) if {θ'_1,…,θ'_τ+1} contains more some sensor type θ_i∈Θ more than ℓ_i times. The rest of the algorithm adapts in a straightforward manner with a resulting running time of 𝒪(n· (k_χ +1)^τ+2)




§ ADDITIONAL MATERIAL FOR <REF>

*

Let σ:T→ [n] be some target ordering. 
We will say that two targets t,t'∈ T are (placed) at distance j (in σ) if |σ(t)-σ(t')|=j. 
Further, for a target t we define its i-surrounding to be the set of all targets whose distance from t is at most i. 

We start the proof with two immediate claims about optimal sensing plans in response to some target ordering σ:

    Let s be a sensor and t be a target so that s is the only sensor capable of sensing t and t has a higher value than all the other targets s can sense combined.  s senses t in an optimal sensing plan. 


Assume that s did not sense t, we could just arrive at a better valid sensing plan by letting s sense only t. 



    Let s be a sensor with recharging time τ and t be some target s is capable of sensing. If there is no other target s capable of sensing in the τ-surrounding of t, then t will be sensed by some sensor in the optimal sensing plan.


    Assume that t is not sensed by any sensor. Then we can just arrive at a better valid sensing plan by letting s sense t: This will not violate the recharging constraint, as s does not sense any other targets in the τ-surrounding of t. 


We reduce from Independent Set on 3-regular triangle-free graphs <cit.>.



  
Construction
Let G=(V,E) be a 3-regular triangle-free graph.
For each vertex v∈ V, we introduce a vertex target α_v of value 1 and for each i∈ [3] a blocker vertex target β^i_v of value 3 and a vertex sensor a^i_v which is capable of sensing α_v and β^i_v. 
Moreover, for each edge e={u,v}∈ E, we introduce an edge target γ_e of value 3 and an edge sensor  g_e that can sense γ_e, α_u, and α_v. 
Additionally, we add a constraining edge target γ̂_e of value 4 and a constraining edge sensor ĝ_e  that is capable of sensing targets γ_e and γ̂_e.  
The recharging time is τ=3. 
We claim that there is an ℓ-sized independent set  X in G if and only if the Stackelberg equilibrium in the constructed ESG has value at least ℓ. 



  
Forward Direction
Assume that X⊆ V is an independent set of size ℓ. 
To construct the target ordering we go through the vertices from the independent set one by one:  
For v∈ X, let e_1,e_2,e_3 be the three edges incident to v. 
Then, we add to the ordering of targets the targets γ̂_e_1β^1_v,β^2_v,γ_e_1,α_v,γ_e_2,γ_e_3,β^3_v,γ̂_e_2,γ̂_e_3 in this order. 
After we have processed all vertices from the independent set like this, we append all other targets in a random ordering. 
First observe that because of  <Ref>, the constraining edge sensors will always sense the constraining edge targets. 
As a result of the structure of our target ordering, the constraint edge sensors for edges incident to v∈ X cannot sense the corresponding edge targets. 
From this, following a reasoning analogous to <Ref>, we get that for each edge e incident to a vertex from X, the edge sensor g_e senses the edge target γ_e.
Moreover, we also get from <Ref> that all three vertex sensors corresponding to a vertex v∈ X sense their respective blocker vertex target. 
All in all, it follows that for each vertex v∈ X all three corresponding vertex sensors and all edge sensors corresponding to incident edges sense a target within distance 3 of α_v. Thus, none of the sensors that are capable of sensing v can do so without violating their recharging constraint.
Thus, all ℓ vertex targets corresponding to vertices from X make it unsensed through the channel.



  
Backward Direction
Assume that we have a target ordering σ so that targets of summed value at least ℓ make it unsensed through the channel under optimal play by . 
Let ψ be the optimal sensing plan played in response by . 
From <Ref>, it is immediate that no blocker vertex target and no constraining edge target can make it unsensed through the channel. Moreover, it is also easy to see that no edge target can make it unsensed through the channel, as otherwise we could always improve the plan by letting the corresponding edge sensor sense the edge target (and delete all vertex targets the sensor senses instead). 
Let X be the set vertices so that the corresponding vertex targets make it unsensed through the channel. 
By the above observation, we have |X|≥ℓ. We will now show that X is an independent set. 

For this, we make a series of observations: 
First, let α_v be an unsensed vertex target. 
Note that α_v can be sensed by 6 different sensors (three vertex sensors and three edge sensors). 
By <Ref>, we have that for all of these sensors there needs to be another target that the sensor is capable of sensing in the 3-surrounding of α_v. 
As there are 6 targets in a 3-surrounding, it follows that the 3-surrounding of α_v contains only targets that can be sensed by one of these six sensors (specifically one target for each of these six sensors). 

Second, we show that for each unsensed vertex target α_v there is no other vertex target in its 3-surrounding. 
Recall from the first observations that the only vertex targets that could be in the 3-surrounding of α_v are vertex targets corresponding to neighbors of v that are sensed by the corresponding edge sensor in ψ. 
For the sake of contradiction assume that there is some u∈ V so that e={u,v}∈ E and |σ(α_v)-σ(α_u)|≤ 3. 
Observe that the 3-surroundings of α_u and α_v overlap in at least 3 targets. 
As G is triangle-free and thus u and v do not share any common neighbors,
this implies that the 3-surrounding of α_u contains at most three targets that can be sensed by one of the six sensors that are capable of sensing α_u (the other three spots will be filled with vertex blocker targets corresponding to v and edge targets corresponding to edges incident to v by the first observation and as v makes it unsensed through the channel). 
Let s be a sensor that is capable of sensing α_u for which no other target that s can sense in the 3-surrounding of α_u. 
Next, note that by the first observation, α_u is sensed by g_e in ψ. 
We alter the sensing plan ψ as follows. We let α_v instead of α_u be sensed by g_e and let α_u be sensed by s. 
As argued above, this does not violate the recharging time of the sensor s. 
Moreover, it also does not violate the recharging time of g_e, as there are no other targets (except α_u) that g_e is capable of sensing in the 3-surrounding of α_v (by the first observation). 
In the altered sensing plan the value of surviving targets is strictly smaller contradicting that ψ is an optimal response by . 

Third, we claim that for each of two unsensed vertex targets α_v and α_v' we have that they are placed at a distance at least 7 from each other, i.e., their 3-surroundings do not overlap. 
In order to show that this cannot be the case, we need to examine the constraining edge sensors. 
Let e be an edge incident to v.
Our first two observations told us that the edge target γ_e needs to be in the 3-surrounding of α_v. 
However, this is not sufficient:
We claim that the constraining edge target γ̂_e needs to be in the 3-surroundings of γ_e. 
Assume that this was not to hold. Then we could alter our sensing plan ψ by making ĝ_e sense γ_e (it can do so without violating its recharging time because γ̂_e is not in the 3-surrounding of γ_e) and make g_e sense α_v, which leads to a strictly better sensing plan. The claim follows.
To prove the observation, for the sake of contradiction assume that the 3-surroundings of α_v and α_v' did overlap.
From the previous two observations, it follows that the only possibility for their 3-surrounding to overlap is that there is an edge e={v,v'}∈ E and that γ_e constitutes the overlap of their 3-surrounding. 
However, in this case γ̂_e cannot be placed in the 3-surrounding of γ_e, as it can neither belong to the 3-surrounding of α_v nor α_v' (by the first observation). This leads to a contradiction to our above claim. 

Now, combining these observations it follows that for each v∈ X, the 3-surrounding of α_v contains edge targets γ_e_1,γ_e_2,γ_e_3 with e_1, e_2, and e_3 being the edges incident to v. 
Moreover, we have shown that the 3-surrounding of each target α_v with v∈ X is disjoint. 
This implies that no two vertices v,u∈ X can be incident to the same edge e, as otherwise, γ_e would be in the 3-surrounding of α_u and α_v, which leads to a contradiction as they are disjoint. 
It follows that X is an independent set of size at least ℓ.

We claim that none of the targets in a r/2+1 distance of the vertex target can be vertex targets. 
Assume that one of them would be u with e={u,v}. 
We alter the current sensing plan as follows. We let γ_e sense v. 
Note that the r/2+1-surrounding of u and v overlaps in at least r/2+2 positions. 
Moreover, not that none of these positions contain vertex targets that 

Note moreover, that for two target vertices at positions i,j that both make it trough we need to have that |i-j|≥ r+2, as for both of them the two targets at distance r/2+1 from their position are vertex blocker targets. As those targets are different for both vertices, they cannot both make it through, as there is no space. 

We introduce a vertex target β_v for each vertex v∈ V of value 1.
For each color class c∈ℓ, we introduce two color targets α^1_c and α^2_c both of value |V|+1 and two color sensors a^1_c and a^2_c with recharging time τ=1. Sensor a^1_c (resp. a^2_c) is capable of sensing the corresponding color target α^1_c (resp. α^2_c and the vertex targets corresponding to vertices from this color. 
Moreover, for each i∈ [r+1] and each e={u,v}∈ E^i, we add an edge target γ_e of value |V|+1 and an edge sensor g_e with recharging time τ=i+1 that is capable of sensing the vertex targets corresponding to its endpoints, i.e., β_u and β_v. 

For each vertex v∈ V, let [γ(v)] be an ordering of the r edge targets corresponding to v's incident edges sorted decreasingly by the index of their edge set E^i, i.e., for two  incident edges e and e' with e∈ E^i and e'∈ E^i' we place e before e' if i>i'. Note that this defines the ordering uniquely as each E^i contains only at most one edge incident to v. 

We claim that there is an ℓ-sized independent set  X with X∩ V^c≠∅ in G if and only if the Stackelberg equilibrium in the constructed ESG has value at least ℓ. 

    We reduce from the NP-hard Multicolored Independent Set on regular graphs problem. In this problem, we are given an r-regular ℓ-partite graph G = (V=V^1  V^2 … V^ℓ, E)  and the question is whether there is a clique X of size ℓ in G with X∩ V^c≠∅ for 
all c∈ [ℓ]. For the sake of simplicity, we assume that r is even. 
As usual, we refer to the vertex classes V^i as color classes and to [ℓ] as the colors. 
By Vizing's theorem, we can partition the edges of E into r+1 sets E=E^1  E^2 … E^r+1 so that no vertex is incident to two edges from one set. 



  
Construction
We introduce a vertex target β_v for each vertex v∈ V of value 1.
For each color class c∈ℓ, we introduce two color targets α^1_c and α^2_c both of value |V|+1 and two color sensors a^1_c and a^2_c with recharging time τ=1. Sensor a^1_c (resp. a^2_c) is capable of sensing the corresponding color target α^1_c (resp. α^2_c and the vertex targets corresponding to vertices from this color. 
Moreover, for each i∈ [r+1] and each e={u,v}∈ E^i, we add an edge target γ_e of value |V|+1 and an edge sensor g_e with recharging time τ=i+1 that is capable of sensing the vertex targets corresponding to its endpoints, i.e., β_u and β_v. 

For each vertex v∈ V, let [γ(v)] be an ordering of the r edge targets corresponding to v's incident edges sorted decreasingly by the index of their edge set E^i, i.e., for two  incident edges e and e' with e∈ E^i and e'∈ E^i' we place e before e' if i>i'. Note that this defines the ordering uniquely as each E^i contains only at most one edge incident to v. 

We claim that there is an ℓ-sized independent set  X with X∩ V^c≠∅ in G if and only if the Stackelberg equilibrium in the constructed ESG has value at least ℓ. 



  
Proof of Correctness: Forward Direction
Assume that X⊆ V is an ℓ-sized independent set in G with X∩ V^c≠∅. 
For c∈ [ℓ], let v_c∈ X∩ V^c be the unique vertex from X of color c. 
We construct the ordering of targets as follows. For each c∈ [ℓ] we start with the targets from [γ(v_c)] followed by the target α^1_c, β_v_c, and α^2_c.
After that, we append all targets in an arbitrary ordering. 
Note that we did not place any target twice, as X is an independent set, and thereby [γ(v_c)] does not contain the same edge target for two different colors. 

We claim that all targets β_v_c make it through the channel assuming that red responds with an optimal sensing plan. 
Fix some c∈ [ℓ]. We prove the claim by iterating over all sensors that can sense β_v_c. 
The color sensor a^1_c (resp. a^2_c) needs to sense the corresponding color target α^1_c (resp. α^2_c) in an optimal sensing plan, as α^1_c (resp. α^2_c) can only be sensed by a^1_c (resp. a^2_c) and is more valuable than all other targets the sensor can sense combined. 
As α^1_c (resp. α^2_c) is placed directly before (resp. beyond) β_v_c and a^1_c and a^2_c both have a recharging period of 1, neither of the two can sense β_v_c. 
It remains to consider the edge sensors corresponding to edges incident to v_c. 
For this, fix some edge e∈ E^i incident to e. 
In an optimal plan, the edge sensor g_e needs to sense the edge target γ_e, as γ_e can only be sensed by g_e and γ_e is more valuable than all other targets that g_e can sense combined. 
Recall that the recharging time of g_e is i+1. Moreover, note that there are at most i targets placed between γ_e and β_v_c, i.e., the color target α^1_c and the at most i-1 edges incident to v_c from ∪_j∈ [i-1] E^j. 
This implies that g_e does not sense β_v_c in an optimal plan. 
This shows that all vertex targets corresponding to vertices from X make it through the channel and thus that the Stackelberg equilibrium has a value of at least ℓ.  



  
Proof of Correctness: Backward Direction
Assume there is a Stackelberg equilibrium in the constructed game with target ordering σ and optimal sensing plan ψ so that v(ψ)≥ℓ. 
As argued above, in ψ, for each c∈ [ℓ], the color sensor a^1_c (resp. a^2_c) will sense the color target α^1_c (resp. α^2_c).
Moreover, in ψ, for each e∈ E, the edge sensor g_e will sense the edge target γ_e. 
Thus, only vertex targets can make it through the channel.

However, note that in order for a vertex target β_v to make it through the channel, all edge sensors corresponding to 

We continue by proving that from each color c∈ [ℓ] at most one target corresponding to a vertex from the color can make it through the channel (we will later prove that these vertices form an independent set). 
Fix some color c∈ [ℓ]. 
Let v_1,…,v_j∈ V^c be some vertices from color class c so that β_v_i is directly placed before β_v_i+1 for i∈ [j-1]. 
Assume that j≥ 2. Then the color sensors α_c^1 and α_c^2 will be able to sense all these vertex targets: Because both α_c^1 and α_c^2 

ERROR: We cannot easily conclude that at most one very from each color can survive because a_c's recharging time of 1.




*

We start by giving the formulation, where the inner-level program is similar to the ILP presented in <Ref>. Note that the outer-level maximizes the value of non-sensed targets, while the inner-level minimizes this value:

    max_z, t   ∑_i∈ [n] v_i · x_i,k+1
    s.t.    z_i  z_i',  ∀ i i' ∈ [n] 
       τ + 1 - n·(1-l_i,j)≤ z_j - z_i,  ∀ i,j∈[n] 
       z_j - z_i ≤ n· l_i,j + τ,  ∀ i,j∈[n] 
       𝐱 = argmin_𝐱̂∑_i∈ [n] v_i ·x̂_i,k+1
    s.t.    ∑_j∈ [k+1]x̂_i,j=1, ∀ i ∈ [n]
       x̂_i,j≤ D_i,j, ∀ i ∈ [n], j ∈ [k] 
       x̂_i,j+x̂_i',j≤ 1+l_i,i'+l_i',i, ∀ i i' ∈ [n], j ∈ [k]

The outer-level program controls an integer variable z_i for each target i∈ [n] that encodes the position in which the target appears in the final ordering. 
Moreover, for each pair of targets i,j∈ [n], we have a binary variable l_i,j capturing whether t_i appears at least τ+1 positions before t_j in the ordering induced by the z variables. 
<Ref> ensures that each target is assigned a unique position[For each i≠ i'∈ [n], to convert <Ref> into a linear constraint, we have to introduce a new binary variable δ_i,i'. We then add two linear constraint z_i ≤ z_j - 1 + n·δ_i,j and z_i ≥ z_j + 1 + n· (1-δ_i,j) that are satisfied if and onlf if z_i≠ z_i'. ] and <Ref> ensures that l_i,j is set to one if and only if  z_j≥ z_i+τ +1 (in <Ref>  we have τ + 1 ≤ z_j - z_i if l_i,j = 1; and z_j - z_i ≤τ if l_i,j=0).

The inner-level program controls binary variables x_i,j for each i∈ [n] and j∈ [k+1], which encode the sensing plan as in <Ref>, i.e, x_i,j=1 for i∈ [n] and j∈ [k] implies that t_i is sensed by s_j. 
The value of the encoded plan is again ∑_i∈ [n] v_i· x_i,k+1, which  wants to maximize (<Ref>) and  wants to minimize (<Ref>). 
The validity of the sensing plan is secured in <Ref>, where <Ref> imposes that a sensor s_j can only sense two targets t_i and t_i' if either t_i is at least τ+1 positions before t_i' in the ordering encoded in the z variables (i.e., l_i,i'=1) or the other way around (i.e., l_i',i=1).


We give our greedy approximation algorithm for Best Red Response in <Ref>. 



§ ADDITIONAL MATERIAL FOR <REF>




 §.§ Proof of <Ref>




  §.§.§ Connection Best Blue Response and Minimum Maximal Matching
 
On an intuitive level, solving Best Blue Response with infinite recharging time τ=∞ and uniform target values has some similarities to solving the classic NP-hard Minimum Maximal Matching problem <cit.>: 
Minimum Maximal MatchingA bipartite graph G=(U∪ V, E) and an integer ℓ.Is there a maximal matching in G containing at most ℓ edges?

We now discuss the intuitive connection as well as reasons why immediate reductions between the two problems are prohibited. 
Assume that we have a solution to our Best Blue Response instance where targets are ordered as (t_1, …, t_n). 
From this let us construct a bipartite graph G with vertices T on the left side and vertices S on the right side. For the edge set E, we add an edge between a target t and a sensor s if s is capable of sensing t
Let now F⊆ E be the set of sensor-target pairs with {t,s}∈ F if s senses t when targets are send according to (t_1, …, t_n). 
It needs to hold that F is a maximal matching in G: 
Otherwise, there is some {t,s}∈ E∖ F. 
The existence of this edge implies that target t made it through the channel and sensor s did not sense any target, which leads to a contradiction. 
Moreover, the size of this matching, i.e., |F|, corresponds to the number of lost targets.
Thus,  wants to find a maximal matching of minimum size. 
This discussion suggests a close connection between Simple Sequential Covering and Minimum Maximal Matching. 

However, there are some crucial differences between the two problems which prohibit immediate reductions from one problem to the other: 
Most crucially, assume we were to model a bipartite graph as an instance of Best Blue Response by letting U be the targets and V the sensors (in some ordering (v_1,…, v_n)) and  let v be capable of sensing u if {u,v}∈ E. 
The problem with this construction is that we cannot model arbitrary matchings E'⊆ E as solutions to the constructed Best Blue Response instance: 
Assume that E' contains some edge {u,v_i} and there is some v_j with j<i, {u,v_j}∈ E, and v_j is not incident to any edges from E'. 
In this case, it is not possible to send the targets through the channel such that E' will be the sensed sensor-target pairs because u will always be sensed by v_j before it can be sensed by v_i (which in turn implies that v_i is still ready to sense other targets). 
As a consequence, intuitively speaking, in instances of Best Blue Response we are only interested in maximal matchings where each matched vertex from the left is matched to its “first” otherwise unmatched neighbor from the right side. 

Because of this, we need to turn to a slightly more involved reduction that draws inspiration from the NP-hardness proof of Minimum Maximal Matching by <cit.>, yet requires some reworking of the construction and a different more involved proof. 
We reduce from the following SAT variant.



  §.§.§ Proof of Correctness

*

In this proof, for a target t, we let D(t) be the set of sensors that are capable of sensing t. 

We reduce from the following problem, which is  NP-hard as proven by <cit.>. 
Restricted 3-SatA propositional formula (X,C) where each clause contains three literals and each variable appears in exactly two clauses positively and in exactly one clause negatively.Is there an assignment to variables in X such that each clause from C is satisfied?

Let (X,C) be a given Restricted 3-Sat instance. 



  
Construction.
Each target has value one.
For each clause c∈ C, we add a clause target t_c and a clause sensor s_c with D(t_c)={s_c}.

For each variable x∈ X, we add a variable gadget.
That is, we add variable targets t_x,1, t_x,2 and t_x̅ together with dummy targets t^du_x,1, t^du_x,2, t^du_x,3, and t^du_x,4.
Next, we add catch sensors s^ca_x,1, s^ca_x,2, and s^ca_x̅, which ensure that none of the variable targets can make it through the channel. 
Moreover, we add variable sensors s_x,1, s_x,2 and s_x̅ together with dummy sensors s^du_x,1 and s^du_x,2.

Let c_i,c_j,c_k∈ C be three clauses such that x appears positive in clauses c_i and c_j and negative in clause c_k. 
The sensing matrix is defined through:

    D(t_x,1)={s_c_i,s_x,1,s^ca_x,1}    D(t_x,1^du)={s_x,1,s^du_x,1}
        D(t_x,2)={s_c_j,s_x,2,s^ca_x,2}    D(t_x,2^du)={s_x̅,s^du_x,1}
        D(t_x̅)={s_c_k,s_x̅,s^ca_x̅}    D(t_x,3^du)={s_x̅,s^du_x,2}
             D(t_x,4^du)={s_x,2,s^du_x,2}


The recharging time is ∞, i.e., each sensor can sense at most one target.
The ordering of the sensors is as follows. 
First, come the clause sensors (in some arbitrary ordering), then the dummy sensors (in some arbitrary ordering), then the variable sensors (in some arbitrary ordering) and last the catch sensors (in some arbitrary ordering).  
We ask whether there is an ordering of targets so that at least ℓ:=|C|+2|X| targets are not sensed.
It is easy to see that the construction satisfies the restrictions from the theorem statement.

[inline]Potentially add some intuition here if wanted.
We reduce from a variant of 3-SAT where each variable appears only twice positive and once negative. We add a target for each literal. Moreover, for each clause we have a "clause sensor" and a "clause target". The clause sensor can sense the clause target as well as targets corresponding to the three literals appearing in the clause. We need all of the clause targets to make it through the channel so we need to cover each clause sensor with a target corresponding to a literal appearing in the clause (this is how we ensure that we have a satisfying assignment). Now for each variable, we add some gadget that ensures that we can either freely use the targets corresponding to positive literals to "cover" clause targets with them or the one target corresponding to a negative literal (this ensures that we properly set each variable either to true or false).  




  
Proof of Correctness: Forward Direction
Assume we are given an assignment of variables X that fulfills C. 
Let X^*⊆ X be the set of variables set to true in this assignment. 
From this, we construct a partition of the targets into four groups that determine the ordering in which the targets move through the channel; the first group comes first and so on;  the ordering of targets within one group is arbitrary. 
The first group consists of  targets t_x,1 and t_x,2 for each x∈ X^* and 
target t_x̅ for each x∉ X^*. 
The second group consists of targets t^du_x,1 and t^du_x,4 for each x∈ X^* and t^du_x,2 and t^du_x,3  for each x∉ X^*. 
The third group contains all remaining variable targets. 
And, finally, the fourth group contains all |C| clause targets and the remaining 2|X| dummy targets. 
It is sufficient to prove that all ℓ targets from the fourth group make it through the channel. 
We prove this via a series of three claims. 



    
  * Each clause sensor senses a target from the first group. 
    
  * Each dummy sensor senses a target from the second group. 
    
  * For each x∈ X^*, s_x̅ senses a target from one of the first three groups, and for each x∉ X^*, s_x,1 and  s_x,2  sense a target from one of the first three groups.



Proof of 1. This follows directly from the fact that X^* is a satisfying assignment and that the clause sensors come first in the ordering of sensors. 

Proof of 2. This follows directly from the fact that the dummy sensors come after the clause sensors in the sensor ordering and as the second group contains for each x∈ X t^du_x,1 or t^du_x,2 (making s^du_x,1 sense a target) and  t^du_x,3 or t^du_x,4 (making s^du_x,2 sense a target).

Proof of 3. Let us focus on one x∈ X, and let c_i,c_j,c_k∈ C be three clauses such that x appears positive in clauses c_i and c_j and negative in clause c_k. 
If x∈ X^*, then t_x̅ is part of the third group. However, from Statement 1 it follows that s_c_k already sensed a previous target. 
As the variable sensors are before the catch sensors in the sensor order, it follows that s_x̅ senses t_x̅. 
Similarly, if x∉ X^*, t_x,1 and t_x,2 are part of the third group. Both s_c_i and s_c_j have already sensed previous targets because of Statement 1. From this Statement 3 follows. 

The claim implies that all sensors that can sense a target from the fourth group have already sensed another target before it is the fourth group's turn, implying that all ℓ targets from the fourth group will make it through the channel. 



  
Proof of Correctness: Backward Direction
Assume that there is an ordering of the targets such that at least ℓ targets move unsensed through the channel, and let P^*⊆ P be the set of these targets.

We prove the backward direction in a series of claims: 

    
        
  * No variable target is part of P^*. 
        
  * For each x∈ X, either t^du_x,1 or t^du_x,2 and either t^du_x,3 or t^du_x,4 is part of P^*. All clause targets are part of P^*.
        
  * For each x∈ X, if t_x̅ is sensed by a clause target, then neither t_x,1 nor t_x,2 are sensed by a clause target. 
        



Proof of 1. This follows immediately from the existence of a designated catch sensor for each variable target which can only sense this target. As a consequence, no variable target can ever make it unsensed through the channel. 

Proof of 2. Note that because of the sensor s_x,1^du it is never possible that both t^du_x,1 and t^du_x,2 make it unsensed through the targets. Similarly, because of s_x,2^du it is never possible that both t^du_x,3 and t^du_x,4 make it unsensed  through the channel. 
Together with Statement 1, this implies that from each variable gadget at most 2 targets can be part of P^*. 
By recalling that ℓ=|C|+2· |X| and that there are only |C| clause targets outside of variable gadgets, the statement follows. 

Proof of 3. Let us focus on t_x,1 (the proof for t_x,2 is analogous). 
For the sake of contradiction assume that t_x,1 and t_x̅ are both sensed by clause sensors, then both s_x,1 and s_x̅ do not sense a variable target, respectively. 
Accordingly, at most one target out of {t^du_x,1,t^du_x,2,t^du_x,3,t^du_x,4} can make it unsensed through the channel, contradicting Statement 2.

Let α be a truth assignment that sets x∈ X to false if t_x̅ is sensed by a clause sensor and x∈ X to true if t_x,1 or t_x,2 is sensed by a clause sensor.
If neither of the two conditions hold, then we set x to true.
Note that the well-definedness of α follows immediately from Statement 3 of <Ref>.
Assume that c∈ C is not satisfied by α. However, this implies that s_c does not sense a variable target corresponding to a literal appearing in c (by the definition of α).
This implies that s_c will sense t_c, a contradiction to Statement 2 of <Ref>. 




 §.§ Proof of <Ref>

*

Recall that we assume that sensors act greedily and that the sensor ordering is fixed and known.
We iteratively construct the target ordering always appending an additional target at the end of the ordering, while storing the types of already sent targets as well as the sensors that sensed the last τ+1 targets.
For our dynamic program, we create a table J[i_1,…,i_n_χ,b_1,…,b_τ+1]∈ℕ with i_j∈ [ℓ_j] for each j∈ [n_χ] and b_1,…,b_τ+1∈ S∪{∅}. 
For a table cell, let i:= ∑_j∈ [n_χ] i_j, i.e., i is the total number of targets that have been sent. 
An entry of the table stores the maximum value of targets that can survive if  sends i_j targets of type γ_j (for each j∈ [n_χ]) through the channel in a way that the tth last target sent for t∈ [max(1,(∑_j∈ [n_χ] i_j)-τ),∑_j∈ [n_χ] i_j] is sensed by sensor 
if b_j≠∅ and by no sensor if b_j=∅ (the intuition is that b_1 is the sensor that sensed the most recent passing target (if existent), b_2 sensed the second most recent one, and so on. 
If the second constraint is not realizable, we set the table entry to -∞. 
The answer to our problem is min_b_1,…,b_τ+1∈ S∪{∅}J[ℓ_1,…,ℓ_n_χ,b_1,…,b_τ+1]. 

For the initialization, we set an entry J[i_1,…,i_n_χ,b_1,…,b_τ+1]  to 0 if ∑_j∈ [n_χ] i_j=0 and to -∞ otherwise. 
Now, we update the table for increasing ∑_j∈ [n_χ] i_j=0,1,…,n by filling J[i_1,…,i_n_χ,b_1,…,b_τ+1] as follows.
We start by assuming that b_1≠∅ implying that the next target to be sent needs to be sensed by b_1.
If b_1 appears among b_2,…, b_τ+1, we set the entry to -∞. Otherwise, let γ_j_1, …, γ_j_z be the target types so that b_1 is capable of sensing targets of this type and only sensors from {b_2,…,b_τ+1} appear before b_1 in the sensor ordering and are capable of sensing targets of this type. Less formally speaking, γ_j_1, …, γ_j_z  are all the target types so that if a target of this type is sent next over the channel b_1 would be the sensor sensing this target (as all other sensors placed before b_1 that can sense targets of this type are still recharging, i.e., they are part of {b_2,…,b_τ+1}). 
 If no such target type exists, we set  J[i_1,…,i_n_χ,b_1,…,b_τ+1] to -∞.
 Otherwise, for each t∈ [z], we check max_s∈ S∪{∅} J[i_1,…,i_γ_j_t-1,…, i_n_χ,b_2,…,b_τ+1,s] and let the entry be the maximum of these values. 

Analogously, if b_1= ∅, we let 
γ_j_1, …, γ_j_z be the target types where {b_2,…,b_τ+1} contains all the sensors that are capable of sensing targets of this type. 
If no such target type exists, we set the entry to be -∞. 
Otherwise,  for each t∈ [z], we compute max_s∈ S∪{∅} J[i_1,…,i_γ_j_t-1,…, i_n_χ,b_2,…,b_τ+1,s] plus the utility of targets of type γ_j_t and let the entry be the maximum of these values. 

The correctness follows from the fact that the initially stated invariant is preserved throughout the algorithm. 
Observing that computing each table entry takes 𝒪(n_χ· (k+1)) time, the claimed running time of 𝒪(n_χ·(∏_i=1^n_χ (ℓ_i+1))· (k+1)^τ+2) follows. 

We assume that the ordering of targets is t_1,…, t_n and solve the problem via dynamic programming. 
The idea is to construct a valid sensing plan iteratively by going through the arriving targets one at a time. For each target, we either decide that it will not be sensed or assign it to one of the sensors so that the resulting plan is still valid.  
Our key observation to bring down the time and space complexity of the dynamic program is that we do not need to store the full current sensing plan to ensure the validity of the resulting plan. Instead, it is sufficient to know for each sensor whether it has sensed a target in the last τ steps. 
More formally, given a valid sensing plan ψ that has been constructed by iterating over the first i∈ [n] targets, we only store the following necessary information: 


  * the id i∈ [n] of the last considered target,

  * the value of all targets {t_1,…, t_i} that have not been assigned to a sensor in ψ, i.e., ∑_t_j∈{t_1,…, t_i}∖∪_s∈ Sψ(s) v_j,

  * the sensors the last τ targets have been assigned to, i.e., for each j∈ [max(1,i-τ),i], ∅ if t_i-j+1 has not been assigned to a sensor and ψ(t_i-j+1)^-1 otherwise. 

For our dynamic program, we create a table J[i,b_1,…,b_τ+1]∈ℕ for each i∈ [n] and b_1,…,b_τ+1∈ S∪{∅}. 
J[i,b_1,…,b_τ+1] stores the minimum value of surviving target from t_1,…, t_i induced by a valid sensing plan ψ that for each j∈ [max(1,i-τ),i] assigns target t_j to sensor b_i-j+1  if b_j≠∅ and to no sensor if b_j=∅. 
In this case, we say that ψ witnesses the table entry.
If no such sensing plan exists, J[i,b_1,…,b_τ+1] is ∞. 
Clearly, 
the answer to the problem is min_b_1,…,b_τ+1∈ S∪{∅}J[n,b_1,…,b_τ+1]. 




 §.§ Proof of <Ref>

*

We model an instance ℐ of Best Blue Response as follows.
For each target i∈ [n], we add an integer variable z_i that encodes the position in which the target appears in the final ordering. 
We add linear constraints so that z_i∈ [1,n] and z_i≠ z_i' for all i≠ i'∈ [1,n] (see <Ref>).

Next, similar as in <Ref>, for each i∈ [n] and j∈ [k+1], we add a binary variable x_i,j. Setting x_i,j to one means that t_i is detected by sensor s_j or in case that j=k+1 that the target makes it unsensed through the channel. 
Accordingly, the objective becomes: 

    max∑_i∈ [n] v_i · x_i,k+1.

For each target i∈ [n], we impose that: 

    ∑_j∈ [k+1] x_i,j=1.

Moreover, we impose x_i,j≤ D_i,j for each i∈ [n] and j∈ [k], enforcing the sensor capabilities. 

To ensure that the recharging times of sensors are respected we add the following set of constraints. 
For each, j∈ [k] and i,i'∈ [n], we add 

    |z_i'-z_i| ≥ -n(2-x_i,j-x_i',j) + τ.

This ensures that if x_i,j=1 and x_i',j=1, then i and i' are placed far enough away from each other, while otherwise the condition is vacant. 
To realize the absolute value from the above equation, we have to introduce another set of binary variables o_i,i' for i,i'∈ [n] and add the constraints: 
z_i'-z_i+n· o_i,i'≥ -n(2-x_i,j-x_i',j) + τ and z_i-z_i'+n· (1-o_i,i)  ≥ -n(2-x_i,j-x_i',j) + τ. 

If a target i is sensed by sensor j then due to the recharging time sensor j will not be able to sense other sensors, thus i “protects” some targets from being sensed by sensor j. 
To capture this information, for each i,i'∈ [n] and j∈ [k], we add a binary variable y_i,i',j that is equal to one target i is sensed by sensor j and because of this j cannot sense i'. 
To ensure this, first, for each i∈ [n] and j∈ [k], we add: 

    ∑_i'∈ [n] y_i,i',j≤ n· x_i,j

(a target i can only protect other targets if the target is sensed by the corresponding sensor).
Moreover, for each i,i'∈ [n] and j∈ [k], we need to make sure that if y_i,i',j=1, then 0≤ z_i'-z_i ≤τ (to exploit of the recharging constraint). For this, we add constraints:
    -n(1-y_i,i',j) ≤ z_i'-z_i≤ n(1-y_i,i',j) + τ.
 

Lastly, we need to make sure that a target will survive until step j if x_i,j=1 i.e., the target needs to be covered by other targets for all sensors that are capable of sensing it placed before j. Note that this together with the first constraint (∑_j∈ [k+1] x_i,j=1) in particular implies that each target is sensed by the first sensor it passes which is not recharging, thereby successfully encoding the greedy behavior of the sensors. 
Specifically, we add the following set of constraints for each i∈ [n] and j∈ [k+1]: 

    ∑_t∈ [j-1]: D_i,t=0 1+    ∑_t∈ [j-1]:D_i,t=1∑_i'∈ [n] y_i',i,t -(j-1)
       ≥ -n(1-∑_t=j^k+1 x_i,t).





§ ADDITIONAL EXPERIMENTAL RESULTS


To begin, we generate a Figure <ref> illustrating the average utility for  across varying probabilities of D_i,j = 1. This visualization aims to demonstrate the impact of the probability of D_i,j = 1 on 's utility under the Default  game settings.



In Table <ref>, we show the effectiveness of our ILP solver. Notably, it demonstrates the capability to solve large instances very fast, completing the task within a second.
This proficiency has been valuable in the development of heuristic algorithms for bilevel optimization in the search for identifying the Stackelberg equilibrium. We also show that our ILP solver can solve extensive instances involving hundreds of targets within a single hour in Table <ref>. 









In the remaining parts of this section, we explore a new game setting (Append) for generating ESGs. The new method is similar to Default, with the distinction that each element D_i,j= 1 with a 0.5 probability. In essence, this configuration increases the likelihood of each target being sensed compared to the Default setting, thereby resulting in a stronger  sensing model.



 §.§ Computing the Follower Strategy








Similar to the Default game setting, we show the scalability results  of Append in table <ref>.





In this new setting, an intriguing observation emerges as the number of sensors increases significantly: the runtime of our ILP decreases, given that the abundant sensors can effectively sense all targets (e.g., when the number of sensors increased from 10 to 20.).



 §.§ Additional Results from Computing the Stackelberg Equilbrium
 

In this subsection, we begin by showing Figure <ref>, illustrating the impact of the choice of ration (μ) on the SA algorithm discussed in Section <ref>. 
Specifically, we present and test three quadratic-time greedy heuristics to build the sensing plan ψ iteratively by trying to sense the most valuable targets first. 
We consider the targets in decreasing order of their value. Let T' be the already processed targets and t_ℓ the target to consider. Moreover, let  S'⊆ S the set of sensors s so that ψ remains a valid sensing plan after adding t_ℓ to ψ(s), i.e., the sensors that are currently free to sense t_ℓ. 
If S' is empty, then we do not assign t_ℓ to any sensor, implying that it will be won by . 
Otherwise, we apply three different methods to decide which sensor from S' to pick: 

    
  random Randomly select a sensor from S'.
    
  remaining_value Pick the sensor s from S' that has the lowest summed value of remaining targets that s is capable of sensing, i.e., _s_j∈ S'∑_t_i∈ T ∖ T': D_i,j=1 v_i.
    
  harm Pick the sensor s from S' that where assigning t_ℓ does the least harm: The harm that t_ℓ does to s in ψ summed value of remaining targets that s is capable of sensing that it can no longer sense when t_ℓ is assigned to s, i.e., _s_j∈ S'∑_t_i∈ T ∖ T': D_i,j=1  and  |i-ℓ|≤τ v_i.




We also present the bilevel ILP's scalability results under the new Append game setting in Table <ref>.





We also provide a comparison of heuristic algorithms under the new Append game setting in Table <ref>.







 §.§ Additional Results from Non-Coordinated Sensing

In Table <ref>, we present the scalability results of the ILP that solves for optimal  responses in the non-coordinated  sensing setting under Append game setting. 





For the non-coordinated sensing setting, we also provide a comparison of heuristic algorithms under the new scenario in Table <ref>.





  §.§.§ Power of Coordination

Finally, in Table <ref>, we present the power of coordination results under Append game setting with  that has stronger sensing capabilities.  


Moreover, in scenarios with large instance sizes, such as n=75, k=10, τ=5, where the optimal (bilevel) ILP is unsolvable, we can compare 's approximately optimal utility under the best heuristic algorithms. Specifically, as shown in Table <ref>, the average 's utility under the SA algorithm is 2.26 ± 1.13 for 9 instances. Given the same 9 instances, the average 's utility when escaping from non-coordinated sensing is 4.43 ± 1.41, which is approximately twice the value observed in the coordinated sensing setting. Once again, due to the constant-sum nature of this game, the utility loss for  in transitioning from non-coordinated sensing to coordinated sensing essentially represents the utility gain for , highlighting the power of coordination.



 Paper Title: Condorcet's Jury Theorem with Abstention.


 
 Condorcet's Jury Theorem with    Abstention
    B. A. Malomed
    
===========================================

§ ABSTRACT 
The well-known Condorcet's Jury theorem posits that  the majority rule selects the best alternative among two available options with probability one, as the population size increases to infinity. We study this result under an asymmetric two-candidate  setup, where supporters of both candidates may have different participation costs.  

When the decision to abstain is fully rational i.e., when the  vote pivotality is the probability of a tie,  the only equilibrium outcome is a trivial equilibrium where all voters except those with zero voting cost, abstain. We propose and analyze a more practical, boundedly rational model  where voters overestimate their pivotality, and show that under this model, non-trivial equilibria emerge where the winning probability of both candidates is bounded away from one. 

We show that when the pivotality estimate strongly depends  on the  margin of victory, victory is not assured to any candidate in any non-trivial equilibrium, regardless of population size and in contrast to Condorcet's assertion. Whereas, under a weak depedence on margin, Condorcet's Jury theorem is restored. 






§ INTRODUCTION

background: Condorcet's result.. 
Consider a population of N voters voting over two alternatives  A and B,  with A being the better alternative  according to some pre-defined criterion. Consider further that the preference of each individual voter is determined independently by an outcome of a coin toss with bias p > 0.5 in favour of the better alternative. That is, each individual voter supports alternative A with probability p and B  with the probability 1-p. Under this setting, the famous Condorcet's Jury Theorem states that the majority rule selects candidate A with probability tending to one when population size increases to infinity. 



gap/problem: condorcet result assumes everyone votes...this is not the case in practice...presidential elections have around 60% turnout

An implicit assumption in Condorcet's  theorem is that everyone votes, or at least that the decision  to vote 
  does not depend on one's preference over alternatives. 
In contrast, in many practical situations such as  political elections, or a local or national referendum, abstention is found to be a  common and prominent phenomenon. For instance, the voter turnout in United States presidential elections has been around 52%-62% over the past 90 years <cit.>. Abstention is also observed to be a significant phenomenon  in small-scale lab experiment <cit.>.





 paradox of voting..why people vote...do those who abtain affect the outcome of the election ... what happens when the size of population grows to infinity 
 From a rational, economic point of view, the surprise is not that some voters abstain, but that they vote at all a.k.a. `the paradox of voting'. As Anthony Downs claimed already in 1957,  a rational voter weighs the benefit of voting (which realizes only if the voter is pivotal) against the cost. When  the  size of the electorate is large, the expected benefit derived from affecting the outcome of the election (i.e. being pivotal) is too small to induce voting  from a significant fraction of voters <cit.> giving rise to the paradox of voting. 

 
paradox of voting vs condorcets result... reality is between... does there exists a boundedly rational voting model that explains large turnout and at the same time does Condorcets result holds..strategic abstention 
While Condorcet's result holds under the assumption that everyone votes,  Down's theory of rational voting suggests that only near-zero cost voters vote in large-scale elections. 

Our goal in this paper is to understand the equilibria that arise from a plausible, heuristic-based abstention model, in an attempt to reconcile the theoretical predictions with the moderate turnout rates we see in practice. The most important questions we focus on are: (1) how many equilibrium points are induced by the abstention  model and where are they located? (2) does the winning probability of the better candidate approach 1 as predicted by Condorcet's Jury theorem  in any of these equilibria?  and (3) what happens to the equilibrium points as population size increase? 


 We now present some important models for voter turnout presented in the literature and then move on to our proposed voter turnout model that depends on heuristic-based perceived pivotality of individual votes.




 


review of theoretical voting models
  
  
  
  The Calculus of Voting model:  
  Originally proposed by <cit.> and later developed by 
 <cit.>, this  model   attributes each  voter's decision to abstain from voting to expected cost-benefit analysis. Let p_i denotes the perceived pivotality of voter i, whereas _i denotes the  personal benefit she receives if her preferred candidate wins an election,   _i denotes the social benefit she receives by performing a civic duty of voting and _i denotes   costs of voting she incurs. These costs  include the cost of obtaining and processing information and the actual cost of  registering and going to polls (see also <cit.>  for discussion of voting and rational choice).  A voter i votes if and only if 
 
    p_i·_i + _i ≥_i.
 

 The calculus of voting  model considers  p_i to be the probability that i's vote would change the outcome a.k.a. probability of an event that all voters except i reach a tie. The tie probabilities are derived from the aggregated stochastic votes, and thus the pivot  computation and subsequent equilibrium analysis quickly become intractable as the number of individual voters increases. 
 
 

Aldrich: there is a paradox in empirical data, where studies based on aggregate data show a strong connection of p and turnout, and studies based on survey data show the opposite. I'm not sure I understand the explanation, or the difference between these studies :(   It may suggest some evidence that voters mis-estimate their pivotality


 
 Myerson and Weber studied voting equilibria in  a 3 or more candidate elections by fixing a distribution over preferences and considering a large population sampled from this distribution, so that candidates' scores are multinomial variables <cit.>. While tie probabilities vanish in the limit, they observe that  the relative tie probabilities for each pair of candidates can still be compared and thus voters' best responses are well defined. 
 In a more recent work, Myerson <cit.> suggested another relaxation by approximating candidates' scores with Poisson distributions.

 Crucially, in the above models, the tie probabilities used for voters' strategic calculations are derived directly from the vote distribution, either exactly or approximately.
    When  the size of the electorate is large, the tie probability  
 p ≈ 0. If voting is costly, these models predict very low turnout—essentially  that only the zero-cost voters vote when the population increases to infinity.

 Heuristic pivot probabilities: Some models assume that the voters act based on estimated or even completely wrong beliefs.  One such heuristic is derived by Osborne and Rubinstein's sampling equilibrium <cit.>, where each voter estimates candidates' scores based on a small random sample of other voters.  That is, even with a large  population when voters compute their beliefs based on limited information, there is a non-negligible fraction of voters who believe they are pivotal. 

 the pivot probability is like Binomial but with the sample size k instead of m. If k is fixed then p≅ (1-m^2)^k which is sort of tie-sensitive since there is no dependency on n at all. If k is increasing with n then it is vanishing. 








  

  

 Non-probabilistic uncertainty: Other models stir away from  probability calculations and consider other voters' heuristics, based on dominated actions, minmax outcomes or regret <cit.>. 
The margin of victory plays a major role in the voter's perceived  pivotality. Controlled experiments on voters' response to poll information show that strategic  voting is more frequent when the winning margin is small <cit.>. 
 <cit.> show  that voters' actions are more consistent with various heuristics based on the margin than with `rational' utility maximization models.  The key point in these models is that voters form a heuristic belief of their pivotality based on the voting profile, and that an `equilibrium' means the belief justifies itself, not that it is necessarily correct <cit.>.  


  here too we can write much less about these models. the key point is that voters form a heuristic belief of their pivot probability based on the voting profile, and that an equilibrium means the belief justifies itself. Another such model is by Rubinstein (see reference and details in my book)
  Minimax regret model (<cit.>):
  The regret is a loss incurred by the voter by not taking a utility maximizing action in a given state of the world. State-of-the-world are often represented in terms of margin of victory <cit.>. That is, if a given voter's preferred candidate looses an election by 1 vote, then the voter will regret the decision to abstain. Similarly, if the voter votes for her preferred candidate and finds out that his candidate wins an election by a large margin he will still incur a positive  regret. The true state-of-the-world is unknown and voters must make their choice without true state-of-the-world. The minimax regret model hypothesize that the voters take a  decision that maximizes his/her  utility in the worst case scenario.       While the minimax regret captures the dependency of voter turnout on margin of victory, it ignores the dependence on the population size. 

  Local Dominance Model <cit.>
  In this behavioral heuristic model, each voter considers a set of possible states of the world without assigning probabilities to them. This set is constructed based on voters information about candidate's likely  votes from sources such as polls, media etc. In a voting equilibrium, all voters vote for candidates not dominated within the set of possible states—where again local domination is determined by the margin of victory.
  Analyzing every such model separately would be tedious and leave us with an isolated set of narrow results. Instead, we identify the main factors common to all of these models, and suggest a flexible framework that captures a wide range of possible rational and boundedly-rational behaviours, without committing to one model in particular.[Although some of our theoretical results make stronger assumptions for technical reasons.] 
We still consider that the voters seek to maximize their own expected utility as in the calculus of voting model (Eq. (<ref>)). However, in contrast to the calculus of voting model, in our model voters decide to abstain based on heuristically estimated perceived pivotality. This estimate is also a function of the margin of victory as in the latter models we mentioned. Our proposed heuristic model for computation of perceived pivotality is inline both with empirical <cit.> and experimental <cit.> findings, and with the models developed therein that aim to explain high turnout in  elections. 









 
 We emphasize here that we do not claim to provide a new universal model of strategic voting.  Rather, we want to capture a broad class of models that share similar properties (dependency on margin and population size) in order to provide results that are not model-specific. 





  
Contributions
We show that the set of equilibrium points  induced  by proposed abstention model consists of equilibrium points predicted by the fully rational  (such as calculus of voting) model that satisfy Condorcet's Jury Theorem (CJT) and some more non-trivial equilibrium points.  Our results show that there are non-trivial equilibria where the popular  candidate is more likely to win, but the winning probability is bounded away from 1. 
This probability, perhaps surprisingly, depends only on the distribution of voting costs in the population. 
The bottom line of this paper  is the result that under a plausible abstention model, induced non-trivial equilibrium points evade both; Down's paradox of voting (only a small  fraction of homogeneous voters vote) as well as CJT (the win probability of popular candidate approaches 1).      

Similarly to other margin-based heuristics in the literature (see below) the model allows for equilibria with a significant fraction of active voters. The main question we  set out to answer is whether the conclusion of Condorset's Jury theorem holds in these equilibria since we already know the premise does not (i.e. there are abstentions). 


A popular candidate has less support among low-cost voters, she is disproportionately affected by  abstention compared with  the less popular candidate. We show that under certain settings with the heuristic abstention model, as the population size increases,  the rate at which  a popular candidate loses her supporters dominates  the rate of  increase in  her win probability. need a sentence or two here

We introduce tie-sensitive pivot probability model. 

We show in Section  that the proposed bounded rational model predict  that the fraction of voting population  decreases. However its rate is very slow and it is lower  bounded by a  constant fraction of voter population. This property is  consistent with the empirical observation that the relative decrease in the participation level with the size of population is much slower than predicted by fully rational models. 


We also show that,  under some mild conditions, a large election is closely contested (Section ). Finally, we show that the probability that a popular candidate wins an election is bounded away from 1. This result highlights the fact that even in a perfectly rational voters behavior model and  without any social, personal  and psychological explanations,  one can expect surprises in election outcomes.    




  
Organization of the Paper
 In  Section <ref>, we provide preliminaries and describe our model in more detail. 
In Section <ref> we propose a specific tie-sensitive heuristic formally. Though this is one of the conceptual contributions of this work, as noted earlier, it is not the main goal of this work.  Our primary goal is to check if a plausible turnout model can also explain  surprises in large elections. We present a detailed example in Section <ref>, that demonstrates that the Jury theorem does not hold    under the  tie-sensitive model, regardless of the number of active voters. 
In Section <ref> we extend the example to any election instance with linear  support  functions. Informally, a support function specifies the probability of actively supporting a given candidate as a function of the cost. 
Finally in Section <ref> we empirically show that most of these results hold in a broader class of tie-sensitive models.   






Now the structure should be (based on existing text):

    
  * Existing models (Calculus of voting, Poisson games, Minmax regret, Local dominance)
    
  * Our general model (distribution of costs, the pB-C formula)
    
  * vanishing pivotality vs. tie-sensitive (describe informally, justify with evidence from the literature)
    
  * other related work









§ MODEL


We study a two-candidate (referred to as A and B)   election with   N voters.  Each  voter  is either a supporter of A (prefers candidate A, i.e. A ≻_i B)  or a supporter of B. 
We adopt the classical calculus of voting model  as follows. We denote  as  c_i := max{ 0,_i - _i/_i} the effective cost of voting for voter i  with _i, _i and _i as defined in Eq. (<ref>). 

The core supporters (voters with zero effective voting cost) derive more utility from voting for their preferred candidate  than costs incurred  in participating in the voting process.  Voters with non-zero effective costs, on the other hand, vote only if the perceived pivotality of their vote exceeds the voting cost.  For mathematical convenience, we  normalize the effective cost of voting to lie between 0 and 1.  




 
  
  





We consider the effective cost of voting and the preference of voter i as  an independent sample from a commonly known  joint distribution 𝒟 over [0,1] ×{A,B} and is denoted by tuple (c_i, T_i). We assume that  has no atoms, expect possibly at c=0 and/or c=1. 



Without loss of generality, we assume  that (weakly) more voters support A  in expectation. 

We therefore consider A as the “better” or “popular” candidate, and see the outcome where A is elected as preferable.

As discussed above, Condorcet Jury Theorem states that  if all voters vote, the probability that A wins goes to 1 as the number of voters grows.
However, this is clearly not always true if A voters are more likely abstain than B voters. This, in turn, may depend both on individual voters' costs, and on voters' perceived pivotality (which we assume to be the same for all voters). 





  
Perceived pivotality
In the discussion above, we highlighted that the two most important factors that voters use to determine the probability of various outcome are (1) the size of voting population, n; and (2) the margin, m. Our simplifying assumption (following e.g. <cit.>) is that voters only consider n and m as expected values. Thus, a general pivotality model is specified by a function p, which maps any pair of n and m to a pivot probability p(n,m)∈[0,1], and is continuous and non-increasing in both parameters. We later provide several concrete pivotality models, which can either approximate the real pivot probability or reflect subjective beliefs. 

 

  
Support functions
While   contains all necessary information regarding the distribution of costs in the population, we would like to present costs in a more intuitive way. 

For T∈{A,B} 
let s_T:[0,1]→ [0,1]  be continuous, non-decreasing function, where s_T(c) should be read as the fraction of the entire population (except voter i) that prefers T and  has individual cost at most c. We call s_T the support function of T, and note that it does not depend on the identity of voter i.
propositionpropOne
    Any distribution  induces a unique pair of support functions s_A,s_B with s_A(1)+s_B(1)=1, and vice-versa.




 §.§ Issues and Elections

An issue is a triple I=(s_A,s_B,p), i.e. it describes the support in the population for the two possible decisions A and B on the issue, as well as the perceived pivotality of voters on that issue.



An issue does not yet induce a valid game, since the set of players is not well defined. 
An Election E=(I,N) is an issue I together with a specific number of players N, who are assumed to be sampled i.i.d. from distribution  (or, equivalently, from the support functions).

An election is therefore a game, where the type of each voter/player is a random variable. Each player has two actions: vote for her preferred candidate or abstain.[Voting for the less preferred candidate is strictly dominated so we ignore this option.] A voter of type (T_i, c_i) suffers a cost of c_i if she votes, irrespective of the outcome, and gains 1 if her preferred candidate T_i wins. 

A pure strategy profile in this game maps voter types to actions (vote/abstain).

We argue that pure Bayes-Nash equilibria in this game have a very simple form, in which all voters with cost lower than some threshold vote.










propositionPropEquilibrium(Election Equilibrium) For a given election  (s_A,s_B,p,N),  every pure Bayes-Nash equilibrium can be described by a single number c∈[0,1] s.t.:

    n(c)   =(s_A(c)+s_B(c))N 
    
        m(c)   =|s_A(c)-s_B(c)|/s_A(c)+ s_B(c)
    
        c   =p(n(c),m(c)),

    and each voter i votes iff c_i≤ c.




For technical reasons we will assume throughout the paper that all derivatives of of s_A,s_B are bounded. 

Denote by C^*(I,N)⊆ [0,1] the set of all equilibrium points of election E=(I,N).


propositionpropTwo
Every election has at least one equilibrium.

The proof follows from the fact that f(c):=p(n(c),m(c)) is a continuous function from [0,1] onto itself and therefore must have
a fixed point.



  
Issue equilibrium
An issue can be thought of as a series of elections, one for every population size N.

An equilibrium of issue I is a series of points c=(c_N)_N s.t. ∀ N, c_N∈ C^*(I,N), and c has a limit. We denote the limit by c^*.


For an issue equilibrium c with limit c^*, if  c_N>c^* for all N we say that c is a right equilibrium. Similarly, if  c_N<c^* for all N we say that c is a left equilibrium. 







  
Trivial equilibrium
An equilibrium of issue I is trivial if its limit is c^*=0, meaning only core supporters vote. Otherwise it is nontrivial. Clearly a trivial equilibrium is always a right equilibrium. 



  
Jury theorems For a given election E=(I,N) and a given equilibrium c∈ C^*(I,N), we can compute the probability of various events. One such event of interest is the victory of the popular candidate A.[We emphasize that we, as `outsiders' to the election, care about the actual probability of the event, which is not affected by the perceived pivotality model p, once c is determined.]

Formally, we define for a given election E=(I,N) and equilibrium c∈ [0,1], a random variable  counting the number of active votes for A (and likewise for B):

    V_A := ∑_i∈ Nc_i≤ c ∧ T_i=A,

where {(c_i,T_i)}_i∈ N are voters sampled from .
 Note that V_A is a Binomial variable with parameters N and s_A(c), and expected value s_A(c)N.

We further denote the Winning Probability of A in equilibrium c of election E=(I,N) as

    𝕎ℙ_A(I,N,c):=(V_A > V_B| I,N,c).


Finally, for any issue I with issue equilibrium c, we define

    𝕎ℙ_A(I, c):=lim_N→∞𝕎ℙ_A(I,N,c_N).


We say that the instance admits a jury theorem at c^*∈ [0,1], if there is  an instance equilibrium c with limit c^* s.t. 𝕎ℙ_A(I, c)=1.

Similarly, I admits a non-jury theorem at c^* if 𝕎ℙ_A(I, c)<1, meaning that regardless of the size of the population, there is some constant probability that the better candidate A will lose.  I admits a strong non-jury theorem if 𝕎ℙ_A(I, c)=1/2.








Abstention Model:
Prior to voting, every voter, given a cost distribution 𝒟 and electorate size N,  determines the perceived pivotality of her vote. Each voter i then privately realizes the tuple (c_i, T_i) sampled independently from 𝒟. With the knowledge of N, (c_i,T_i) and 𝒟, a voter i then makes a choice of whether to vote or to abstain. This choice crucially depends on the perceived pivotality p_i of her vote. We consider that p_i is  a function of the expected[Expectation is taken with respect to  distributions 𝒟_A and 𝒟_B.] number of votes  for each candidate and the expected winning margin.  Note that since 𝒟 and N are fixed and commonly known we have, p_i = p for all i ∈ [N].   In particular, the perceived pivotality p_i does not depend on the voters' type T_i. However, note that her choice to abstain may depend on her type.  


 A voter i votes if p≥ c_i and  abstains otherwise. As mentioned previously, the perceived pivotality p denotes  the voter's belief about the extent to which a single vote (her vote) is important. Thus a voter votes if  she perceives herself as sufficiently pivotal.  While c_i  is privately known, p is common knowledge and this value depends only on the aggregate voting profile. 

Support Functions:


 be the expected  fraction of supporters of candidate T with cost at-most c. We call s_T(.) the support function of candidate T.   

We consider that s_A, s_B:[0,1]→ [0,1]  are continuous, non-decreasing functions such that  s_A(c)+s_B(c)≤ 1 for all c ∈ [0,1] with   s_A(1)+s_B(1)= 1 (see Figures. <ref> and <ref>). An election instance ℐ is thus  given by tuple ⟨ N, s_A, s_B  ⟩. 





 Given  c ∈ [0,1],  
the expected  number of voters with effective voting cost  at most c is given by  
    n(c)=(s_A(c)+s_B(c))N.


We remark here that n(c) is a  point estimate of the voter turnout  with  cost threshold c.  Similarly, the estimated margin of victory m is derived directly from the number of active voters n and the support functions as  

    m(c)=|s_A(c)-s_B(c)|/s_A(c)+ s_B(c).


We omit the argument c from m and n whenever it is clear from  context. 
A  Perceived Pivotality Model (PPM) is a decreasing function p:ℕ× [0,1] →ℝ, mapping the number of active voters n and the estimated margin of victory m to a perceived pivot  probability.   

















  

 
 


As mentioned earlier, perceived pivotality is each agents' subjective assessment of the likelihood that her vote is pivotal. In the next section, we briefly overview  fully-rational and heuristic-based  PPMs  studies in the literature and propose tie-sensitive PPM studied in the rest of the paper. 




§  PERCEIVED PIVOTALITY MODELS








We first describe models where the perceived pivotality p(n,m)  represents the actual tie probability V_A=V_B (so a single vote would determine the  winner of the election). 
We present two  close approximations of the actual tie probability that have been used in the literature, and that can be presented as pivot functions in our model. 

  


 §.§ Fully Rational models


We should have a proposition for the `correct' pivot probability (which can be computed either as the expected ti-probability over n' sampled from Bin(N,n/N); or as the probability that in the multinomial distribution (N,(s_A,s_B,1-s_A-s_B)) the first two elements are equal). 
This is easy, and then we need another proposition bounding this by Θ(p(n,m)) of the Binomial PPM.


 




The first model maintains the Binomial distribution, but assumes that the n active voters are the entire population,[The true pivot probability can also be written as a function of m and n: it follows the Binomial model p(n',m) from Eq. (<ref>), but where the number of active voters n' is a random variable sampled from Bin(N,n/N). Numerically this function is nearly identical to Eq. (<ref>), and in particular it has strong vanishing pivotality.needs proof]
 as in the original Condorcet Jury Theorem and in early Calculus of Voting models <cit.>. 
 [Binomial PPM] 
 
    (n,m)=_x∼Bin ( n, (1+m)/2 )(x = n/2).

 

  A later model by <cit.> suggested drawing the scores of each candidate independently from a Poisson distribution. 

 
  [Poisson PPM]
  
    (n,m)=_x_A∼Poisson((1+m)n/2)
     x_B∼Poisson ((1+m)n/2)(x_A=x_B)

 
Conceptually, the Poisson model is more appropriate in situations where voters can abstain (as the total number of active voters is not fixed). 
However p(n,m) behaves very similarly to the Binomial model, and for our purpose they are almost the same.
In fact, both models belong in a much larger class of PPMs:

 We say that a PPM has strong [weak] vanishing pivotality if   lim_n→∞p (n,m)=0 for all  [m > 0] m≥ 0.
 

As we will later see, instances with weak vanishing pivotality always admit a trivial equilibrium. 
Clearly at the trivial equilibrium, Jury theorems are irrelevant: the candidate with more core support always wins with probability that approaches 1 as the population grows, regardless of who is more popular overall. It is not hard to verify (e.g. using Stirling approximation) that in both the Binomial and Poisson PPMs, p(n,m)=Θ(1/√(n)) for  m=0, and decreases exponentially fast in n for any m>0. 





 For large n using Stirling's approximation, we have  
 
    (n,m)    =  n ⌊n/2⌋ ( s_A(c)/s_A(c) + s_B(c) )^n/2 ( s_B(c)/s_A(c) + s_B(c) ) ^n-n/2
    (n,m)   ≅√(2/π n) (4 s_A(c)/s_A(c) + s_B(c)s_B(c)/s_A(c) + s_B(c) )^(n/2)
       = √(2/π n)((1+m)(1-m))^(n/2)   < 1/√( n)


    (n,m)   ≅√(2/π n)((1+m)(1-m))^(n/2)   < 1/√( n)


We note that the pivot  probability  decreases at rate 1/√(n) for m≈ 0, and much faster for a fixed positive margin.  Thus the model suggests that in a large election only extremely low-cost voters would vote. That is, in large elections the outcome is determined only by the fraction of core supporters of the candidates. 

Note that we would like to say that p(n,m)=Pr(V_A=V_B|N) however this is not accurate, as V_A,V_B are independent Binomial variables, and while V_A and x have the same expected value (s_A(c)N=s_A(c)/s_A(c)+s_B(c)n), the variance of V_A could be much larger. One could have argued that using Pr(V_A=V_B) would be `more rational', but this would only make pivot probability smaller so would not change much.
 

The Binomial model introduces a dependency between candidates' scores that is difficult to work with. Hence
  














 



 §.§ Tie-Sensitive models


We saw that even in the rational model (which have strong vanishing pivotality), the case of m=0 is different, with a substantially higher probability to be pivotal. 

A simple and perhaps more cognitively plausible assumption is that voters consider themselves pivotal if the margin is small enough, regardless of the number of voters.[An alternative justification that maintains full rationality is an argument that is sometimes made about large elections being `more important', as a possible explanation for the paradox of voting <cit.>. While this suggestion fails to explain high turnout in general (unless importance increases exponentially with n), even a modest increase in importance of Ω(√(n)) is enough to justify Tie-sensitivity.] 
The dependence of such tie-sensitive models on the margin m and number of active voters n is captured by the following two properties. We can see these properties as an abstraction of various bounded rational behaviors described in the introduction.
 
 
Given m >0, there exists n_0 ∈ℕ such that p(n,m) ∝1/√(n) for n≥ n_0.

 
Property <ref> captures the dependence of perceived pivotality on the population size, which is  aligned with the fully rational models.[The √(n) reflects the standard deviation of summing n independent variables.] 


In contrast, the heuristic dependence of PPM as decreasing in the margin m,  diverges from the rational models: 
There exists α∈ (0,∞) such that
for any fixed n ∈ℕ,  p(n,m) ∝1/m^α. 
 

The simplest model that satisfies both properties is obtained by directly combining the dependency on n and m:



 (n,m)=min{1,1/m^α√(n)} for some α>0. 







For the theoretical analysis, we assume in the remainder of this section that α=1 (linear margin PPM), and extend these results using simulations to other values of α in Section <ref>. We are interested in characterizing the equilibria as N →∞. 


















 



 


 



 §.§ Equilibiria



A cost value c ∈ (0,1) is called a pivot point if s_A(c) = s_B(c). 


Note that  the set of pivot points may be empty.



We say that a PPM has q-tie-sensitive pivotality if p(n,0)≥ q for all n.

That is, if the expected outcome is a tie, everyone thinks they are pivotal at least to some extent, regardless of the number of active voters. 




For example, <cit.>  suggested a model where each voter decides whether they are pivotal based on a fixed number of other voters they sample i.i.d. from the population. This induces a PPM that is identical to the Binomial one, except that the parameter k is used instead of the number of voters n.  Thus for a fixed k, the `Sampling PPM' is q_k-tie-sensitive for some q_k=Θ(1/√(k)). 

Another tie-sensitive PPM is induced by assuming a polynomial dependency on both m and n. 
[Polynomial PPM] For α,β>0

    p(n,m)=min{q,1/m^α n^β}


It is immediate from the definition that the Polynomial PPM is both q-tie-sensitive and has a weakly vanishing pivotality. 
One particularly natural choice is when β=1/2 (so that the dependency on n is similar to the previous models), and α=1, i.e. linear dependency on the margin.



§ CHARACTERIZING EQUILIBRIUM LIMITS

We begin by characterizing the trivial equilibrium. 



propositionpropThree
    Suppose s_A(0)+s_B(0)>0. Any  instance with strongly vanishing pivotality admits only  a trivial equilibrium.


propositionpropFour
    Suppose s_A(0)≠ s_B(0). Then any instance with weakly vanishing pivotality admits a trivial equilibrium.




The proofs of Propositions <ref> and <ref> are given in Appendix. Intuitively, the proofs of both the propositions rely on the fact that we can always consider sufficiently large population size N, thereby making p(n,m) small enough. 

Next we  show that that  the intersection points of the support functions form the limiting points of equilibrium points.  


For a given pair of support functions s_A,s_B, a pivot point is any  c∈(0,1) where s_A(c)=s_B(c)>0, and s_A(c+)≠ s_B(c+) for all >0. 

A pivot point is a cost c such that if  all voters with c_i≤ c vote then both candidates have equal support, and thus the expected margin at a pivot point  is 0. 


    Let I be an instance with a PPM a that has weakly vanishing  and q-tie-sensitive pivotality.  Any pivot point c^*<q has a right equilibrium with limit c^*. Moreover, for support functions that intersect at a finite number of points, the limit of any equilibrium is either 0 or some pivot point c^*.


We start with existence. Let c^*<q be some pivot point of I, and let δ>0. We need to show there is some N_δ and some c_δ∈(c^*,c^*+δ) s.t. c_δ∈ C^*(I,N_δ). 


Since all derivatives are bounded, there is some open interval (c^*,c^*+t) where s_A,s_B differ, and w.l.o.g. s_A(c)>s_B(c) for any c∈ (c^*,c^*+t). Let δ:=min{t,δ} and note that by the definition of pivot point, :=m(c^*+δ)>0. Also, n(c^*+δ)=(s_A(c^*+δ)+s_B(c^*+δ))N<N. Thus

    p(n(c^*+δ),m(c^*+δ))≤ p(N,) 0,

so there is some N_δ for which p(n(c^*+δ),m(c^*+δ))<δ.

On the other hand, 

    p(n(c^*),m(c^*))=p(n(c^*),0) ≥ p(N,0)≥ q > c^*.

Define f(c^*+x):=p(n(c^*+x),m(c^*+x))-(c^*+x) then f is continuous in the range x∈[0,δ) with f(0)>0 and f(δ)<0. From intermediate value theorem there is some x where f(x)=0 and thus c_δ:=c^*+x is an equilibrium of (I,N_δ).

In the other direction, assume towards a contradiction that there a nontrivial equilibrium with limit c^* that is not a pivot point. Note that by our assumption of finite intersection points, and due to bounded derivatives, s_A(c)-s_B(c)>>0 in some interval [c^*-δ,c^*+δ].
Thus in any point in this interval the pivotality goes to 0 for sufficiently large N, and in particular is lower than c^*-δ, which means it is not an equilibrium of (I,N) for any N≥ N.

Also note that if q is tight (i.e. the PPM is not q'-tie-sensitive for any q'>q) then pivot points above q cannot be the limit of any equilibrium either, as the pivotality in some environment of each such point c^*>q is under q for sufficiently large N.



So we have a rather complete characterization of equilibria, or at least of their limit points, in every instance. A natural question is whether any of these equilibria are “good” in the sense of the Condorcet Jury Theorem.



§ JURY THEOREMS FOR PIVOT POINTS

As observed above, a trivial equilibrium admits a jury theorem if and only if A has more core supporters, but this is not very interesting. 

Our main question regards the non-trivial equilibria, whose limits are the pivot points. Clearly any equilibrium from a side where B has more support admits a non-jury (as A gets less than half the votes, in expectation), but we could still hope that, say, a right equilibrium at a pivot point where the derivative of s_A is higher, would admit a jury theorem. 

To provide an answer, we would need to apply a concrete PPM, so the results in this section would be limited to the Polynomial PPMs, and for simplicity we also consider support functions with finite intersection. By the results of the previous section, we know that every pivot point has a right equilibrium and left equilibrium, and that (except the trivial equilibrium) there are no others.  

We first characterize the rate at which election equilibria c_N approach their limit.
Let I be an instance with a Polynomial PPM and let c=(c_N)_N be a  non-trivial equilibrium of I with limit c^*. Then c_N=c^*+Θ(N^-β/α).

That is, the point c_N must be at distance that decreases proportionally to N^-β/α: not closer neither farther away. 

We will prove for c_N>c^*. The proof for c_N<c^* is symmetric.

    By Theorem <ref>, c^* is a pivot point. 

    Since the first and second derivatives of s_A,s_B are bounded, so are those of the functions n(c) and m(c) (as per Eq. (<ref>),(<ref>)). Also note that n'(c)≥ 0 everywhere and m'(c^*)>0, n'(c^*)>0 by the definition of pivot point. We also denote s^*:=s_A(c^*)+s_B(c^*) and note that s^*· N = n(c^*).

    Therefore there must be some interval [c^*,c^*+δ] and constants m, m, n, n>0 such that m'(c)∈ [ m, m] and n'(c)/N∈[ n, n] for all c∈ [c^*,c^*+δ]. Moreover, m, m can be arbitrarily close to m'(c^*) and likewise for n' (the functions are nearly-linear near c^*).  We consider only N's large enough so that c_N is in this interval, and so that p(n(c_N),m(c_N))<1.

    Now, let :=c_N-c^*, and w.l.o.g. <min{0.1,0.1/ n}. We want to show =Θ(1/N^β/α). From the definition of equilibrium,
    
    c^*+   = c_N = p(n(c_N),m(c_N)) = 1/m(c_N)^α n(c_N)^β
       =1/m(c^*+)^α n(c^*+)^β⇒

    
    m(c^*+) 
            = 1/(c^*+)^1/αn(c^*+)^β/α.

        
    Now, by the constant bounds on the derivatives, 
    
    m(c^*+)    ∈ [m(c^*)+ m , m(c^*)+ m ] 
       =  [ m ,  m ], 
    
            n(c^*+)    ∈ [n(c^*)+ n · N, n(c^*)+ n · N] 
       = [(s^*+ n )N, (s^*+ n)N]
       ⊂ [s^* N, (s^*+0.1)N]

For the upper bound, we get from Eqs.(<ref>),(<ref>),(<ref>) 

    m    ≤ m(c^*+) ≤1/(c^*)^1/α(s^* N)^β/α⇒
       ≤1/ m· (c^*)^1/α(s^*)β/α N^β/α=O(1/N^β/α),

since m, c^* and s^* are all constants. 
Likewise, for the lower bound, 

    m    ≥ m(c^*+) ≥1/(c^*+0.1)^1/α((s^*+0.1) N)^β/α⇒
       = Ω(1/N^β/α)  as required.


It is also interesting that the exact rate of convergence depends on the derivative of m(c), but not on that of n(c), meaning that the individual slopes of s_A,s_B at c^* don't matter, only the difference between them.



 §.§ Computing the win probability

We denote by Φ the CDF of the Normal distribution, i.e. Φ(x)=_X∼(0,1)(X<x).

    Let I be an instance with a Polynomial PPM with parameters q,α,β>0, and let c be a nontrivial right- or left-equilibrium of I with limit c^*<q and popular candidate A. Then there are three cases:
   
        
  * If α>2β,  then 𝕎ℙ_A(I, c)=1;
        
  * If α<2β, then 𝕎ℙ_A(I, c)=𝕎ℙ_B(I, c)=0.5;
        
  * If α=2β, then 𝕎ℙ_A(I, c)=Φ((c^*)^-1/α).
    

That is, we get a phase transition at α=2β, where above the threshold all non-trivial equilibria admit a jury theorem, below the threshold they admit a strong non-jury theorem, where both candidates are equally likely to win, and at the threshold (including that natural case of α =1, β=0.5 mentioned above), we still have a non-jury theorem where the popular candidate only has some constant <1 win probability. Interestingly, this probability does not depend at all on the shape of the support functions, only on the position of the pivot point. 

    Let c=(c_N)_N be a nontrivial equilibrium with limit at pivot point c^*<q. 
    The random variable V_A is a Binomial variable with parameters p_A=0.5(1+m(c_N)) and  n=n(c_N), and A wins if V_A>n/2.[To be completely accurate, we should sample v_A from a Binomial distribution with a number of samples that is also a Binomial variable (see Footnote <ref>), but the difference is negligible.]
    
    Since n(c_N)=Ω(N), for large enough N we can approximate the distribution of V_A-n/2 with a Normal distributions with mean and variance as follows:
    
    μ_A    = 0.5m(c_N)n(c_N); 
    σ_A^2    = 0.5(1+m(c_N))0.5(1-m(c_N))n(c_N) 
       = 0.25(1-(m(c_N)^2)n(c_N)= 0.25(1-(m(c_N)^2)n(c_N);

    
    
    
    
    

    

    
    From Lemma <ref>, there are some constants t, t>0 s.t. 
     
     
    t· N^-β/α< m(c_N)< t· N^-β/α,

     and we know from Eq. (<ref>) that
     s^*N < n(c_N) < (s^*+0.1)N.
     
    μ   >0.5  t· N^-β/αs^*N =  0.5 s^* t· N^1-β/α, and
    μ   < 0.5  t· N^-β/α(s^*+0.1)N =  0.5 (s^*+0.1) t· N^1-β/α.

     We can similarly bound σ^2_A, as m(c_N)^2 is negligible, 
     
    0.24 s^* N < σ_A^2 < 0.25 (s^*+0.1)N.

    Using the upper and lower bounds on μ and σ^2, we get the following:
    
    μ/σ   < 0.5  t· N^1-β/α(s^*+0.1)/√(0.24 s^* N) <  r· N^0.5-β/α
    μ/σ   > 0.5 t· N^1-β/αs^*/√(0.25( s^*+0.1) N) >  r· N^0.5-β/α,

    for some positive constants r,  r.

    Finally, we can use the last bounds to estimate the probability of the popular candidate winning:
    
    𝕎ℙ_A(I,N,c_N)≅_x∼(μ,σ^2)(x>0) = _x∼(0,1)(x<μ/σ) ⇒
       Φ( r N^0.5-β/α) <  𝕎ℙ_A(I,N,c_N)
           < Φ( r N^0.5-β/α),

    where Φ is the CDF of the Normal standard distribution function.[As the approximation becomes more accurate with N, the inequalities would hold without approximation for sufficiently high N and the right constants r, r.]
    We can now easily distinguish three cases, with a sharp phase transition:
    
        
   1. α>2β. In this case, where the sensitivity to the margin is much higher than the sensitivity to the number of voters, we have that N^1/2-β/α→∞, so 
        
    𝕎ℙ_A(I, c)=lim_N𝕎ℙ_A(I,N,c_N) = 1.

        That is, we get a jury theorem as the more popular candidate wins w.p. 1 at the limit. 
        
  2. α<2β. When the sensitivity to the margin is smaller than the threshold,  N^1/2-β/α→ 0, so both candidates win with equal probability at the limit. 
        
   3. α=2β. This includes the α=1,β=1/2 case mentioned in the preliminaries. Here N^1/2-β/α=N^0=1, and thus A wins with some probability that is bounded between two constants [Φ( r), Φ( r)].
    
    It is not hard to see that we can tighten the gap between r, r, so in the limit they both equal r=(s^*)^0.5-β/α·(c^*)^-1/α; meaning that in the threshold case, r=(c^*)^-1/α. In particular, this means that the winning probability is completely independent of the shape of the support functions, and only depends on the position of the pivot point c^*. Higher c^* means a larger number of active voters and higher winning probability of the popular candidate, in the limit. 
    

    For the example with the linear supports and α=1 we indeed get 𝕎ℙ_A = Φ(1/0.6) = 0.9522

    






 
 
  




 
 0
The expected margin is voters assessment of how robust the winner's position is in terms of vote share. The larger the margin, higher is the vote share of the winner.  Expected margin is given by 
    m(c):=|s_A(c)-s_B(c)|/s_A(c)+s_B(c).
 
 Given a voter population N and a threshold cost c, the expected number of total votes is given by 
    n(c) := N (s_A(c) + s_B(c)).

Furthermore, the probability that a random voter votes for candidate A is given as 

    P(A) = s_A(c)/s_A(c) + s_B(c)


We remark here that  the parameters m(c), n(c) and P(A) are voters assessment of expected margin of victory, voting population and fraction of votes for candidate A respectively; i.e. these are not the  parameters obtained after the voting process. However, given the knowledge of cost distribution, these are unbiased estimates of the true parameters.   Furthermore,  since the cost function is common knowledge each voter has same estimate for all these parameters. 





 

§ STABILITY OF  EQUILIBRIUM POINTS

 
 In this section we show that the right side equilibrium i.e. c^+ is stable by  showing that when the perceived pivotality is different across voter types, a constant fraction of voters have an incentive  to participate (or abstain) such that the equilibrium c^+ is restored. Note that c^0 is clearly a stable  equilibrium since the margin is a near-constant. 
 
propositionProbOne
Let c_A ≥c and c_B ≥c be cost thresholds for voters of type A and B respectively, and perceived pivotality p is given as p = p(c_A, c_B) = p(m(c_A, c_B), n(c_A, c_B)). Then for any voting instance I with N voters, the best response of voters  is such that  c_A = c^+ and c_B = c^+.   









 We provide the intuition of the proof.  The detailed proof is given in Appendix. Notice in equilibrium  both types have the same cost  threshold and hence the equilibrium points must be on the line  c_A= c_B as shown in Figure <ref>. Suppose first that  c_A > c_B. In this case,  the   supporters of A shown  in the shaded area (in Figure <ref>) to the right of c^+ are less pivotal and hence are   incentivized to abstain.  Similarly,  when c_A <  c_B,  the A voters in the shaded region on the left side of c^+ are more pivotal and have an incentive to participate. The B supporters on the other hand will participate in the first case and abstain  in the second case.  Thus, the population of participating voters adjusts itself (i.e. c_A ↓ c^+ and s_B ↑ c^+ when c_A > c_B and c_A ↑ c^+ and s_B ↓ c^+ when c_A<c_B) such that the equilibrium c^+ is restored.


The left-side equilibrium, on the other, hand may not always be stable. Consider c" < c^- be an equilibrium with cost thresholds c_A and c_B with c_A <  c_B. Then,  the supporters of B are  incentivized to participate as, with additional participation from supporters of B, the margin m increases, increasing  the probability of win for B. The supporters of A, on the other hand  are also incentivized to participate. An additional participation from A supporters would mean that m decreases. Hence stability of c^- depends on the relative increase in the participation from each type of agent. 




The emergence of an unstable equilibrium between two stable ones (one of which is trivial) also occurs e.g. in markets with positive externalities <cit.>. In our case externalities behave non-monotonically (positive under ĉ and negative above but the results are similar.




§ INDUCING UNBIASED PARTICIPATION
 

added this:
In the previous section we saw that allowing everyone to participate may introduce multiple equilibria, including some where the less-popular candidate wins with higher probability, and where the probability of the popular candidate to win is bounded regardless. 

We next explore a different idea: instead of allowing everyone to vote, we sample a sufficiently small group of voters (odd n), such that they are all guaranteed to perceive themselves as pivotal, and hence all of them vote. The result is unbiased voting, but with fewer voters.  A small sample size  also guarantees unique, stable equilibrium that favours the popular candidate. 






 

 Denote by ε:=s_A(1)-s_B(1), the expected margin under full  participation, so 
 s_A(1) =  ( 1+ ε)/2 then n ≤ 1/ε^2. We then need to select sufficiently small n so that p(n,)≥ 1. In the fully rational models, this can only occur for a single active voter. However under our tie-sensitive model we  have 
 
    1 ≤ (n, ε)     = 1/ε√(n)

 so any n ≤  1/ε^2 will guarantee participation of the entire sample. 
 The perceived margin   provides sufficient information to bound the winning probability of popular candidate; independent of the shapes of support functions. 
 
 We now first consider a single round.
This already provides us with a uniform lower bound of 0.84134 on the win probability of A when the margin ε goes to 0. removed due to even n: and is only higher for larger margin.  The number 0.84134 is not coincidental but is the probability that a standard Normal random variable does not exceed a single standard deviation (See figure <ref>). Of course, the  number is  sensitive to the parameters of the exact PPM we use (as are the results in the previous sections), but not to the  shapes of the support functions. 

For larger margin, the win probability of A may be either higher or lower, and this depends mainly on the parity of 1/^2 (where for odd numbers it is always higher). 

Still, we would like to further improve the winning probability. One direction (which we do not explore in this work) is to sample a higher number of voters in an attempt to balance bias and quantity. Instead, we will explore the possibility of allowing multiple rounds.










 


  
Multi-round voting



Next, we consider the multi-round voting. The idea here is to improve win probability of the  more preferred candidate by randomly  sampling a small subpopulation repeatedly over multiple rounds. A randomly sampled population of size N_i is selected in round i. The selected voters vote according to the equilibrium corresponding to population size N_i. If any candidate wins with a certain margin,  a winner is declared and voting procedure stops. If no candidate wins in round i,  the election mechanism moves to the next round. 

It is not straightforward to see that multi-round voting necessarily improves the winning probability of the preferred candidate.  On one hand,  small number  induces high participation whereas on other,   the winning probability of preferred candidate is upper bounded by the fact that  only a small fraction of voters are selected in any given round. Furthermore, selecting small number of voters also,  potentially increases the number of rounds. 






Note that if we require a supermajority instead of a majority, the winning probability of A decreases, but not as much as the winning probability of B (see Fig <ref>). So in the extreme case, we could just try again and again until we get a unanimous vote.  However such an extreme approach may substantially alter voters' behavior, which do not even know the number of rounds. 

We can thus limit voting to at most two rounds. Given an expected margin :

    
  * Set a supermajority threshold '  <;
    
  * sample n=n() voters;
    
  * If the first round ends with a margin at least ', the winner is determined and there is no second round;
    
  * Otherwise, we run a second round with n'=n(') voters and simple majority.

Note that the chances of the unpopular candidate B winning in either round are small, as the first round require a supermajority, and the second round, if occurs, has n'>n voters. 









lemmaMultiRoundLem
[Informal] For any δ > 0 there exist a sequence of voting 


Recall that

    (n(c), m(c)) = c = min(1, 1/ (m(c) √(n(c))))






 


 
 




  
 Large value of ε: Here the perceived margin of voters given by ε is large and hence  the maximum  number of voter samples needed to induce each voter to vote is small. A natural lower bound on the win probability of the preferred candidate in this case is given by the probability that every sampled voter votes for B and given by  ( 1-ε/2)^1/ε^2.   



  
 Small value of ε:   Our goal here is twofold; first, the win probability of the preferred candidate (candidate A) must be significantly larger than the win probability of B and second, the probability that the voting mechanism goes in the second round must be small. We will characterize the trade-off between these two objectives and show that the optimal trade-off could be achieved by setting the margin of victory for the first round carefully. 
Dont think we need this...
Let X_i = ∑_i=1^N_i1 ( i   votes  for  A) be a random variable denoting total number of votes to A in round i. With N_1 = 1/ε^2,  X_1 is a Binomial distribution with parameters 1/ε^2 and (1+ε)/2.    The standard deviation of X_1 is δ_1  = √(1-ε^2)/2ε≈1/2ε. Further, let c > 0 be such that the candidate with at-least N_1/2 +  c δ_1 votes wins the election.  

    ( tie  in  round   1)    =  ( X_1 ∈ [1/2ε^2  -  c δ, 1/2ε^2  +  c δ ]  )  
         = F_X_1(1/2ε^2 + (c-1) δ) - F_X_1(1/2ε^2 - cδ) 
       ≈Φ ( μ + (c-1) δ) - Φ(μ - (c+1)δ)  using,  Bin(N_1, 1+ε/2)  ≈𝒩(N_1 1+ε/2, N_1 1 - ε^2/ 4ε^2  )

 Further, 

    ( B  wins)    =  ( X_1 ≤1/2ε^2  -  c δ )  
         =  ( X_1 ≤1/2ε^2 + 1/2ε_μ     - 1/2ε   -  c δ ) 
        =   ( X_1  - μ≤     - 1/2ε (1  +  c)   )  
       ≤exp( - (1  +  c)^2/2 ). from Hoeffding's lemma

 A few observations are in order. 

Firstly, the equilibrium point, including the choice of N_i, depends solely on the perceived margin of victory. This dependence is independent of the specific functional form of support functions, making the predictions more robust.




Define X = ∑_i = 1^N_1 X_i where X_i =1 if ith voter is supporter of candidate A and 0 otherwise. Note that 𝔼[X] = 1  + ε/ 2 ε^2. Using Hoeffdings inequality,  we have for any δ > 0, 
 
    (X ≥1 + ε/   2 ε^2 - δ ) ≥ 1 - e^ - 2  ε^2 δ^2


The win probability of candidate A is given by 

    (A   wins )  = _ X ∼ Bin(1/ε^2,  1 + ε/2 )   ( X ≥1/2  ε^2  )
 
 
 Choosing, δ = 1/ 2 ε gives 
 (A  wins ) ≥ 1 - e^ - 1/2 ≈ 0.4. NOt a good bound, must be better than coin toss. Chernoffs ineauality may give a better bound.   

 
     
  * Surprisingly, N_1 depends on ε but lower bound on ( A  wins) does not depend on ε and is a constant. 
      
  * Define γ
 

    ( A  wins  with  atleast  k  votes  )     =  _X ∼ Bin(1/ε^2, (1+ ε)/2)( X ≥1/2 ε^2 + k/2 ) 
        =  (  X ≥𝔼[X]   -  1/2(1/ε - k) ) 
       ≥ 1 - e^- ε^2 (1/ε - k)^2/2from Hoeffding's inequality
        = 1 - e^-  (1 - ε k)^2/2


 Similarly, we have

    ( B  wins  with  atleast  k  votes  )    =  _X ∼ Bin(1/ε^2, (1+ ε)/2)( X ≤1/2 ε^2  - k/2 ) 
        =  (  X ≤𝔼[X]   -  1/2(1/ε + k) ) 
       ≤    e^- ε^2 (1/ε +  k)^2/2from Hoeffding's inequality
        =   e^-(1 + ε k)^2/2 := γ

 
 Note that we must have k ∈ [0, 1/ε].  Choose k = 1/ε  to get 
 
 
    ( A  wins  with  atleast  k  votes  )    ≥ 1 - e^ -  ( 1 - 1/ε )^2/2
    ( B  wins  with  atleast  k  votes  )   ≤ e^ - (1+ε)^2 / 2

 For instance, for ε = 0.125 we have 
 
    ( A  wins  with  atleast  k  votes  )    ≥ 0.98889 
    ( B  wins  with  atleast  k  votes  )   ≤ 3.72665 × 10^-6

   



§ NUMERICAL STUDY WITH AN EXAMPLE


In this section, we extend the theoretical results of the previous section with the help of an example for different model parameters, numerically. 

 exampleexOne
Suppose s_A(c)=0.1+c/2 and s_B(c)=0.4. That is, B has 40% overall support, all of them core supporters and A has 60% overall support and a fraction of  voters are distributed  over cost range [0,1]. 


 Figure <ref> shows the win probability of candidate A under c^+ and  for a fixed population size of 10^6 for different values of α. The larger value of α pushes the equilibrium   point c^+ towards the right, increasing the win probability of the popular candidate. However, as shown in the blowup box, this probability is still bounded away from 1. 







    
Suppose in the example of Fig 1, we fix c^+=0.6+. How large N needs to be so that c^+ is an equilibrium, for α=0.75, α=1 and α=1.25? conjecture: for α<1 we have N=O(f(1/)) and for α>1 we have N=Ω(g(1/)) where g≫ f e.g. exponential vs. polynomial.  phase transition at α=1.

    
  * Write about lipschitz functions and local election lemma 
    
  * Plot alpha vs winning probabilities (claim: weaker dependence of m will increase  equilibrium win probability for A (need to check!))
    
  * Priority! 

propositionNonTrivialEquilibria
Every stable non-trivial  equilibrium  converges to the nearest left anchor point. 


lemmaboundOnMargin
There exist constants β_1≤β_2 < ∞ such that for every α>0 we have 
    β_1/N^α≤ m^⋆≤β_2/N^α
 for all  N≥ N_0. 


corollarycorOne
Given a finite N, 
the equilibrium cost threshold is bounded away from anchor point. 

propositionNonJuryOne
In an election instance ℐ with a weak pivotal PPM  model,  there is a constant F<1 such that q^⋆(N)≤ F  for every  N ≥ N_0 for some N_0>1. 








In Figure <ref>, we observe that the  win probabilities under c^+ and c^- for respective winning candidates are significantly different. While B wins with a high probability under c^-, A wins under c^+ with a high probability for the same population size. Hence  B would always prefer a smaller fraction of the   population to vote whereas the popular candidate prefers a large  population to vote. Furthermore, for  a fixed population size, the win probabilities of these candidates under their favoured equilibria are not the same.





    patterns
      south west lines-0pt-0pt3pt3pt3pt3pt
            0.4pt
            0pt0pt
            3pt3pt
            2.8pt-0.2pt
            3.2pt.2pt
            -.2pt2.8pt
            .2pt3.2pt
            stroke
    
    south east lines-0pt-0pt3pt3pt3pt3pt
        0.4pt
        0pt3pt
        3pt0pt
        .2pt-.2pt
        -.2pt.2pt
        3.2pt2.8pt
        2.8pt3.2pt
        stroke
  


§ A NON-JURY THEOREM WITH LINEAR SUPPORT 

 
Sections <ref>, <ref>, <ref> should be subsections of a section called 'Equilibrium with Abstentions' or something like that.  Then all the new stuff goes under `Full Participation Equilibrium' (with one or more rounds).
Let 𝒞 be the set of all pivot points. 
propositionthmTrivial
    Let ℐ_N be an election instance with N voters and    positive, continuous and  increasing functions s_A, s_B  defined  over [0,1] such that   s_A(c) > s_B(c) for all c ∈ [0,1]. Also let c^0 (N) be an equilibrium cost threshold (Eq. <ref>) induced by a tie-sensitive PPM, then 
    lim_N →∞ c^0(N) = 0. 
    Furthermore, the better alternative (candidate A) wins an election with a probability of 1 in the limit. That is (A  wins ) → 1 as N →∞. 
    

We begin by showing that for large values of N, non-trivial equilibria emerge   around the pivot point and converge to the pivot point as N goes to infinity. 
 


 

§ CONCLUSION

   Our  results show  that under the boundedly rational PPM    the equilibrium outcome does not guarantee a  decisive win for any of the candidates even with an arbitrarily large electorate size. Interestingly, the proposed model also captures the equilibria induced in fully rational models (Binomial and Poisson models) as trivial equilibria.     Our boundedly rational model satisfies the property that    both candidates enjoy almost equal support  due to disproportionate abstention from their supporters.  The almost equal support for candidates is a result of a trade-off between the following two factors;  
we have the  trade-off between following two factors. 
then contested by  concluding that agents also receive utility from the performing the civic duty .
a)  The candidate having majority support among high-cost voters  faces  more abstention. This increases the winning probability  of the unpopular candidate against the popular candidate and       b) voters are incentivized to vote  depending on how many other voters  vote. Hence, when a large fraction (overall) of  voters abstain,  the size of the voting population shrinks increasing the pivotality parameter which causes more voters to vote.  

  The number of votes on the ballot increases with the voting population, however, this does not benefit  both the candidates equally; disproportionate  increase in the support of unpopular candidates leads to anti-Jury results. It is interesting to see how and if  this happens in practice. We also leave the sensitivity analysis and robustness of these results as future work. Finally, coming up with a fully rational model that explains both high turnouts and surprises in large elections is an interesting future problem.  
   
   

   

 
[Ald93]Aldrich
John H. Aldrich.
Rational choice and turnout.
American Journal of Political Science, 37(1):246–278, 1993.

[Bla00]Blais
André Blais.
To Vote or Not to Vote: The Merits and Limits of Rational Choice Theory.
University of Pittsburgh Press, 2000.

[Dow57]Downs57
Anthony Downs.
An economic theory of political action in a democracy.
Journal of Political Economy, 65(2):135–150, 1957.

[EGK07]edlin2007voting
Aaron Edlin, Andrew Gelman, and Noah Kaplan.
Voting as a rational choice: Why and how people vote to improve the well-being of others.
Rationality and society, 19(3):293–314, 2007.

[FF75]Ferejohn75
John A. Ferejohn and Morris P. Fiorina.
Closeness counts only in horseshoes and dancing.
The American Political Science Review, 69(3):920–925, 1975.

[FLMG19]fairstein2019modeling
Roy Fairstein, Adam Lauz, Reshef Meir, and Kobi Gal.
Modeling people's voting behavior with poll information.
In Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems, pages 1422–1430, 2019.

[FP96]fedderson12
Timothy J. Feddersen and Wolfgang Pesendorfer.
The swing voter's curse.
American Economic Review, 86(3):408–424, 1996.

[KS85]katz1985network
Michael L Katz and Carl Shapiro.
Network externalities, competition, and compatibility.
The American economic review, 75(3):424–440, 1985.

[Mei18]2018Meir
Reshef Meir.
Strategic Voting.
Synthesis Lectures on Artificial Intelligence and Machine Learning. Morgan & Claypool Publishers, 2018.

[Mer81]Merrill1981
Samuel Merrill.
Strategic decisions under one-stage multi-candidate voting systems.
Public Choice, 36(1):115–134, 1981.

[MGT20]meir2020strategic
Reshef Meir, Kobi Gal, and Maor Tal.
Strategic voting in the lab: compromise and leader bias behavior.
Autonomous Agents and Multi-Agent Systems, 34(1):1–37, 2020.

[MLR14]Meir14
Reshef Meir, Omer Lev, and Jeffrey S. Rosenschein.
A local-dominance theory of voting equilibria.
In Proceedings of the Fifteenth ACM Conference on Economics and Computation, EC '14, page 313–330, New York, NY, USA, 2014. Association for Computing Machinery.

[MW93]MyersonWeber
Roger B. Myerson and Robert J. Weber.
A theory of voting equilibria.
The American Political Science Review, 87(1):102–114, 1993.

[Mye98]Myerson1998
Roger B. Myerson.
Population uncertainty and poisson games.
International Journal of Game Theory, 27(3):375–392, 1998.

[Mye02]MYERSON2002219
Roger B Myerson.
Comparison of scoring rules in poisson voting games.
Journal of Economic Theory, 103(1):219–251, 2002.

[OG84]Owen1984
Guillermo Owen and Bernard Grofman.
To vote or not to vote: The paradox of nonvoting.
Public Choice, 42(3):311–325, 1984.

[OR03]OSBORNE2003434
Martin J. Osborne and Ariel Rubinstein.
Sampling equilibrium, with an application to strategic voting.
Games and Economic Behavior, 45(2):434–441, 2003.
Special Issue in Honor of Robert W. Rosenthal.

[RO68]riker68
William H. Riker and Peter C. Ordeshook.
A theory of the calculus of voting.
The American Political Science Review, 62(1):25–42, 1968.

[Wik22]wiki:Voter_turnout
Wikipedia.
Voter turnout in United States presidential elections — Wikipedia, the free encyclopedia.
<http://en.wikipedia.org/w/index.php?title=Voter%20turnout%20in%20United%20States%20presidential%20elections   oldid=1116061047>, 2022.
[Online; accessed 28-October-2022].







§ MISSING PROOFS










*
 Suppose there are N voters sampled from .
    Given a cost threshold c,  let the random variable X_A^(i)(c) := 1/N-1∑_j ≠ i1[ c_j ≤ c  and  A ≻_j B ] denote fraction of supporters—other than voter i—of candidate A having the voting cost at-most c and similarly, define  X_B^(i)(c):= 1/N-1∑_j ≠ i^N-11[ c_j ≤ c  and  B ≻_j A ]. Note that, as X_T^(i)(c) does not depend on her type T_i and her private cost c_i and since every agent  is exposed to the same knowledge, we have, X_T(c) := X_T^(1)(c) = X_T^(2)(c) = ⋯ = X_T^(N)(c) for all T ∈{A,B}. Finally, let s_T(c) := 𝔼[X_T(c)], and note that it depends neither on i nor on N.

    To see that s_T(c) is continuous, let s^+, s^- the right and left limits of s_T(c) at c^*∈(0,1). If s^-<s^+ then Pr_[(c^*,T)] = s^+ - s^- >0, in contradiction to out assumptions that  has no atoms at (0,1).

    In the other direction, we define Pr_[T_i=A]:=s_A(1), then by assumption Pr_[T_i=B]:=1-s_A(1)=s_B(1). For each T∈{A,B}, then function s_T(c)/s_T(1) is the CDF of Pr_[c_i| T_i].

*

Let c be the cost value such that c=p(n(c),m(c)) and fix a player i.   We show that the best response of player i is to vote when c_i≤ c and abstain otherwise irrespective of her type. 

If not, say c_i > c and i votes. Then the perceived  utility of player i is given by 

    p(n(c), m(c)) - c_i  = c  - c_i< 0.


We note that the perceived pivotality p  only defines agents subjective belief on the value of individual vote and may not be from well defined probability space.  

Similarly,  when c_i < c, the best response strategy is to vote as voting gives strictly positive payoff to agent i.[Note that as agent considers his vote to be of value p(.), this value is not realized if agent abstains giving her the payoff of zero.] Since the perceived pivotality is independent of player type, the voter strategies are the same.    


Conversely, suppose there are two values 0< c_1<c_2≤ 1 such that agents i votes if c_i ∈ [c_1, c_2] and abstains otherwise. Consider the following cases. 


    
  * c ∈ [c_1, c_2]: In this case,  either there is an interval ℐ := [c, c_2] such that agent i votes (i.e. when c_i ∈ℐ) but  in this case  voting is not a best response strategy for i  since c_i < c, or there is an interval ℐ' = [0,c] such that voting is a best response strategy (when c_i > c). In either case, from continuity of support function we have that the probability measure on at least one of I and ℐ' is nonzero. 
    
  * c ∉ [c_1, c_2]: We consider two subcases 
    
        
  * c < c_1: In this case there is an interval [c,c_1) such that the best response of agent i is not to vote when  c≥ c_i. 
        
  * c > c_2: In this case there is an interval [c_2,c) such that the best response of agent i is not to vote when  c≥ c_i. 
    
This completes the proof. 


*

    For any c∈ [0,1], define f(c):=p(n(c),m(c)). Since the support functions are continuous, n(c) and m(c) are continuous (by Eqs. (<ref>),(<ref>)). By continuity of p, the function f is also continuous. Finally, every continuous function from [0,1] to itself has a fixed point, due to intermediate value theorem applied to f(x)-x.


*

    Assume towards a contradiction that there is a nontrivial equilibrium c=(c_N)_N with limit c^*>δ for some δ>0. This means  that c_N>q for all sufficiently large N. However, for any N, the number of active voters n(c_N) is at least (s_A(δ)+s_B(δ))N= Ω(N), and thus p(n,m)= O(1/√(n)) = O(1/√(N)). This means that for sufficiently large N, the pivot probability is p(n(c_N),m(c_N))<δ. A contradiction.

*

For any δ>0, we should show that there is some N_δ and c_δ<δ such that c_δ∈ C^*(I,N_δ).

W.l.o.g. s_B(0)>s_A(0), and by the bounded derivatives, the margin m(c) is at least some >0 in some neighborhood c∈ [0,q].
Let δ:=min{q,δ}.

 By weakly vanishing pivotality, 
 
    p(n(δ),m(δ))≤ p(n(δ),) = p(s_B(0)N,)→ 0

 as N grows. In particular there is some N_δ for which p(s_B(0)N_δ,)<δ. We argue that there must be an equilibrium in the range [0,δ].

 Indeed, fix N_δ and let f(x):=p(n(x),m(x))-x. We know that f is continuous, that f(0)≥ 0 and that f(δ) = p(n(δ),m(δ))-δ< δ-δ=0. From the intermediate value theorem, there must be some c_δ∈ [0,δ) for which f(c_δ)=0, meaning c_δ=p(n(c_δ),m(c_δ)). Thus c_δ∈ C^*(I,N_δ).



*


As c >0 and s_A and s_B are linear,  we have s_B(c) > s_A(c) for all c < c. 
 The proof of convergence of trivial equilibrium to 0 follows from Proposition <ref>  with roles of A and B reversed. 
 
 
 We now prove the convergence of non-trivial equilibria to the pivot point. We first show that for any fixed value of N,  there are exactly two equilibria; one on each side of the pivot point.  
1) We first show the existence of trivial equilibrium i.e. c_1(N) such that lim_N →∞ c_1(N) =0. Let without loss of generality that A is the popular candidate i.e. s_A(1) > s_B(1). Then, since c >0 and s_A and s_B are linear,  we have s_B(c) > s_A(c) for all c < c. Let for a given  c < c, N_c := (s_A(c)+s_B(c)/(s_A(c)-s_B(c))^2.

We have  
    min (1, 1/m(c)√(n(c)) ) = 1/m(c)√(n(c)) = √(N_c/ N).


Note that,  since m(c) → m(0) >0 and n (c) → n(0) > 0 we have  lim_N →∞  N_c = N_0 >0.  Hence   c^0(N) = min ( 1, 1/m(c^0) √(n(c^0)))  = √(N_c^0/N)→ 0. 


   
   

Towards this, define the function h(c)  as follows.   

    h(c) =  c  |s_A(c) - s_B(c)|  √(N) - √(s_A(c) + s_B(c)).

It is easy to see that 
 h(c) < 0 and  h(1) >0 for large value of N. The fact that there is at least one equilibria in (c,1 ) follows from the intermediate value theorem. We now show that there is only one such equilibrium. Let, for contradiction,  c < c_1 < c_2 < 1 be two equilibrium points. That is 

 
    1/m(c_1)√(n(c_1)) = c_1    < c_2 =  1/m(c_2)√(n(c_2))
    √(s_A(c_1) + s_B(c_1)/s_A(c_2) + s_B(c_2))    <  |s_A(c_1) - s_B(c_1)|/|s_A(c_2) - s_B(c_2)|
        =  s_A(c_1) - s_B(c_1) / s_A(c_2) - s_B(c_2) Since s_A(c_1) > s_B(c_1) and s_A(c_2) > s_B(c_2)

 Furthermore note that as functions s_A and s_B are linear we have s_A(c_1) - s_B(c_1) / s_A(c_2) - s_B(c_2)  < 1. This gives

    s_A(c_1) + s_B(c_1)/s_A(c_2) + s_B(c_2) <  s_A(c_1) - s_B(c_1) / s_A(c_2) - s_B(c_2)

Write s_A(c) = a_1 c + b_1 and s_B(c) = a_2 c + b_2 with constants a_1, a_2, b_1, b_2 satisfying following conditions. 

    a_1, a_2, b_1, b_2    ≥ 0   Condition 1
     
     a_1 + a_2 + a_3 + a_4     = 1 Condition 2
     
     b_1 - b_2     <   0  Condition 3


Condition 1 follows from the monotonicity and non-negativity of support functions. The condition 2 follows from the fact that s_A(1) + s_B(1) = 1 and finally the condition 3 follows from the fact that s_B(0) > s_A(0). With this the Equation  <ref> can be written as 

    (a_1 + a_2)c_1 + b_1 + b_2 /(a_1 + a_2)c_2 + b_1 + b_2 < (a_1 - a_2)c_1 + b_1 - b_2/ (a_1 - a_2)c_2 + b_1 - b_2.

The nonnegativity condition (Condition 1)  implies 

    c_1/c_2≤(a_1 + a_2)c_1 + b_1 + b_2 /(a_1 + a_2)c_2 + b_1 + b_2.

The above equation (Equation <ref>) and Equation <ref> implies  

    c_1/c_2 <  (a_1 - a_2)c_1 + b_1 - b_2/ (a_1 - a_2)c_2 + b_1 - b_2.

We obtain the contradiction to  our assumption that c_1 < c_2 using Condition 3 and Equation <ref>. 




Finally, we show the convergence of equilibrium points.  Towards this, consider the sequence  (c^i)_i=1^∞ with  c^1 = 1 and c^i = c +1/i for i >1 and denote by U_i an interval (c, c^i). Note that   for all N ≥(s_A(c^i) - s_B(c^i))^2 /s_A(c^i) + s_B(c^i) we have c^+(N) ∈ U_i.  This shows that for any ε > 0 there is  an i with |U_i|  < ε and large enough N such that c^+(N) ∈ U_i. 
Finally,  we use  similar construction for c^+(N) < c with c^i = i/i+1c. 











propositionNonTrivialEquilibria
Every stable non-trivial  equilibrium  converges to the nearest left anchor point. 


Let Ĉ≠∅ and ĉ∈Ĉ be the nearest anchor point to the equilibrium threshold τ.
We call such τ an  equilibrium anchor. Note that m(ĉ)=0. Let  c^⋆(N)=ĉ + _N,  and assume that the second derivative of s_A,s_B is bounded, then m(ĉ+)≅∂ m(ĉ)/∂ c, i.e. linearly increasing in  for small . The pivot probability at c=ĉ+, which is decreasing in the margin, is also decreasing in .



The proof is from continuity, similarly to the proof of trivial equilibrium above.[Tie sensitivity and ĉ<T are necessary: otherwise even when there is a  tie at ĉ, only voters with negligible voting cost (in particular lower than ĉ) will   vote.] Moreover, this equilibrium is stable. Suppose that we perturb to c'=c^*+δ, then we will get more voters, both the margin m(c') and the number of active voters n(c') will increase, and the pivot probability will decrease so voters with cost c' are disincentivized to vote.  Similarly for c”=c^*-δ.








*

Outline: Consider similar inequalities as in the example and apply lipschitz continuity to the support functions.  τ (ĉ+τ)N^α≤ (s_A + s_B)^1-α

*


Let   p_N =  s_A(c^⋆(N))/ s_A(c^⋆(N)) + s_B(c^⋆(N)). Note that  each of the n^⋆ voters vote independently. Let X_i be a binary random variable with mean p_N. and S_n^⋆ = ∑_i=1^n^⋆ X_i. We have that the candidate A's expected votes are given by μ_N := n^⋆ p_N. We have 

    (A  wins)     =  (S_n^⋆≥ n^⋆/2 +1 ) 
       ≤exp( n^⋆/2 + 1 - n^⋆/2 - (n^⋆/2 + 1)  log(n^⋆/2 + 1/n^⋆/2))   
       ≤exp(1 - (n^⋆/2 + 1) log(1+   2/ n^⋆)) 
        < 1

SOMETHING WRONG in above proof! Will check..



*

    Let s_A(c) = a_1c + b_1 and s_B(c) = a_2 c + b_2 be support functions of candidates A and B respectively and let N be the size of the population. Consider the two cases below. Note that s_A(c^+) > s_B(c^+). By definition of pivot points we have  s_A(c) = s_B(c). Hence, 

    c = b_2 - b_1/a_1 - a_2

By definition the equilibrium point c^+ must satisfy 

    c <  c^+     = 1/m(c^+) √(n(c^+))
        = √((a_1 + a_2)c^+ + (b_1+b_2) )/ (a_1 - a_2) (c^+-c) √(N)From Eq. (<ref>)
        = √((a_1 + a_2)c^+ + (b_1+b_2) )/Δ_a ε√(N)ε : = c^+ - c and Δ_a := a_1 - a_2
       ≤1/Δ_a ε√(N)since s_A(c) + s_B(c) ≤ 1

The above sequence of inequalities imply 
ε <  1/Δ_a c√(N).
Now we lower bound the value of ε. Towards this, consider 

    1 ≥ c^+     = √((a_1 + a_2)c^+ + (b_1+b_2) )/ (a_1 - a_2) (c^+-c) √(N)≥√(2 s_A(c))/Δ_aε√(N)Since s_A(c) = s_A(c) and monotonicity of s_T


    This implies ε≥√(2 s_A(c)) / Δ_a√(N).  We have 

    √(2 s_A(c))/Δ_a√(N)≤ε≤1/Δ_a c√(N).

We obtain the bounds on m(c^+) as follows. Using the definition  of m and the fact that s_A(c^+) > s_B(c^+) we can write  

    m(c^+) = Δ_a c^+ + b_1 - b_2/s_A(c^+) + s_B(c^+) = Δ_a ε/s_A(c^+) + s_B(c^+)≤Δ_a ε/2  s_A(c)  .
 The  inequality above follows from the  fact that s_A(c) = s_B(c) ≤ s_B(c^+) < s_A(c^+). 
To obtain a lower bound, observe that s_A(c^+) + s_B(c^+) ≤ 1. This gives, 
m(c^+)  ≥Δ_a ε. Finally, we use the bounds on ε from  Eq. <ref> as follows 


 
 
 
 



    √(2s_A(c)/N)≤ m(c^+) ≤1/2c s_A(c) √(N)

Furthermore, since  s_A(c^+) ≥ s_B(c^+) we have ℙ(vote A)  =     1/2 + m(c^+)/2 and  





    1/2 + √(s_A(c)/2N)≤ℙ(vote A) ≤1/2 + 1/4c s_A(c) √(N)

  

*

We have 
    c  > c^-    =     1/m(c^-) √(n(c^-))≥√((max(s_A(0), s_B(0)) )/Δ_a  ε√(N)
       ε >  √((max(s_A(0), s_B(0)) )/Δ_a  c√(N)

From the convergence lemma (Lemma <ref>) we have that for large value of N we have 


    c /2    ≤  c^- = 1/m(c^-) √(n( c^-))≤√(2s_A(c))/Δ_a  ε√(N)
       ε <  2√(2s_A(c)  )/Δ_a  c√(N)

  Equations <ref> and <ref> gievs following bound on the  margin at the equilibrium point c^- 
  
    √((max(s_A(0), s_B(0)) )/c√(N)   <  m(c^-) <   2√(2s_A(c)  )/ε√(N)

The required bound on ℙ(vote  B) can be computed using the fact that ℙ(vote  B) = 1/2 + m(c^-)/2. 



*

Note that 

    c≤ 1     c   ≤b_1 + b_2/a_1 + a_2 + b_1 + b_2 - a_1 -a_2
       = b_1 + b_2/1 - (a_1 +a_2)since a_1 + a_2 + b_1 + b_2 = 1
    c   ≤   (a_1 + a_2) c + b_1 + b_2  
       = s_A(c) + s_B(c) =2 s_A(c).




*

We prove the result by looking at the best response of each type of voters. Let u_i(p_i,c_i) be the utility of voter i under perceived probability p and cost of voting c. We will consider that two types (A supporters and B supporters) of voters have different perceived pivotalities p and p' respectively.   We will drop the subscript whenever it is clear from the context. 

First, note that since n = n(c^+, c^+) we have that  either c_A > c^+ > c_B or c_B > c^+ >c_A. That is, with same voting population as in the equilibrium,  either A's supporters are more in number or B's supporters are more than their number under the    equilibrium. 

 Case 1 (c_A  > c^+ > c_B > c): In this case we have 


    s_A(c_A) > s_A(c^+) > s_B(c^+) > s_B (c_B) > s_A(c) = s_B(c).
 Hence, there is a more participation from A supporters compared to the equilibrium point c^+ and a correspondingly less participation from B supporters than equilibrium point. 
This gives p  <  p^+ <  p'.   


The utility for A supporters  with cost  c ∈ [c^+, c_A]  (the shaded region on the right of c^+ in Figure <ref>) from voting is given by 
    u(p,c) = p - c <  p^+ - c = c^+ - c ≤  0.
  Hence these voters are incentivized to abstain from election.  

The B supporters in cost  c ∈ [c_B, c^+], on the other hand, obtain strictly positive  utility by voting given by 
    u(p',c) = p'-c   >  p^+ - c ≥  p^+ - c^+ =0
 

Case 2 (c_B > c^+ > c_A):  In this case, the A voters' perceived pivotality p is larger than that of equilibrium pivotality p^+ whereas B supporters' perceived pivotality p' is less that p^+ i.e., p' <  p^+ < p. The utility of A supporters in range [c_A, c^+] (shaded region left of c^+ in Figure  <ref>) from voting is 

    u(p,c) = p - c > p - c^+ = p - p^+ > 0.
 Similarly the utility obtained by B's  supporters in cost range c ∈ [c^+, c_B] by voting is given by 

    u(p',c) = p' - c < p^+ - c  = c^+ - c ≤  0.

Hence these B supporters  are incetivized to vote.  





     









§ A DETAILED EXAMPLE 



*



  
Non-trivial Equilibrium: The proposed PPM also induce equilibria around (and close to) each pivot point i.e. the cost value c for which s_A(c) = s_B(c). probability that A wins equals

    _x∼ Bin(n^*, l_m)[x≥ n^*/2]≅_x∼ Norm(n^* l_m,n^* l_m(1-l_m))[x≥ n^*/2].

This Normal distribution has a mean of μ=n^* l_m=n^*(0.5+m^*/2)≅ n^*(0.5+^*/3.2). The standard  deviation σ of this Normal distribution satisfies    
σ=√(n^* l_m(1-l_m))> √(0.2n^*)>√(0.2· 0.6 N)> 1/3√(N).
On the other hand, 

    |n^*/2 - μ| = n^*^*/3.2< (0.6 N)^*/3.2 < 0.5√(N)· 0.5√(N)^*<0.5√(N)· 1.6 < 0.8· 3 σ< 3σ.

In words, the difference between the mean and the threshold for A's victory is bounded by 3 standard deviations, meaning that the winning probability is at most 99.7%. This contradicts the Jury theorem. 



 
 

 §.§ Analysis of Example <ref> under fully rational models


Binomial PPM:  The expected number of active voters for a given cost c is given as n(c) = N(s_A(c)+s_B(c)) = N(1+c)/2. As for the winning  margin, we have 
m(c) = |0.3-c/2|/0.5+c/2=|0.6-c|/1+c.
  The equilibrium c^* satisfies 

    c^*   = (n(c^*),m(c^*)) = (N1+c^*/2,0.6-c^*/1+c^*)
       ≅1/√(π N(1+c^*))( 1 -  ( 0.6-c^*/1+c^*)^2  )^N(1+c^*)/4≤1/√(π N )



For large value of N, c^* is close to 0 and hence the margin remains close to 0.6 in favor of the unpopular candidate B,   increasing the winning probability of B. Note that this equilibrium point indeed satisfies the Jury theorem as given above. 
 


 

 §.§ Analysis of Example <ref> under proposed tie-sensitive PPM model

Consider the proposed tie-sensitive model  (n,m)=min{1,1/m√(n)} and observe that


c^*  = (n(c^*),m(c^*)) = min(1, 1/m(c^*)√(n(c^*)))  
 = min (1, √(1+c^*)/|0.6-c^*|√(2/N)  ). 

For large values of N we obtain  three non-negative solutions for c^*; 0 <c^0 <  c^- < 0.6 <  c^+ ≤ 1. For instance. for N = 10^4 we have. (c^0, c^- , c^+ ) = (0.02489,0.5689, 0.62823)  and for N=10^6 it is (0.002369, 0.5970, 0.602964). See Figure <ref>.



 Trivial Equilibrium: We first track the equilibrium around 0; i.e. c^0. Similar to the strongly vanishing Binomial  PPM model, this model also supports  an equilibrium solution that goes to 0 with rate N^-1/2. Furthermore, a trivial solution always exists and candidate B wins in a trivial equilibrium satisfying the Jury theorem.  
 

Non-trivial Equilibria: 
We will consider the equilibrium c^+ i.e. the one in  which popular candidate (candidate A) wins with large probability. The analysis of c^- is analogous.   
Let c^+:=0.6 + and note that for large N, the winning margin  m(c^+) is approximately  /1.6 and the number of active voters is   
n(c^+)= N(1+c^+)/2=N(1.6+)/2=0.8N+0.5 N. Hence 

    c^+     =  +0.6  =  (0.8N+N/2,/1.6) 
       ≅min{1,1.6/√(0.8N+ 0.5 N)}
  

 The minimum on the right side of the above equation is less than 1 for large enough N. We have  
    (+0.6)√(0.8N+0.5 N) = 1.6





The following sequence of inequalities   

    0.5√(N)< 0.6√(0.8N)< ( +0.6)√(0.8N+ 0.5N) = 1.6
 
imply   < 3.2/√(N). This gives an upper bound on the rate at which ε decreases as a function of population size N. Similarly, we lower bound the value of ε using

    1.6 = ( +0.6)√(0.8N+ 0.5N)≤ε√(N)
 to 1.6/√(N).
The last inequality above follows from the fact that c^+ = 0.6 + ε≤ 1 i.e. ε≤ 0.4.
 Together we  have 
    1.6/√(N)≤ε≤3.2/√(N)

For large value of N, Eq. (<ref>) above gives a tight characterization of the equilibrium threshold point c^+.  This, consequently, gives a reasonably tight bound on the probability that a random voter will vote for  popular candidate A is given by 
    (A) := s_A(c^+)/s_A(c^+) + s_B(c^+) = 0.2+c^+/1+c^+.
 

    0.8 √(N) (√(N) +1)  < n(c^+)    < 0.8 √(N) (√(N) +2)   and,

This gives √(N) + 2/ 2(  √(N) + 1)< (A) = 0.2 + c^+/ 1 + c^+ < √(N) + 4/2(√(N) + 2). 
A straightforward calculation, gives the following bounds on P(A).

    0.5 + 1/2(√(N)+1) < P(A) < 0.5 + 1/2+ √(N)

Let  δ_1 = 1/2(√(N)+1) and δ_2 = 1/2+ √(N). 

Using this, we now analyze the win  probability of candidate A. For this, we will approximate the Binomial random variable by the appropriate gaussian random variable. Let,  μ = n(c^+)(A) and  σ = √(n(c^+) (A)(1 - (A))). Note that (A wins) ≃_X ∼𝒩(μ, σ^2)  (X ≥n(c^+)/2). 

Using the bounds above  we see that

    σ   > √(0.8 N (0.25 - ε_2)^2) =  √(0.8 N (0.25 - 1/(2 + √(N))^2))
       ≈√(0.2 N) > 0.4 √(N)
 
On the other hand 
    |n(c^+)/2 - μ |    ≤  n(c^+)  ε_2 =   0.8√(N)   <  2 σ.

This implies that the difference between the mean and the threshold for A's victory is upper bounded by two times standard deviations.  Hence the candidate B stands at least a 0.6 percentage chance of winning the election. 












 





§  ADDITIONAL NUMERICAL RESULTS

In this section, we empirically evaluate the theoretical results.  We first show in Figure <ref>   that the number of active voters in large elections is  a constant fraction of the population size.  We consider an election instance from Section <ref>  and observe that for a large value of N both the equilibria induce (almost) a constant fraction of voters to vote. This also implies that for a large value of N both equilibrium points are nearby, favouring different candidates whereas not guaranteeing any candidate a certain victory. 




Next, in Figure <ref> we observe that for the weaker dependence on the margin of victory i.e. smaller values of α induce smaller equilibrium cost values. For instance, for N = 10^3  the equilibrium is  c^+ =0.8 for  α =1.5 whereas the equilibrium cost is c^+ = 0.605 when α = 0.5.  This implies that for fully rational models, when α =0 our models predicts that only equilibrium is the trivial equilibrium. 






 §.§ Comparing c^- to c^+




Fig. <ref> shows empirically that

    
  * The locally popular candidate at c^- (B) has decreasing winning probabilities;
    
  * The locally popular candidate at c^+ (A) has increasing winning probabilities;
    
  * The winning probability of B at c^- is always larger.

At least for the third observation we can provide a theoretical justification. 



For a linear-support, tie-sensitive PPM, and any N, it holds that the winning probability of B at c^- is strictly greater than the winning probability of A at c^+.


    Since c^-<c^+, there are strictly less active voters (of both types) in c^-, thus n(c^-)<n(c^+).

    Since both points are equilibrium point, we know that voters in c^+ participate iff their perceived pivotality is at least c^+. That is, 
    
    p(n(c^+),m(c^+))=c^+.


    Similarly, 
    
    p(n(c^-),m(c^-))=c^-.

    Now, p is decreasing in both arguments (for any PPM), thus
    
    p(n(c^-),m(c^-)) > p(n(c^+),m(c^-)).

    Together, 
    
    p(n(c^+),m(c^+))=c^+>c^-=p(n(c^-),m(c^-)) > p(n(c^+),m(c^-)).

    Since p is decreasing in m, this means m(c^+)<m(c^-). 






