Empirical

Mechanism

Design:

Methods,

with

Application

to

a

Supply-Chain

Scenario

Yevgeniy

Vorobeychik,

Christopher

Kiekintveld,

and

Michael

P.

Wellman

University

of

Michigan

Computer

Science

Engineering

Ann

Arbor,

MI

USA

yvorobey,

ckiekint,

wellman

}@umich.edu

ABSTRACT

Our

proposed

methods

employ

learning

and

search

techniques

to

estimate

outcome

features

of

interest

as

a

function

of

mechanism

parameter

settings.

We

illustrate

our

approach

with

a

design

task

from

a

supply-chain

trading

competition.

Designers

adopted

several

rule

changes

in

order

to

deter

particular

procurement

behavior,

but

the

measures

proved

insufficient.

Our

empirical

mechanism

analysis

models

the

relation

between

a

key

design

parameter

and

outcomes,

confirming

the

observed

behavior

and

indicating

that

no

reasonable

parameter

settings

would

have

been

likely

to

achieve

the

desired

effect.

More

generally,

we

show

that

under

certain

conditions,

the

estimator

of

optimal

mechanism

parameter

setting

based

on

empirical

data

is

consistent.

Categories

and

Subject

Descriptors

[Computing

Methodologies]:

Simulation

and

Modeling;

[Computer

Applications]:

Social

and

Behavioral

Sciences-Economics

General

Terms

Algorithms,

Economics,

Design

MOTIVATION

We

illustrate

our

problem

with

an

anecdote

from

a

supply

chain

research

exercise:

the

and

Trading

Agent

Competition

(TAC)

Supply

Chain

Management

(SCM)

game.

TAC/SCM

defines

a

scenario

where

agents

compete

to

maximize

their

profits

as

manufacturers

in

a

supply

chain.

The

agents

procure

components

from

the

various

suppliers

and

assemble

finished

goods

for

sale

to

customers,

repeatedly

over

a

simulated

year.

As

it

happened,

the

specified

negotiation

behavior

of

suppliers

provided

a

great

incentive

for

agents

to

procure

large

quantities

of

components

on

day

the

very

beginning

of

the

simulation.

During

the

early

rounds

of

the

SCM

competition,

several

agent

developers

discovered

this,

and

the

apparent

success

led

to

most

agents

performing

the

majority

of

their

purchasing

on

day

Although

jockeying

for

procurement

turned

out

to

be

an

interesting

strategic

issue

in

itself

the

phenomenon

detracted

from

other

interesting

problems,

such

as

adapting

production

levels

to

varying

demand

(since

component

costs

were

already

sunk),

and

dynamic

management

of

production,

sales,

and

inventory.

Several

participants

noted

that

the

predominance

of

procurement

overshadowed

other

key

research

issues,

such

as

factory

scheduling

and

optimizing

bids

for

customer

orders

After

the

tournament,

there

was

a

general

consensus

in

the

TAC

community

that

the

rules

should

be

changed

to

deter

large

procurement.

The

task

facing

game

organizers

can

be

viewed

as

a

problem

in

mechanism

design.

The

designers

have

certain

game

features

under

their

control,

and

a

set

of

objectives

regarding

game

outcomes.

Unlike

most

academic

treatments

of

mechanism

design,

the

objective

is

a

behavioral

feature

(moderate

procurement)

rather

than

an

allocation

feature

like

economic

efficiency,

and

the

allowed

mechanisms

are

restricted

to

those

judged

to

require

only

an

incremental

modification

of

the

current

game.

Replacing

the

supplychain

negotiation

procedures

with

a

one-shot

direct

mechanism,

for

example,

was

not

an

option.

We

believe

that

such

operational

restrictions

and

idiosyncratic

objectives

are

actually

quite

typical

of

practical

mechanism

design

settings,

where

they

are

perhaps

more

commonly

characterized

as

incentive

engineering

problems.

In

response

to

the

problem,

the

TAC/SCM

designers

adopted

several

rule

changes

intended

to

penalize

large

orders.

These

included

modifications

to

supplier

pricing

policies

and

introduction

of

storage

costs

assessed

on

inventories

of

components

and

finished

goods.

Despite

the

changes,

procurement

was

very

high

in

the

early

rounds

of

the

competition.

In

a

drastic

measure,

the

GameMaster

imposed

a

fivefold

increase

of

storage

costs

midway

through

the

tournament.

Even

this

did

not

stem

the

tide,

and

procurement

in

the

final

rounds

actually

increased

(by

some

measures)

from

The

apparent

difficulty

in

identifying

rule

modifications

that

effect

moderation

in

procurement

is

quite

striking.

Although

the

designs

were

widely

discussed,

predictions

for

the

effects

of

various

proposals

were

supported

primarily

by

intuitive

arguments

or

at

best

by

back-of-the-envelope

calculations.

Much

of

the

difficulty,

of

course,

is

anticipating

the

agents"

(and

their

developers")

responses

without

essentially

running

a

gaming

exercise

for

this

purpose.

The

episode

caused

us

to

consider

whether

new

proaches

or

tools

could

enable

more

systematic

analysis

of

design

options.

Standard

game-theoretic

and

mechanism

design

methods

are

clearly

relevant,

although

the

lack

of

an

analytic

description

of

the

game

seems

to

be

an

impediment.

Under

the

assumption

that

the

simulator

itself

is

the

only

reliable

source

of

outcome

computation,

we

refer

to

our

task

as

empirical

mechanism

design.

In

the

sequel,

we

develop

some

general

methods

for

empirical

mechanism

design

and

apply

them

to

the

TAC/SCM

redesign

problem.

Our

analysis

focuses

on

the

setting

of

storage

costs

(taking

other

game

modifications

as

fixed),

since

this

is

the

most

direct

deterrent

to

early

procurement

adopted.

Our

results

confirm

the

basic

intuition

that

incentives

for

purchasing

decrease

as

storage

costs

rise.

We

also

confirm

that

the

high

procurement

observed

in

the

tournament

is

a

rational

response

to

the

setting

of

storage

costs

used.

Finally,

we

conclude

from

our

data

that

it

is

very

unlikely

that

any

reasonable

setting

of

storage

costs

would

result

in

acceptable

levels

of

procurement,

so

a

different

design

approach

would

have

been

required

to

eliminate

this

problem.

Overall,

we

contribute

a

formal

framework

and

a

set

of

methods

for

tackling

indirect

mechanism

design

problems

in

settings

where

only

a

black-box

description

of

players"

utilities

is

available.

Our

methods

incorporate

estimation

of

sets

of

Nash

equilibria

and

sample

Nash

equilibria,

used

in

conjuction

to

support

general

claims

about

the

structure

of

the

mechanism

designer"s

utility,

as

well

as

a

restricted

probabilistic

analysis

to

assess

the

likelihood

of

conclusions.

We

believe

that

most

realistic

problems

are

too

complex

to

be

amenable

to

exact

analysis.

Consequently,

we

advocate

the

approach

of

gathering

evidence

to

provide

indirect

support

of

specific

hypotheses.

PRELIMINARIES

A

normal

form

is

denoted

by

[I,

{Ri},

{ui(r)}],

where

I

refers

to

the

set

of

players

and

m

|I|

is

the

number

of

players.

Ri

is

the

set

of

strategies

available

to

player

i

I,

with

R

.×Rm

representing

the

set

of

joint

strategies

of

all

players.

We

designate

the

set

of

pure

strategies

available

to

player

i

by

Ai,

and

denote

the

joint

set

of

pure

strategies

of

all

players

by

A

.×Am.

It

is

often

convenient

to

refer

to

a

strategy

of

player

i

separately

from

that

of

the

remaining

players.

To

accommodate

this,

we

use

a−i

to

denote

the

joint

strategy

of

all

players

other

than

player

i.

Let

Si

be

the

set

of

all

probability

distributions

(mixtures)

over

Ai

and,

similarly,

S

be

the

set

of

all

distributions

over

A.

An

s

S

is

called

a

mixed

strategy

profile.

When

the

game

is

finite

(i.e.,

A

and

I

are

both

finite),

the

probability

that

a

A

is

played

under

s

is

written

s(a)

s(ai,

a−i).

When

the

distribution

s

is

not

correlated,

we

can

simply

say

si(ai)

when

referring

to

the

probability

player

i

plays

ai

under

s.

Next,

we

define

the

payoff

(utility)

function

of

each

player

i

by

ui

·×Am

R,

where

ui(ai,

a−i)

indicates

the

payoff

to

player

i

to

playing

pure

strategy

ai

when

the

remaining

players

play

a−i.

We

can

extend

this

definition

to

mixed

strategies

by

assuming

that

ui

are

von

Neumann-Morgenstern

(vNM)

utilities

as

follows:

ui(s)

Es[ui],

where

Es

is

the

expectation

taken

with

respect

to

the

probability

distribution

of

play

induced

by

the

players"

mixed

strategy

s.

By

employing

the

normal

form,

we

model

agents

as

playing

a

single

action,

with

decisions

taken

simultaneously.

This

is

appropriate

for

our

current

study,

which

treats

strategies

(agent

programs)

as

atomic

actions.

We

could

capture

finer-grained

decisions

about

action

over

time

in

the

extensive

form.

Although

any

extensive

game

can

be

recast

in

normal

form,

doing

so

may

sacrifice

compactness

and

blur

relevant

distinctions

(e.g.,

subgame

perfection).

Occasionally,

we

write

ui(x,

y)

to

mean

that

x

Ai

or

Si

and

y

A−i

or

S−i

depending

on

context.

We

also

express

the

set

of

utility

functions

of

all

players

as

u(·)

um(·)}.

We

define

a

function,

R

R,

interpreted

as

the

maximum

benefit

any

player

can

obtain

by

deviating

from

its

strategy

in

the

specified

profile.

(r)

max

i∈I

max

ai∈Ai

[ui(ai,

r−i)

ui(r)],

where

r

belongs

to

some

strategy

set,

R,

of

either

pure

or

mixed

strategies.

Faced

with

a

game,

an

agent

would

ideally

play

its

best

strategy

given

those

played

by

the

other

agents.

A

configuration

where

all

agents

play

strategies

that

are

best

responses

to

the

others

constitutes

a

Nash

equilibrium.

DEFINITION

A

strategy

profile

r

rm)

constitutes

a

Nash

equilibrium

of

game

[I,

{Ri},

{ui(r)}]

if

for

every

i

I,

ri

Ri,

ui(ri,

r−i)

ui(ri,

r−i).

When

r

A,

the

above

defines

a

pure

strategy

Nash

equilibrium;

otherwise

the

definition

describes

a

mixed

strategy

Nash

equilibrium.

We

often

appeal

to

the

concept

of

an

approximate,

or

-Nash

equilibrium,

where

is

the

maximum

benefit

to

any

agent

for

deviating

from

the

prescribed

strategy.

Thus,

(r)

as

defined

above

is

such

that

profile

r

is

an

-Nash

equilibrium

iff

(r)

In

this

study

we

devote

particular

attention

to

games

that

exhibit

symmetry

with

respect

to

payoffs,

rendering

agents

strategically

identical.

DEFINITION

A

game

[I,

{Ri},

{ui(r)}]

is

symmetric

if

for

all

i,

j

I,

(a)

Ri

Rj

and

(b)

ui(ri,

r−i)

uj

(rj,

r−j)

whenever

ri

rj

and

r−i

r−j

THE

MODEL

We

model

the

strategic

interactions

between

the

designer

of

the

mechanism

and

its

participants

as

a

two-stage

game.

The

designer

moves

first

by

selecting

a

value,

θ,

from

a

set

of

allowable

mechanism

settings,

Θ.

All

the

participant

agents

observe

the

mechanism

parameter

θ

and

move

simultaneously

thereafter.

For

example,

the

designer

could

be

deciding

between

a

first-price

and

second-price

sealed-bid

auction

mechanisms,

with

the

presumption

that

after

the

choice

has

been

made,

the

bidders

will

participate

with

full

awareness

of

the

auction

rules.

Since

the

participants

play

with

full

knowledge

of

the

mechanism

parameter,

we

define

a

game

between

them

in

the

second

stage

as

Γθ

[I,

{Ri},

{ui(r,

θ)}].

We

refer

to

Γθ

as

a

game

induced

by

θ.

Let

N(θ)

be

the

set

of

strategy

profiles

considered

solutions

of

the

game

Γθ.3

Suppose

that

the

goal

of

the

designer

is

to

optimize

the

value

of

some

welfare

function,

W

(r,

θ),

dependent

on

the

mechanism

parameter

and

resulting

play,

r.

We

define

a

pessimistic

measure,

W

ˆR,

θ)

inf{W

(r,

θ)

r

ˆR},

representing

the

worst-case

welfare

of

the

game

induced

by

θ,

assuming

that

agents

play

some

joint

strategy

in

ˆR.

Typically

we

care

about

W

(N(θ),

θ),

the

worst-case

outcome

of

playing

some

On

some

problems

we

can

gain

considerable

advantage

by

using

an

aggregation

function

to

map

the

welfare

outcome

of

a

game

We

generally

adopt

Nash

equilibrium

as

the

solution

concept,

and

thus

take

N(θ)

to

be

the

set

of

equilibria.

However,

much

of

the

methodology

developed

here

could

be

employed

with

alternative

criteria

for

deriving

agent

behavior

from

a

game

definition.

Again,

alternatives

are

available.

For

example,

if

one

has

a

probability

distribution

over

the

solution

set

N(θ),

it

would

be

natural

to

take

the

expectation

of

W

(r,

θ)

instead.

specified

in

terms

of

agent

strategies

to

an

equivalent

welfare

outcome

specified

in

terms

of

a

lower-dimensional

summary.

DEFINITION

A

function

φ

Rm

Rq

is

an

aggregation

function

if

m

q

and

W

(r,

θ)

V

(φ(r),

θ)

for

some

function

V

We

overload

the

function

symbol

to

apply

to

sets

of

strategy

profiles:

φ(

ˆR)

{φ(r)

r

ˆR}.

For

convenience

of

exposition,

we

write

φ∗

(θ)

to

mean

φ(N(θ)).

Using

an

aggregation

function

yields

a

more

compact

representation

of

strategy

profiles.

For

example,

suppose-as

in

our

application

below-that

an

agent"s

strategy

is

defined

by

a

numeric

parameter.

If

all

we

care

about

is

the

total

value

played,

we

may

take

φ(a)

Pm

ai.

If

we

have

chosen

our

aggregator

carefully,

we

may

also

capture

structure

not

obvious

otherwise.

For

example,

φ∗

(θ)

could

be

decreasing

in

θ,

whereas

N(θ)

might

have

a

more

complex

structure.

Given

a

description

of

the

solution

correspondence

N(θ)

(equivalently,

φ∗

(θ)),

the

designer

faces

a

standard

optimization

problem.

Alternatively,

given

a

simulator

that

could

produce

an

unbiased

sample

from

the

distribution

of

W

(N(θ),

θ)

for

any

θ,

the

designer

would

be

faced

with

another

much

appreciated

problem

in

the

literature:

simulation

optimization

However,

even

for

a

game

Γθ

with

known

payoffs

it

may

be

computationally

intractable

to

solve

for

Nash

equilibria,

particularly

if

the

game

has

large

or

infinite

strategy

sets.

Additionally,

we

wish

to

study

games

where

the

payoffs

are

not

explicitly

given,

but

must

be

determined

from

simulation

or

other

experience

with

the

Accordingly,

we

assume

that

we

are

given

a

(possibly

noisy)

data

set

of

payoff

realizations:

Do

(θk

ak

Uk

where

for

every

data

point

θi

is

the

observed

mechanism

parameter

setting,

ai

is

the

observed

pure

strategy

profile

of

the

participants,

and

Ui

is

the

corresponding

realization

of

agent

payoffs.

We

may

also

have

additional

data

generated

by

a

(possibly

noisy)

simulator:

Ds

(θk+l

ak+l

Uk+l

Let

D

{Do,

Ds}

be

the

combined

data

set.

(Either

Do

or

Ds

may

be

for

a

particular

problem.)

In

the

remainder

of

this

paper,

we

apply

our

modeling

approach,

together

with

several

empirical

game-theoretic

methods,

in

order

to

answer

questions

regarding

the

design

of

the

TAC/SCM

scenario.

EMPIRICAL

DESIGN

ANALYSIS

Since

our

data

comes

in

the

form

of

payoff

experience

and

not

as

the

value

of

an

objective

function

for

given

settings

of

the

control

variable,

we

can

no

longer

rely

on

the

methods

for

optimizing

functions

using

simulations.

Indeed,

a

fundamental

aspect

of

our

design

problem

involves

estimating

the

Nash

equilibrium

correspondence.

Furthermore,

we

cannot

rely

directly

on

the

convergence

results

that

abound

in

the

simulation

optimization

literature,

and

must

establish

probabilistic

analysis

methods

tailored

for

our

problem

setting.

TAC/SCM

Design

Problem

We

describe

our

empirical

design

analysis

methods

by

presenting

a

detailed

application

to

the

TAC/SCM

scenario

introduced

above.

Recall

that

during

the

tournament,

the

designers

of

the

supplychain

game

chose

to

dramatically

increase

storage

costs

as

a

measure

aimed

at

curbing

procurement,

to

little

avail.

Here

we

systematically

explore

the

relationship

between

storage

costs

and

This

is

often

the

case

for

real

games

of

interest,

where

natural

language

or

algorithmic

descriptions

may

substitute

for

a

formal

specification

of

strategy

and

payoff

functions.

the

aggregate

quantity

of

components

procured

on

day

in

equilibrium.

In

doing

so,

we

consider

several

questions

raised

during

and

after

the

tournament.

First,

does

increasing

storage

costs

actually

reduce

procurement?

Second,

was

the

excessive

procurement

that

was

observed

during

the

tournament

rational?

And

third,

could

increasing

storage

costs

sufficiently

have

reduced

procurement

to

an

acceptable

level,

and

if

so,

what

should

the

setting

of

storage

costs

have

been?

It

is

this

third

question

that

defines

the

mechanism

design

aspect

of

our

To

apply

our

methods,

we

must

specify

the

agent

strategy

sets,

the

designer"s

welfare

function,

the

mechanism

parameter

space,

and

the

source

of

data.

We

restrict

the

agent

strategies

to

be

a

multiplier

on

the

quantity

of

the

requests

by

one

of

the

finalists,

Deep

Maize,

in

the

TAC/SCM

tournament.

We

further

restrict

it

to

the

set

since

any

strategy

below

is

illegal

and

strategies

above

are

extremely

aggressive

(thus

unlikely

to

provide

refuting

deviations

beyond

those

available

from

included

strategies,

and

certainly

not

part

of

any

desirable

equilibrium).

All

other

behavior

is

based

on

the

behavior

of

Deep

Maize

and

is

identical

for

all

agents.

This

choice

can

provide

only

an

estimate

of

the

actual

tournament

behavior

of

a

typical

agent.

However,

we

believe

that

the

general

form

of

the

results

should

be

robust

to

changes

in

the

full

agent

behavior.

We

model

the

designer"s

welfare

function

as

a

threshold

on

the

sum

of

purchases.

Let

φ(a)

ai

be

the

aggregation

function

representing

the

sum

of

procurement

of

the

six

agents

participating

in

a

particular

supply-chain

game

(for

mixed

strategy

profiles

s,

we

take

expectation

of

φ

with

respect

to

the

mixture).

The

designer"s

welfare

function

W

(N(θ),

θ)

is

then

given

by

I{sup{φ∗

(θ)}

α},

where

α

is

the

maximum

acceptable

level

of

procurement

and

I

is

the

indicator

function.

The

designer

selects

a

value

θ

of

storage

costs,

expressed

as

an

annual

percentage

of

the

baseline

value

of

components

in

the

inventory

(charged

daily),

from

the

set

Θ

R+

Since

the

designer"s

decision

depends

only

on

φ∗

(θ),

we

present

all

of

our

results

in

terms

of

the

value

of

the

aggregation

function.

Estimating

Nash

Equilibria

The

objective

of

TAC/SCM

agents

is

to

maximize

profits

realized

over

a

game

instance.

Thus,

if

we

fix

a

strategy

for

each

agent

at

the

beginning

of

the

simulation

and

record

the

corresponding

profits

at

the

end,

we

will

have

obtained

a

data

point

in

the

form

(a,

U(a)).

If

we

also

have

fixed

the

parameter

θ

of

the

simulator,

the

resulting

data

point

becomes

part

of

our

data

set

D.

This

data

set,

then,

contains

data

only

in

the

form

of

pure

strategies

of

players

and

their

corresponding

payoffs,

and,

consequently,

in

order

to

formulate

the

designer"s

problem

as

optimization,

we

must

first

determine

or

approximate

the

set

of

Nash

equilibria

of

each

game

Γθ.

Thus,

we

need

methods

for

approximating

Nash

equilibria

for

infinite

games.

Below,

we

describe

the

two

methods

we

used

in

our

study.

The

first

has

been

explored

empirically

before,

whereas

the

second

is

introduced

here

as

the

method

specifically

designed

to

approximate

a

set

of

Nash

equilibria.

Payoff

Function

Approximation

The

first

method

for

estimating

Nash

equilibria

based

on

data

uses

supervised

learning

to

approximate

payoff

functions

of

We

do

not

address

whether

and

how

other

measures

(e.g.,

constraining

procurement

directly)

could

have

achieved

design

objectives.

Our

approach

takes

as

given

some

set

of

design

options,

in

this

case

defined

by

the

storage

cost

parameter.

In

principle

our

methods

could

be

applied

to

a

different

or

larger

design

space,

though

with

corresponding

complexity

growth.

anism

participants

from

a

data

set

of

game

experience

Once

approximate

payoff

functions

are

available

for

all

players,

the

Nash

equilibria

may

be

either

found

analytically

or

approximated

using

numerical

techniques,

depending

on

the

learning

model.

In

what

follows,

we

estimate

only

a

sample

Nash

equilibrium

using

this

technique,

although

this

restriction

can

be

removed

at

the

expense

of

additional

computation

time.

One

advantage

of

this

method

is

that

it

can

be

applied

to

any

data

set

and

does

not

require

the

use

of

a

simulator.

Thus,

we

can

apply

it

when

Ds

If

a

simulator

is

available,

we

can

generate

additional

data

to

build

confidence

in

our

initial

We

tried

the

following

methods

for

approximating

payoff

functions:

quadratic

regression

(QR),

locally

weighted

average

(LWA),

and

locally

weighted

linear

regression

(LWLR).

We

also

used

control

variates

to

reduce

the

variance

of

payoff

estimates,

as

in

our

previous

empirical

game-theoretic

analysis

of

The

quadratic

regression

model

makes

it

possible

to

compute

equilibria

of

the

learned

game

analytically.

For

the

other

methods

we

applied

replicator

dynamics

to

a

discrete

approximation

of

the

learned

game.

The

expected

total

procurement

in

equilibrium

was

taken

as

the

estimate

of

an

outcome.

Search

in

Strategy

Profile

Space

When

we

have

access

to

a

simulator,

we

can

also

use

directed

search

through

profile

space

to

estimate

the

set

of

Nash

equilibria,

which

we

describe

here

after

presenting

some

additional

notation.

DEFINITION

A

strategic

neighbor

of

a

pure

strategy

profile

a

is

a

profile

that

is

identical

to

a

in

all

but

one

strategy.

We

define

Snb(a,

D)

as

the

set

of

all

strategic

neighbors

of

a

available

in

the

data

set

D.

Similarly,

we

define

Snb(a,

˜D)

to

be

all

strategic

neighbors

of

a

not

in

D.

Finally,

for

any

a

Snb(a,

D)

we

define

the

deviating

agent

as

i(a,

a

DEFINITION

The

-bound,

ˆ,

of

a

pure

strategy

profile

a

is

defined

as

maxa

∈Snb(a,D)

max{ui(a,a

)(a

)−ui(a,a

)(a),

We

say

that

a

is

a

candidate

δ-equilibrium

for

δ

ˆ.

When

Snb(a,

˜D)

(i.e.,

all

strategic

neighbors

are

represented

in

the

data),

a

is

confirmed

as

an

ˆ-Nash

equilibrium.

Our

search

method

operates

by

exploring

deviations

from

candidate

equilibria.

We

refer

to

it

as

BestFirstSearch,

as

it

selects

with

probability

one

a

strategy

profile

a

Snb(a,

˜D)

that

has

the

smallest

ˆin

D.

Finally

we

define

an

estimator

for

a

set

of

Nash

equilibria.

DEFINITION

For

a

set

K,

define

Co(K)

to

be

the

convex

hull

of

K.

Let

Bδ

be

the

set

of

candidates

at

level

δ.

We

define

ˆφ∗

(θ)

Co({φ(a)

a

Bδ})

for

a

fixed

δ

to

be

an

estimator

of

φ∗

(θ).

In

words,

the

estimate

of

a

set

of

equilibrium

outcomes

is

the

convex

hull

of

all

aggregated

strategy

profiles

with

-bound

below

some

fixed

δ.

This

definition

allows

us

to

exploit

structure

arising

from

the

aggregation

function.

If

two

profiles

are

close

in

terms

of

aggregation

values,

they

may

be

likely

to

have

similar

-bounds.

In

particular,

if

one

is

an

equilibrium,

the

other

may

be

as

well.

We

present

some

theoretical

support

for

this

method

of

estimating

the

set

of

Nash

equilibria

below.

Since

the

game

we

are

interested

in

is

infinite,

it

is

necessary

to

terminate

BestFirstSearch

before

exploring

the

entire

space

of

For

example,

we

can

use

active

learning

techniques

to

improve

the

quality

of

payoff

function

approximation.

In

this

work,

we

instead

concentrate

on

search

in

strategy

profile

space.

egy

profiles.

We

currently

determine

termination

time

in

a

somewhat

ad-hoc

manner,

based

on

observations

about

the

current

set

of

candidate

Data

Generation

Our

data

was

collected

by

simulating

TAC/SCM

games

on

a

local

version

of

the

TAC/SCM

server,

which

has

a

configuration

setting

for

the

storage

cost.

Agent

strategies

in

simulated

games

were

selected

from

the

set

in

order

to

have

positive

probability

of

generating

strategic

A

baseline

data

set

Do

was

generated

by

sampling

randomly

generated

strategy

profiles

for

each

θ

Between

and

games

were

run

for

each

profile

after

discarding

games

that

had

various

We

used

search

to

generate

a

simulated

data

set

Ds,

performing

between

and

iterations

of

BestFirstSearch

for

each

of

the

above

settings

of

θ.

Since

simulation

cost

is

extremely

high

(a

game

takes

nearly

hour

to

run),

we

were

able

to

run

a

total

of

games

over

the

span

of

more

than

six

months.

For

comparison,

to

get

the

entire

description

of

an

empirical

game

defined

by

the

restricted

finite

joint

strategy

space

for

each

value

of

θ

would

have

required

at

least

games

(sampling

each

profile

times).

Results

Analysis

of

the

Baseline

Data

Set

We

applied

the

three

learning

methods

described

above

to

the

baseline

data

set

Do.

Additionally,

we

generated

an

estimate

of

the

Nash

equilibrium

correspondence,

ˆφ∗

(θ),

by

applying

Definition

with

δ

The

results

are

shown

in

Figure

As

we

can

see,

the

correspondence

ˆφ∗

(θ)

has

little

predictive

power

based

on

Do,

and

reveals

no

interesting

structure

about

the

game.

In

contrast,

all

three

learning

methods

suggest

that

total

procurement

is

a

decreasing

function

of

storage

costs.

Storage

Cost

LWA

LWLR

QR

BaselineMin

BaselineMax

Figure

Aggregate

procurement

estimates

based

on

Do.

The

correspondence

ˆφ∗

(θ)

is

the

interval

between

BaselineMin

and

BaselineMax.

Generally,

search

is

terminated

once

the

set

of

candidate

equilibria

is

small

enough

to

draw

useful

conclusions

about

the

likely

range

of

equilibrium

strategies

in

the

game.

Of

course,

we

do

not

restrict

our

Nash

equilibrium

estimates

to

stay

in

this

discrete

subset

of

For

example,

if

we

detected

that

any

agent

failed

during

the

game

(failures

included

crashes,

network

connectivity

problems,

and

other

obvious

anomalies),

the

game

would

be

thrown

out.

Analysis

of

Search

Data

To

corroborate

the

initial

evidence

from

the

learning

methods,

we

estimated

ˆφ∗

(θ)

(again,

using

δ

on

the

data

set

D

{Do,

Ds},

where

Ds

is

data

generated

through

the

application

of

BestFirstSearch.

The

results

of

this

estimate

are

plotted

against

the

results

of

the

learning

methods

trained

on

Do

in

Figure

First,

we

note

that

the

addition

of

the

search

data

narrows

the

range

of

potential

equilibria

substantially.

Furthermore,

the

actual

point

predictions

of

the

learning

methods

and

those

based

on

-bounds

after

search

are

reasonably

close.

Combining

the

evidence

gathered

from

these

two

very

different

approaches

to

estimating

the

outcome

correspondence

yields

a

much

more

compelling

picture

of

the

relationship

between

storage

costs

and

procurement

than

either

method

used

in

isolation.

Storage

Cost

LWA

LWLR

QR

SearchMin

SearchMax

Figure

Aggregate

procurement

estimates

based

on

search

in

strategy

profile

space

compared

to

function

approximation

techniques

trained

on

Do.

The

correspondence

ˆφ∗

(θ)

for

D

{Do,

Ds}

is

the

interval

between

SearchMin

and

SearchMax.

This

evidence

supports

the

initial

intuition

that

procurement

should

be

decreasing

with

storage

costs.

It

also

confirms

that

high

levels

of

procurement

are

a

rational

response

to

the

tournament

setting

of

average

storage

cost,

which

corresponds

to

θ

The

minimum

prediction

for

aggregate

procurement

at

this

level

of

storage

costs

given

by

any

experimental

methods

is

approximately

This

is

quite

high,

as

it

corresponds

to

an

expected

commitment

of

of

the

total

supplier

capacity

for

the

entire

game.

The

maximum

prediction

is

considerably

higher

at

In

the

actual

competition,

aggregate

procurement

was

equivalent

to

on

the

scale

used

here

Our

predictions

underestimate

this

outcome

to

some

degree,

but

show

that

any

rational

outcome

was

likely

to

have

high

procurement.

Extrapolating

the

Solution

Correspondence

We

have

reasonably

strong

evidence

that

the

outcome

correspondence

is

decreasing.

However,

the

ultimate

goal

is

to

be

able

to

either

set

the

storage

cost

parameter

to

a

value

that

would

curb

procurement

in

equilibrium

or

conclude

that

this

is

not

possible.

To

answer

this

question

directly,

suppose

that

we

set

a

conservative

threshold

α

on

aggregate

Linear

It

is

unclear

how

meaningful

the

results

of

learning

would

be

if

Ds

were

added

to

the

training

data

set.

Indeed,

the

additional

data

may

actually

increase

the

learning

variance.

Recall

that

designer"s

objective

is

to

incentivize

aggergate

procurement

that

is

below

the

threshold

α.

Our

threshold

here

still

represents

a

commitment

of

over

of

the

suppliers"

capacity

for

extrapolation

of

the

maximum

of

the

outcome

correspondence

estimated

from

D

yields

θ

The

data

for

θ

were

collected

in

the

same

way

as

for

other

storage

cost

settings,

with

randomly

generated

profiles

followed

by

iterations

of

BestFirstSearch.

Figure

shows

the

detailed

-bounds

for

all

profiles

in

terms

of

their

corresponding

values

of

φ.

Total

Procurement

ε−boundFigure

Values

of

ˆ

for

profiles

explored

using

search

when

θ

Strategy

profiles

explored

are

presented

in

terms

of

the

corresponding

values

of

φ(a).

The

gray

region

corresponds

to

ˆφ∗

with

δ

The

estimated

set

of

aggregate

outcomes

is

very

close

to

that

for

θ

indicating

that

there

is

little

additional

benefit

to

raising

storage

costs

above

Observe,

that

even

the

lower

bound

of

our

estimated

set

of

Nash

equilibria

is

well

above

the

target

procurement

of

Furthermore,

payoffs

to

agents

are

almost

always

negative

at

θ

Consequently,

increasing

the

costs

further

would

be

undesirable

even

if

procurement

could

eventually

be

curbed.

Since

we

are

reasonably

confident

that

φ∗

(θ)

is

decreasing

in

θ,

we

also

do

not

expect

that

setting

θ

somewhere

between

and

will

achieve

the

desired

result.

We

conclude

that

it

is

unlikely

that

procurement

could

ever

be

reduced

to

a

desirable

level

using

any

reasonable

setting

of

the

storage

cost

parameter.

That

our

predictions

tend

to

underestimate

tournament

outcomes

reinforces

this

conclusion.

To

achieve

the

desired

reduction

in

procurement

requires

redesigning

other

aspects

of

the

mechanism.

Probabilistic

Analysis

Our

empirical

analysis

has

produced

evidence

in

support

of

the

conclusion

that

no

reasonable

setting

of

storage

cost

was

likely

to

sufficiently

curb

excessive

procurement

in

TAC/SCM

All

of

this

evidence

has

been

in

the

form

of

simple

interpolation

and

extrapolation

of

estimates

of

the

Nash

equilibrium

correspondence.

These

estimates

are

based

on

simulating

game

instances,

and

are

subject

to

sampling

noise

contributed

by

the

various

stochastic

elements

of

the

game.

In

this

section,

we

develop

and

apply

methods

for

evaluating

the

sensitivity

of

our

-bound

calculations

to

such

stochastic

effects.

Suppose

that

all

agents

have

finite

(and

small)

pure

strategy

sets,

A.

Thus,

it

is

feasible

to

sample

the

entire

payoff

matrix

of

the

game.

Additionally,

suppose

that

noise

is

additive

with

zero-mean

the

entire

game

on

average,

so

in

practice

we

would

probably

want

the

threshold

to

be

even

lower.

and

finite

variance,

that

is,

Ui(a)

ui(a)

˜ξi(a),

where

Ui(a)

is

the

observed

payoff

to

i

when

a

was

played,

ui(a)

is

the

actual

corresponding

payoff,

and

˜ξi(a)

is

a

mean-zero

normal

random

variable.

We

designate

the

known

variance

of

˜ξi(a)

by

σ2

i

(a).

Thus,

we

assume

that

˜ξi(a)

is

normal

with

distribution

σ2

i

(a)).

We

take

¯ui(a)

to

be

the

sample

mean

over

all

Ui(a)

in

D,

and

follow

Chang

and

Huang

to

assume

that

we

have

an

improper

prior

over

the

actual

payoffs

ui(a)

and

sampling

was

independent

for

all

i

and

a.

We

also

rely

on

their

result

that

ui(a)|¯ui(a)

¯ui(a)−Zi(a)/[σi(a)/

p

ni(a)]

are

independent

with

posterior

distributions

N(¯ui(a),

σ2

i

(a)/ni(a)),

where

ni(a)

is

the

number

of

samples

taken

of

payoffs

to

i

for

pure

profile

a,

and

Zi(a)

We

now

derive

a

generic

probabilistic

bound

that

a

profile

a

A

is

an

-Nash

equilibrium.

If

ui(·)|¯ui(·)

are

independent

for

all

i

I

and

a

A,

we

have

the

following

result

(from

this

point

on

we

omit

conditioning

on

¯ui(·)

for

brevity):

PROPOSITION

Pr

max

i∈I

max

b∈Ai

ui(b,

a−i)

ui(a)

Y

i∈I

Z

R

Y

b∈Ai\ai

Pr(ui(b,

a−i)

u

)fui(a)(u)du,

where

fui(a)(u)

is

the

pdf

of

N(¯ui(a),

σi(a)).

The

proofs

of

this

and

all

subsequent

results

are

in

the

Appendix.

The

posterior

distribution

of

the

optimum

mean

of

n

samples,

derived

by

Chang

and

Huang

is

Pr

(ui(a)

c)

Φ

"p

ni(a)(¯ui(a)

c)

σi(a)

where

a

A

and

Φ(·)

is

the

distribution

function.

Combining

the

results

and

we

obtain

a

probabilistic

confidence

bound

that

(a)

γ

for

a

given

γ.

Now,

we

consider

cases

of

incomplete

data

and

use

the

results

we

have

just

obtained

to

construct

an

upper

bound

(restricted

to

profiles

represented

in

data)

on

the

distribution

of

sup{φ∗

(θ)}

and

inf{φ∗

(θ)}

(assuming

that

both

are

attainable):

Pr{sup{φ∗

(θ)}

x}

≤D

Pr{∃a

D

φ(a)

x

a

N(θ)}

X

a∈D:φ(a)≤x

Pr{a

N(θ)}

X

a∈D:φ(a)≤x

Pr{

(a)

where

x

is

a

real

number

and

≤D

indicates

that

the

upper

bound

accounts

only

for

strategies

that

appear

in

the

data

set

D.

Since

the

events

{∃a

D

φ(a)

x

a

N(θ)}

and

{inf{φ∗

(θ)}

x}

are

equivalent,

this

also

defines

an

upper

bound

on

the

probability

of

{inf{φ∗

(θ)}

x}.

The

values

thus

derived

comprise

the

Tables

and

φ∗

(θ)

θ

θ

θ

Table

Upper

bounds

on

the

distribution

of

inf{φ∗

(θ)}

restricted

to

D

for

θ

when

N(θ)

is

a

set

of

Nash

equilibria.

φ∗

(θ)

θ

θ

θ

Table

Upper

bounds

on

the

distribution

of

inf{φ∗

(θ)}

restricted

to

D

for

θ

when

N(θ)

is

a

set

of

Nash

equilibria.

Tables

and

suggest

that

the

existence

of

any

equilibrium

with

φ(a)

is

unlikely

for

any

θ

that

we

have

data

for,

although

this

judgment,

as

we

mentioned,

is

only

with

respect

to

the

profiles

we

have

actually

sampled.

We

can

then

accept

this

as

another

piece

of

evidence

that

the

designer

could

not

find

a

suitable

setting

of

θ

to

achieve

his

objectives-indeed,

the

designer

seems

unlikely

to

achieve

his

objective

even

if

he

could

persuade

participants

to

play

a

desirable

equilibrium!

Table

also

provides

additional

evidence

that

the

agents

in

the

TAC/SCM

tournament

were

indeed

rational

in

procuring

large

numbers

of

components

at

the

beginning

fo

the

game.

If

we

look

at

the

third

column

of

this

table,

which

corresponds

to

θ

we

can

gather

that

no

profile

a

in

our

data

with

φ(a)

is

very

likely

to

be

played

in

equilibrium.

The

bounds

above

provide

some

general

evidence,

but

ultimately

we

are

interested

in

a

concrete

probabilistic

assessment

of

our

conclusion

with

respect

to

the

data

we

have

sampled.

Particularly,

we

would

like

to

say

something

about

what

happens

for

the

settings

of

θ

for

which

we

have

no

data.

To

derive

an

approximate

probabilistic

bound

on

the

probability

that

no

θ

Θ

could

have

achieved

the

designer"s

objective,

let

∪J

be

a

partition

of

Θ,

and

assume

that

the

function

sup{φ∗

(θ)}

satisfies

the

Lipschitz

condition

with

Lipschitz

constant

Aj

on

each

subset

Since

we

have

determined

that

raising

the

storage

cost

above

is

undesirable

due

to

secondary

considerations,

we

restrict

attention

to

Θ

We

now

define

each

subset

j

to

be

the

interval

between

two

points

for

which

we

have

produced

data.

Thus,

Θ

with

j

running

between

and

corresponding

to

subintervals

above.

We

will

further

denote

each

Θj

by

(aj

Then,

the

following

Proposition

gives

us

an

approximate

upper

on

the

probability

that

sup{φ∗

(θ)}

α.

PROPOSITION

Pr{

θ∈Θ

sup{φ(θ)}

α}

≤D

X

y,z∈D:y+z≤cj

X

a:φ(a)=z

Pr{

(a)

A

X

a:φ(a)=y

Pr{

(a)

A

where

cj

2α

Aj(bj

aj)

and

≤D

indicates

that

the

upper

bound

only

accounts

for

strategies

that

appear

in

the

data

set

D.

A

function

that

satisfies

the

Lipschitz

condition

is

called

Lipschitz

continuous.

The

treatment

for

the

interval

is

identical.

It

is

approximate

in

a

sense

that

we

only

take

into

account

strategies

that

are

present

in

the

data.

Due

to

the

fact

that

our

bounds

are

approximate,

we

cannot

use

them

as

a

conclusive

probabilistic

assessment.

Instead,

we

take

this

as

another

piece

of

evidence

to

complement

our

findings.

Even

if

we

can

assume

that

a

function

that

we

approximate

from

data

is

Lipschitz

continuous,

we

rarely

actually

know

the

Lipschitz

constant

for

any

subset

of

Θ.

Thus,

we

are

faced

with

a

task

of

estimating

it

from

data.

Here,

we

tried

three

methods

of

doing

this.

The

first

one

simply

takes

the

highest

slope

that

the

function

attains

within

the

available

data

and

uses

this

constant

value

for

every

subinterval.

This

produces

the

most

conservative

bound,

and

in

many

situations

it

is

unlikely

to

be

informative.

An

alternative

method

is

to

take

an

upper

bound

on

slope

obtained

within

each

subinterval

using

the

available

data.

This

produces

a

much

less

conservative

upper

bound

on

probabilities.

However,

since

the

actual

upper

bound

is

generally

greater

for

each

subinterval,

the

resulting

probabilistic

bound

may

be

deceiving.

A

final

method

that

we

tried

is

a

compromise

between

the

two

above.

Instead

of

taking

the

conservative

upper

bound

based

on

data

over

the

entire

function

domain

Θ,

we

take

the

average

of

upper

bounds

obtained

at

each

Θj.

The

bound

at

an

interval

is

then

taken

to

be

the

maximum

of

the

upper

bound

for

this

interval

and

the

average

upper

bound

for

all

intervals.

The

results

of

evaluating

the

expression

for

Pr{

θ∈Θ

sup{φ∗

(θ)}

α}

when

α

are

presented

in

Table

In

terms

of

our

claims

in

maxj

Aj

Aj

max{Aj

,ave(Aj)}

Table

Approximate

upper

bound

on

probability

that

some

setting

of

θ

will

satisfy

the

designer

objective

with

target

α

Different

methods

of

approximating

the

upper

bound

on

slope

in

each

subinterval

j

are

used.

this

work,

the

expression

gives

an

upper

bound

on

the

probability

that

some

setting

of

θ

(i.e.,

storage

cost)

in

the

interval

will

result

in

total

procurement

that

is

no

greater

in

any

equilibrium

than

the

target

specified

by

α

and

taken

here

to

be

As

we

had

suspected,

the

most

conservative

approach

to

estimating

the

upper

bound

on

slope,

presented

in

the

first

column

of

the

table,

provides

us

little

information

here.

However,

the

other

two

estimation

approaches,

found

in

columns

two

and

three

of

Table

suggest

that

we

are

indeed

quite

confident

that

no

reasonable

setting

of

θ

would

have

done

the

job.

Given

the

tremendous

difficulty

of

the

problem,

this

result

is

very

Still,

we

must

be

very

cautious

in

drawing

too

heroic

a

conclusion

based

on

this

evidence.

Certainly,

we

have

not

checked

all

the

profiles

but

only

a

small

proportion

of

them

(infinitesimal,

if

we

consider

the

entire

continuous

domain

of

θ

and

strategy

sets).

Nor

can

we

expect

ever

to

obtain

enough

evidence

to

make

completely

objective

conclusions.

Instead,

the

approach

we

advocate

here

is

to

collect

as

much

evidence

as

is

feasible

given

resource

constraints,

and

make

the

most

compelling

judgment

based

on

this

evidence,

if

at

all

possible.

CONVERGENCE

RESULTS

At

this

point,

we

explore

abstractly

whether

a

design

parameter

choice

based

on

payoff

data

can

be

asymptotically

reliable.

Since

we

did

not

have

all

the

possible

deviations

for

any

profile

available

in

the

data,

the

true

upper

bounds

may

be

even

lower.

As

a

matter

of

convenience,

we

will

use

notation

un,i(a)

to

refer

to

a

payoff

function

of

player

i

based

on

an

average

over

n

i.i.d.

samples

from

the

distribution

of

payoffs.

We

also

assume

that

un,i(a)

are

independent

for

all

a

A

and

i

I.

We

will

use

the

notation

Γn

to

refer

to

the

game

[I,

R,

{ui,n(·)}],

whereas

Γ

will

denote

the

underlying

game,

[I,

R,

{ui(·)}].

Similarly,

we

define

n(r)

to

be

(r)

with

respect

to

the

game

Γn.

In

this

section,

we

show

that

n(s)

(s)

a.s.

uniformly

on

the

mixed

strategy

space

for

any

finite

game,

and,

furthermore,

that

all

mixed

strategy

Nash

equilibria

in

empirical

games

eventually

become

arbitrarily

close

to

some

Nash

equilibrium

strategies

in

the

underlying

game.

We

use

these

results

to

show

that

under

certain

conditions,

the

optimal

choice

of

the

design

parameter

based

on

empirical

data

converges

almost

surely

to

the

actual

optimum.

THEOREM

Suppose

that

|I|

|A|

Then

n(s)

(s)

a.s.

uniformly

on

S.

Recall

that

N

is

a

set

of

all

Nash

equilibria

of

Γ.

If

we

define

Nn,γ

{s

S

n(s)

γ},

we

have

the

following

corollary

to

Theorem

COROLLARY

For

every

γ

there

is

M

such

that

∀n

M,

N

Nn,γ

a.s.

PROOF.

Since

(s)

for

every

s

N,

we

can

find

M

large

enough

such

that

Pr{supn≥M

sups∈N

n(s)

γ}

By

the

Corollary,

for

any

game

with

a

finite

set

of

pure

strategies

and

for

any

all

Nash

equilibria

lie

in

the

set

of

empirical

-Nash

equilibria

if

enough

samples

have

been

taken.

As

we

now

show,

this

provides

some

justification

for

our

use

of

a

set

of

profiles

with

a

non-zero

-bound

as

an

estimate

of

the

set

of

Nash

equilibria.

First,

suppose

we

conclude

that

for

a

particular

setting

of

θ,

sup{ˆφ∗

(θ)}

α.

Then,

since

for

any

fixed

N(θ)

Nn,

(θ)

when

n

is

large

enough,

sup{φ∗

(θ)}

sup

s∈N

(θ)

φ(s)

sup

s∈Nn,

(θ)

φ(s)

sup{ˆφ∗

(θ)}

α

for

any

such

n.

Thus,

since

we

defined

the

welfare

function

of

the

designer

to

be

I{sup{φ∗

(θ)}

α}

in

our

domain

of

interest,

the

empirical

choice

of

θ

satisfies

the

designer"s

objective,

thereby

maximizing

his

welfare

function.

Alternatively,

suppose

we

conclude

that

inf{ˆφ∗

(θ)}

α

for

every

θ

in

the

domain.

Then,

α

inf{ˆφ∗

(θ)}

inf

s∈Nn,

(θ)

φ(s)

inf

s∈N

(θ)

φ(s)

sup

s∈N

(θ)

φ(s)

sup{φ∗

(θ)},

for

every

θ,

and

we

can

conclude

that

no

setting

of

θ

will

satisfy

the

designer"s

objective.

Now,

we

will

show

that

when

the

number

of

samples

is

large

enough,

every

Nash

equilibrium

of

Γn

is

close

to

some

Nash

equilibrium

of

the

underlying

game.

This

result

will

lead

us

to

consider

convergence

of

optimizers

based

on

empirical

data

to

actual

optimal

mechanism

parameter

settings.

We

first

note

that

the

function

(s)

is

continuous

in

a

finite

game.

LEMMA

Let

S

be

a

mixed

strategy

set

defined

on

a

finite

game.

Then

S

R

is

continuous.

For

the

exposition

that

follows,

we

need

a

bit

of

additional

notation.

First,

let

(Z,

d)

be

a

metric

space,

and

X,

Y

Z

and

define

directed

Hausdorff

distance

from

X

to

Y

to

be

h(X,

Y

sup

x∈X

inf

y∈Y

d(x,

y).

Observe

that

U

X

h(U,

Y

h(X,

Y

Further,

define

BS(x,

δ)

to

be

an

open

ball

in

S

Z

with

center

x

S

and

radius

δ.

Now,

let

Nn

denote

all

Nash

equilibria

of

the

game

Γn

and

let

Nδ

x∈N

BS(x,

δ),

that

is,

the

union

of

open

balls

of

radius

δ

with

centers

at

Nash

equilibria

of

Γ.

Note

that

h(Nδ,

N)

δ.

We

can

then

prove

the

following

general

result.

THEOREM

Suppose

|I|

and

|A|

Then

almost

surely

h(Nn,

N)

converges

to

We

will

now

show

that

in

the

special

case

when

Θ

and

A

are

finite

and

each

Γθ

has

a

unique

Nash

equilibrium,

the

estimates

ˆθ

of

optimal

designer

parameter

converge

to

an

actual

optimizer

almost

surely.

Let

ˆθ

arg

maxθ∈Θ

W

(Nn(θ),

θ),

where

n

is

the

number

of

times

each

pure

profile

was

sampled

in

Γθ

for

every

θ,

and

let

θ∗

arg

maxθ∈Θ

W

(N(θ),

θ).

THEOREM

Suppose

|N(θ)|

for

all

θ

Θ

and

suppose

that

Θ

and

A

are

finite.

Let

W

(s,

θ)

be

continuous

at

the

unique

s∗

(θ)

N(θ)

for

each

θ

Θ.

Then

ˆθ

is

a

consistent

estimator

of

θ∗

if

W

(N(θ),

θ)

is

defined

as

a

supremum,

infimum,

or

expectation

over

the

set

of

Nash

equilibria.

In

fact,

ˆθ

θ∗

a.s.

in

each

of

these

cases.

The

shortcoming

of

the

above

result

is

that,

within

our

framework,

the

designer

has

no

way

of

knowing

or

ensuring

that

Γθ

do,

indeed,

have

unique

equilibria.

However,

it

does

lend

some

theoretical

justification

for

pursuing

design

in

this

manner,

and,

perhaps,

will

serve

as

a

guide

for

more

general

results

in

the

future.

RELATED

WORK

The

mechanism

design

literature

in

Economics

has

typically

explored

existence

of

a

mechanism

that

implements

a

social

choice

function

in

equilibrium

Additionally,

there

is

an

extensive

literature

on

optimal

auction

design

of

which

the

work

by

Roger

Myerson

is,

perhaps,

the

most

relevant.

In

much

of

this

work,

analytical

results

are

presented

with

respect

to

specific

utility

functions

and

accounting

for

constraints

such

as

incentive

compatibility

and

individual

rationality.

Several

related

approaches

to

search

for

the

best

mechanism

exist

in

the

Computer

Science

literature.

Conitzer

and

Sandholm

developed

a

search

algorithm

when

all

the

relevant

game

parameters

are

common

knowledge.

When

payoff

functions

of

players

are

unknown,

a

search

using

simulations

has

been

explored

as

an

alternative.

One

approach

in

that

direction,

taken

in

and

is

to

co-evolve

the

mechanism

parameter

and

agent

strategies,

using

some

notion

of

social

utility

and

agent

payoffs

as

fitness

criteria.

An

alternative

to

co-evolution

explored

in

was

to

optimize

a

well-defined

welfare

function

of

the

designer

using

genetic

programming.

In

this

work

the

authors

used

a

common

learning

strategy

for

all

agents

and

defined

an

outcome

of

a

game

induced

by

a

mechanism

parameter

as

the

outcome

of

joint

agent

learning.

Most

recently,

Phelps

et

al.

compared

two

mechanisms

based

on

expected

social

utility

with

expectation

taken

over

an

empirical

distribution

of

equilibria

in

games

defined

by

heuristic

strategies,

as

in

CONCLUSION

In

this

work

we

spent

considerable

effort

developing

general

tactics

for

empirical

mechanism

design.

We

defined

a

formal

gametheoretic

model

of

interaction

between

the

designer

and

the

participants

of

the

mechanism

as

a

two-stage

game.

We

also

described

in

some

generality

the

methods

for

estimating

a

sample

Nash

equilibrium

function

when

the

data

is

extremely

scarce,

or

a

Nash

equilibrium

correspondence

when

more

data

is

available.

Our

techniques

are

designed

specifically

to

deal

with

problems

in

which

both

the

mechanism

parameter

space

and

the

agent

strategy

sets

are

infinite

and

only

a

relatively

small

data

set

can

be

acquired.

A

difficult

design

issue

in

the

TAC/SCM

game

which

the

TAC

community

has

been

eager

to

address

provides

us

with

a

setting

to

test

our

methods.

In

applying

empirical

game

analysis

to

the

problem

at

hand,

we

are

fully

aware

that

our

results

are

inherently

inexact.

Thus,

we

concentrate

on

collecting

evidence

about

the

structure

of

the

Nash

equilibrium

correspondence.

In

the

end,

we

can

try

to

provide

enough

evidence

to

either

prescribe

a

parameter

setting,

or

suggest

that

no

setting

is

possible

that

will

satisfy

the

designer.

In

the

case

of

TAC/SCM,

our

evidence

suggests

quite

strongly

that

storage

cost

could

not

have

been

effectively

adjusted

in

the

tournament

to

curb

excessive

procurement

without

detrimental

effects

on

overall

profitability.

The

success

of

our

analysis

in

this

extremely

complex

environment

with

high

simulation

costs

makes

us

optimistic

that

our

methods

can

provide

guidance

in

making

mechanism

design

decisions

in

other

challenging

domains.

The

theoretical

results

confirm

some

intuitions

behind

the

empirical

mechanism

design

methods

we

have

introduced,

and

increases

our

confidence

that

our

framework

can

be

effective

in

estimating

the

best

mechanism

parameter

choice

in

relatively

general

settings.

Acknowledgments

We

thank

Terence

Kelly,

Matthew

Rudary,

and

Satinder

Singh

for

helpful

comments

on

earlier

drafts

of

this

work.

This

work

was

supported

in

part

by

NSF

grant

and

the

DARPA

REAL

strategic

reasoning

program.

REFERENCES

R.

Arunachalam

and

N.

M.

Sadeh.

The

supply

chain

trading

agent

competition.

Electronic

Commerce

Research

and

Applications,

M.

Benisch,

A.

Greenwald,

V.

Naroditskiy,

and

M.

Tschantz.

A

stochastic

programming

approach

to

scheduling

in

TAC

SCM.

In

Fifth

ACM

Conference

on

Electronic

Commerce,

pages

New

York,

Y.-P.

Chang

and

W.-T.

Huang.

Generalized

confidence

intervals

for

the

largest

value

of

some

functions

of

parameters

under

normality.

Statistica

Sinica,

D.

Cliff.

Evolution

of

market

mechanism

through

a

continuous

space

of

auction-types.

In

Congress

on

Evolutionary

Computation,

D.

A.

Cohn,

Z.

Ghahramani,

and

M.

I.

Jordan.

Active

learning

with

statistical

models.

Journal

of

Artificial

Intelligence

Research,

V.

Conitzer

and

T.

Sandholm.

An

algorithm

for

automatically

designing

deterministic

mechanisms

without

payments.

In

Third

International

Joint

Conference

on

Autonomous

Agents

and

Multi-Agent

Systems,

pages

D.

Friedman.

Evolutionary

games

in

economics.

Econometrica,

May

R.

Keener.

Statistical

Theory:

A

Medley

of

Core

Topics.

University

of

Michigan

Department

of

Statistics,

C.

Kiekintveld,

Y.

Vorobeychik,

and

M.

P.

Wellman.

An

analysis

of

the

supply

chain

management

trading

agent

competition.

In

Workshop

on

Trading

Agent

Design

and

Analysis,

Edinburgh,

A.

Mas-Colell,

M.

Whinston,

and

J.

Green.

Microeconomic

Theory.

Oxford

University

Press,

R.

B.

Myerson.

Optimal

auction

design.

Mathematics

of

Operations

Research,

February

S.

Olafsson

and

J.

Kim.

Simulation

optimization.

In

E.

Yucesan,

C.-H.

Chen,

J.

Snowdon,

and

J.

Charnes,

editors,

Winter

Simulation

Conference,

D.

Pardoe

and

P.

Stone.

A

supply

chain

management

agent.

SIGecom

Exchanges,

S.

Phelps,

S.

Parsons,

and

P.

McBurney.

Automated

agents

versus

virtual

humans:

an

evolutionary

game

theoretic

comparison

of

two

double-auction

market

designs.

In

Workshop

on

Agent

Mediated

Electronic

Commerce

VI,

S.

Phelps,

S.

Parsons,

P.

McBurney,

and

E.

Sklar.

Co-evolution

of

auction

mechanisms

and

trading

strategies:

towards

a

novel

approach

to

microeconomic

design.

In

ECOMAS

Workshop,

S.

Phelps,

S.

Parsons,

E.

Sklar,

and

P.

McBurney.

Using

genetic

programming

to

optimise

pricing

rules

for

a

double-auction

market.

In

Workshop

on

Agents

for

Electronic

Commerce,

Y.

Vorobeychik,

M.

P.

Wellman,

and

S.

Singh.

Learning

payoff

functions

in

infinite

games.

In

Nineteenth

International

Joint

Conference

on

Artificial

Intelligence,

pages

W.

E.

Walsh,

R.

Das,

G.

Tesauro,

and

J.

O.

Kephart.

Analyzing

complex

strategic

interactions

in

multi-agent

systems.

In

Workshop

on

Game

Theoretic

and

Decision

Theoretic

Agents,

M.

P.

Wellman,

J.

J.

Estelle,

S.

Singh,

Y.

Vorobeychik,

C.

Kiekintveld,

and

V.

Soni.

Strategic

interactions

in

a

supply

chain

game.

Computational

Intelligence,

February

APPENDIX

A.

PROOFS

Proof

of

Proposition

Pr

max

i∈I

max

b∈Ai\ai

ui(b,

a−i)

ui(a)

Y

i∈I

Eui(a)

Pr(

max

b∈Ai\ai

ui(b,

a−i)

ui(a)

|ui(a))

Y

i∈I

Z

R

Y

b∈Ai\ai

Pr(ui(b,

a−i)

u

)fui(a)(u)du.

Proof

of

Proposition

First,

let

us

suppose

that

some

function,

f(x)

defined

on

[ai,

bi],

satisfy

Lipschitz

condition

on

(ai,

bi]

with

Lipschitz

constant

Ai.

Then

the

following

claim

holds:

Claim:

infx∈(ai,bi]

f(x)

f(bi)

Ai(bi

ai).

To

prove

this

claim,

note

that

the

intersection

of

lines

at

f(ai)

and

f(bi)

with

slopes

−Ai

and

Ai

respectively

will

determine

the

lower

bound

on

the

minimum

of

f(x)

on

[ai,

bi]

(which

is

a

lower

bound

on

infimum

of

f(x)

on

(ai,

bj

The

line

at

f(ai)

is

determined

by

f(ai)

−Aiai

cL

and

the

line

at

f(bi)

is

determined

by

f(bi)

Aibi

+cR.

Thus,

the

intercepts

are

cL

f(ai)+Aiai

and

cR

f(bi)

Aibi

respectively.

Let

x∗

be

the

point

at

which

these

lines

intersect.

Then,

x∗

f(x∗

cR

A

f(x∗

cL

A

By

substituting

the

expressions

for

cR

and

cL,

we

get

the

desired

result.

Now,

subadditivity

gives

us

Pr{

θ∈Θ

sup{φ∗

(θ)}

α}

Pr{

θ∈Θj

sup{φ∗

(θ)}

α},

and,

by

the

claim,

Pr{

θ∈Θj

sup{φ∗

(θ)}

α}

Pr{

inf

θ∈Θj

sup{φ∗

(θ)}

α}

Pr{sup{φ∗

(aj)}

sup{φ∗

(bj)}

2α

Aj(bj

aj

Since

we

have

a

finite

number

of

points

in

the

data

set

for

each

θ,

we

can

obtain

the

following

expression:

Pr{sup{φ∗

(aj)}

sup{φ∗

(bj)}

cj

=D

X

y,z∈D:y+z≤cj

Pr{sup{φ∗

(bj

y}

Pr{sup{φ∗

(aj)}

z}.

We

can

now

restrict

attention

to

deriving

an

upper

bound

on

Pr{sup{φ∗

(θ)}

y}

for

a

fixed

θ.

To

do

this,

observe

that

Pr{sup{φ∗

(θ)}

y}

≤D

Pr{

a∈D:φ(a)=y

(a)

X

a∈D:φ(a)=y

Pr{

(a)

by

subadditivity

and

the

fact

that

a

profile

a

is

a

Nash

equilibrium

if

and

only

if

(a)

Putting

everything

together

yields

the

desired

result.

Proof

of

Theorem

First,

we

will

need

the

following

fact:

Claim:

Given

a

function

fi(x)

and

a

set

X,

maxx∈X

maxx∈X

maxx∈X

To

prove

this

claim,

observe

that

max

x∈X

max

x∈X

maxx

maxx

if

maxx

maxx

maxx

maxx

if

maxx

maxx

In

the

first

case,

max

x∈X

max

x∈X

max

x∈X

max

x∈X

Similarly,

in

the

second

case,

max

x∈X

max

x∈X

max

x∈X

max

x∈X

max

x∈X

Thus,

the

claim

holds.

By

the

Strong

Law

of

Large

Numbers,

un,i(a)

ui(a)

a.s.

for

all

i

I,

a

A.

That

is,

Pr{

lim

n→∞

un,i(a)

ui(a)}

or,

equivalently

for

any

α

and

δ

there

is

M(i,

a)

such

that

Pr{

sup

n≥M(i,a)

|un,i(a)

ui(a)|

δ

α.

By

taking

M

maxi∈I

maxa∈A

M(i,

a),

we

have

Pr{max

i∈I

max

a∈A

sup

n≥M

|un,i(a)

ui(a)|

δ

α.

Thus,

by

the

claim,

for

any

n

M,

sup

n≥M

n(s)

(s)|

max

i∈I

max

ai∈Ai

sup

n≥M

|un,i(ai,

s−i)

ui(ai,

s−i)|+

sup

n≥M

max

i∈I

|un,i(s)

ui(s)|

max

i∈I

max

ai∈Ai

X

b∈A−i

sup

n≥M

|un,i(ai,

b)

ui(ai,

b)|s−i(b)+

max

i∈I

X

b∈A

sup

n≥M

|un,i(b)

ui(b)|s(b)

max

i∈I

max

ai∈Ai

X

b∈A−i

sup

n≥M

|un,i(ai,

b)

ui(ai,

b)|+

max

i∈I

X

b∈A

sup

n≥M

|un,i(b)

ui(b)|

max

i∈I

max

ai∈Ai

X

b∈A−i

δ

max

i∈I

X

b∈A

δ

δ

with

probability

at

least

α.

Note

that

since

s−i(a)

and

s(a)

are

bounded

between

and

we

were

able

to

drop

them

from

the

expressions

above

to

obtain

a

bound

that

will

be

valid

independent

of

the

particular

choice

of

s.

Furthermore,

since

the

above

result

can

be

obtained

for

an

arbitrary

α

and

δ

we

have

Pr{limn→∞

n(s)

(s)}

uniformly

on

S.

Proof

of

Lemma

We

prove

the

result

using

uniform

continuity

of

ui(s)

and

preservation

of

continuity

under

maximum.

Claim:

A

function

f

Rk

R

defined

by

f(t)

Pk

ziti,

where

zi

are

constants

in

R,

is

uniformly

continuous

in

t.

The

claim

follows

because

|f(t)−f(t

Pk

zi(ti−ti)|

Pk

|zi||ti

ti|.

An

immediate

result

of

this

for

our

purposes

is

that

ui(s)

is

uniformly

continuous

in

s

and

ui(ai,

s−i)

is

uniformly

continuous

in

s−i.

Claim:

Let

f(a,

b)

be

uniformly

continuous

in

b

B

for

every

a

A,

with

|A|

Then

V

(b)

maxa∈A

f(a,

b)

is

uniformly

continuous

in

b.

To

show

this,

take

γ

and

let

b,

b

B

such

that

b

b

δ(a)

|f(a,

b)

f(a,

b

γ.

Now

take

δ

mina∈A

δ(a).

Then,

whenever

b

b

δ,

|V

(b)

V

(b

max

a∈A

f(a,

b)

max

a∈A

f(a,

b

max

a∈A

|f(a,

b)

f(a,

b

γ.

Now,

recall

that

(s)

maxi[maxai∈Ai

ui(ai,

s−i)

ui(s)].

By

the

claims

above,

maxai∈Ai

ui(ai,

s−i)

is

uniformly

continuous

in

s−i

and

ui(s)

is

uniformly

continuous

in

s.

Since

the

difference

of

two

uniformly

continuous

functions

is

uniformly

continuous,

and

since

this

continuity

is

preserved

under

maximum

by

our

second

claim,

we

have

the

desired

result.

Proof

of

Theorem

Choose

δ

First,

we

need

to

ascertain

that

the

following

claim

holds:

Claim:

mins∈S\Nδ

(s)

exists

and

Since

Nδ

is

an

open

subset

of

compact

S,

it

follows

that

S

Nδ

is

compact.

As

we

had

also

proved

in

Lemma

that

(s)

is

continuous,

existence

follows

from

the

Weierstrass

theorem.

That

is

clear

since

(s)

if

and

only

if

s

is

a

Nash

equilibrium

of

Γ.

Now,

by

Theorem

for

any

α

there

is

M

such

that

Pr{

sup

n≥M

sup

s∈S

n(s)

(s)|

α.

Consequently,

for

any

δ

Pr{

sup

n≥M

h(Nn,

Nδ)

δ}

Pr{∀n

M

Nn

Nδ}

Pr{

sup

n≥M

sup

s∈N

(s)

Pr{

sup

n≥M

sup

s∈S

n(s)

(s)|

α.

Since

this

holds

for

an

arbitrary

α

and

δ

the

desired

result

follows.

Proof

of

Theorem

Fix

θ

and

choose

δ

Since

W

(s,

θ)

is

continuous

at

s∗

(θ),

given

there

is

δ

such

that

for

every

s

that

is

within

δ

of

s∗

(θ),

|W

(s

θ)

W

(s∗

(θ),

θ)|

By

Theorem

we

can

find

M(θ)

large

enough

such

that

all

s

Nn

are

within

δ

of

s∗

(θ)

for

all

n

M(θ)

with

probability

Consequently,

for

any

we

can

find

M(θ)

large

enough

such

that

with

probability

we

have

supn≥M(θ)

sups

∈Nn

|W

(s

θ)

W

(s∗

(θ),

θ)|

Let

us

assume

without

loss

of

generality

that

there

is

a

unique

optimal

choice

of

θ.

Now,

since

the

set

Θ

is

finite,

there

is

also

the

second-best

choice

of

θ

(if

there

is

only

one

θ

Θ

this

discussion

is

moot

anyway):

θ∗∗

arg

max

Θ\θ∗

W

(s∗

(θ),

θ).

Suppose

w.l.o.g.

that

θ∗∗

is

also

unique

and

let

W

(s∗

(θ∗

θ∗

W

(s∗

(θ∗∗

θ∗∗

Then

if

we

let

and

let

M

maxθ∈Θ

M(θ),

where

each

M(θ)

is

large

enough

such

that

supn≥M(θ)

sups

∈Nn

|W

(s

θ)−

W

(s∗

(θ),

θ)|

a.s.,

the

optimal

choice

of

θ

based

on

any

empirical

equilibrium

will

be

θ∗

with

probability

Thus,

in

particular,

given

any

probability

distribution

over

empirical

equilibria,

the

best

choice

of

θ

will

be

θ∗

with

probability

(similarly,

if

we

take

supremum

or

infimum

of

W

(Nn(θ),

θ)

over

the

set

of

empirical

equilibria

in

constructing

the

objective

function).

outcome feature of interest
interest outcome feature
supply-chain trading
two-stage game
empirical mechanism design
game theory

empirical mechanism
parameter setting
observed behavior
player
participant
gametheoretic model
analysis
nash equilibrium