Fairness

in

Dead-Reckoning

based

Distributed

Multi-Player

Games

Sudhir

Aggarwal

Hemant

Banavar

Department

of

Computer

Science

Florida

State

University,

Tallahassee,

FL

Email:

{sudhir,

banavar}@cs.fsu.edu

Sarit

Mukherjee

Sampath

Rangarajan

Center

for

Networking

Research

Bell

Laboratories,

Holmdel,

NJ

Email:

{sarit,

sampath}@bell-labs.com

ABSTRACT

In

a

distributed

multi-player

game

that

uses

dead-reckoning

vectors

to

exchange

movement

information

among

players,

there

is

inaccuracy

in

rendering

the

objects

at

the

receiver

due

to

network

delay

between

the

sender

and

the

receiver.

The

object

is

placed

at

the

receiver

at

the

position

indicated

by

the

dead-reckoning

vector,

but

by

that

time,

the

real

position

could

have

changed

considerably

at

the

sender.

This

inaccuracy

would

be

tolerable

if

it

is

consistent

among

all

players;

that

is,

at

the

same

physical

time,

all

players

see

inaccurate

(with

respect

to

the

real

position

of

the

object)

but

the

same

position

and

trajectory

for

an

object.

But

due

to

varying

network

delays

between

the

sender

and

different

receivers,

the

inaccuracy

is

different

at

different

players

as

well.

This

leads

to

unfairness

in

game

playing.

In

this

paper,

we

first

introduce

an

error

measure

for

estimating

this

inaccuracy.

Then

we

develop

an

algorithm

for

scheduling

the

sending

of

dead-reckoning

vectors

at

a

sender

that

strives

to

make

this

error

equal

at

different

receivers

over

time.

This

algorithm

makes

the

game

very

fair

at

the

expense

of

increasing

the

overall

mean

error

of

all

players.

To

mitigate

this

effect,

we

propose

a

budget

based

algorithm

that

provides

improved

fairness

without

increasing

the

mean

error

thereby

maintaining

the

accuracy

of

game

playing.

We

have

implemented

both

the

scheduling

algorithm

and

the

budget

based

algorithm

as

part

of

BZFlag,

a

popular

distributed

multi-player

game.

We

show

through

experiments

that

these

algorithms

provide

fairness

among

players

in

spite

of

widely

varying

network

delays.

An

additional

property

of

the

proposed

algorithms

is

that

they

require

less

number

of

DRs

to

be

exchanged

(compared

to

the

current

implementation

of

BZflag)

to

achieve

the

same

level

of

accuracy

in

game

playing.

Categories

and

Subject

Descriptors

[Computer-Communication

Networks]:

Distributed

Systems-Distributed

applications

General

Terms

Algorithms,

Design,

Experimentation,

Performance

INTRODUCTION

In

a

distributed

multi-player

game,

players

are

normally

distributed

across

the

Internet

and

have

varying

delays

to

each

other

or

to

a

central

game

server.

Usually,

in

such

games,

the

players

are

part

of

the

game

and

in

addition

they

may

control

entities

that

make

up

the

game.

During

the

course

of

the

game,

the

players

and

the

entities

move

within

the

game

space.

A

player

sends

information

about

her

movement

as

well

as

the

movement

of

the

entities

she

controls

to

the

other

players

using

a

Dead-Reckoning

(DR)

vector.

A

DR

vector

contains

information

about

the

current

position

of

the

player/entity

in

terms

of

x,

y

and

z

coordinates

(at

the

time

the

DR

vector

was

sent)

as

well

as

the

trajectory

of

the

entity

in

terms

of

the

velocity

component

in

each

of

the

dimensions.

Each

of

the

participating

players

receives

such

DR

vectors

from

one

another

and

renders

the

other

players/entities

on

the

local

consoles

until

a

new

DR

vector

is

received

for

that

player/entity.

In

a

peer-to-peer

game,

players

send

DR

vectors

directly

to

each

other;

in

a

client-server

game,

these

DR

vectors

may

be

forwarded

through

a

game

server.

The

idea

of

DR

is

used

because

it

is

almost

impossible

for

players/entities

to

exchange

their

current

positions

at

every

time

unit.

DR

vectors

are

quantization

of

the

real

trajectory

(which

we

refer

to

as

real

path)

at

a

player.

Normally,

a

new

DR

vector

is

computed

and

sent

whenever

the

real

path

deviates

from

the

path

extrapolated

using

the

previous

DR

vector

(say,

in

terms

of

distance

in

the

x,

y,

z

plane)

by

some

amount

specified

by

a

threshold.

We

refer

to

the

trajectory

that

can

be

computed

using

the

sequence

of

DR

vectors

as

the

exported

path.

Therefore,

at

the

sending

player,

there

is

a

deviation

between

the

real

path

and

the

exported

path.

The

error

due

to

this

deviation

can

be

removed

if

each

movement

of

player/entity

is

communicated

to

the

other

players

at

every

time

unit;

that

is

a

DR

vector

is

generated

at

every

time

unit

thereby

making

the

real

and

exported

paths

the

same.

Given

that

it

is

not

feasible

to

satisfy

this

due

to

bandwidth

limitations,

this

error

is

not

of

practical

interest.

Therefore,

the

receiving

players

can,

at

best,

follow

the

exported

path.

Because

of

the

network

delay

between

the

sending

and

receiving

players,

when

a

DR

vector

is

received

and

rendered

at

a

player,

the

original

trajectory

of

the

player/entity

may

have

already

changed.

Thus,

in

physical

time,

there

is

a

deviation

at

the

receiving

player

between

the

exported

path

and

the

rendered

trajectory

(which

we

refer

to

as

placed

path).

We

refer

to

this

error

as

the

export

error.

Note

that

the

export

error,

in

turn,

results

in

a

deviation

between

the

real

and

the

placed

paths.

The

export

error

manifests

itself

due

to

the

deviation

between

the

exported

path

at

the

sender

and

the

placed

path

at

the

receiver

(i)

before

the

DR

vector

is

received

at

the

receiver

(referred

to

as

the

before

export

error,

and

(ii)

after

the

DR

vector

is

received

at

the

receiver

(referred

to

as

the

after

export

error).

In

an

earlier

paper

we

showed

that

by

synchronizing

the

clocks

at

all

the

players

and

by

using

a

technique

based

on

time-stamping

messages

that

carry

the

DR

vectors,

we

can

guarantee

that

the

after

export

error

is

made

zero.

That

is,

the

placed

and

the

exported

paths

match

after

the

DR

vector

is

received.

We

also

showed

that

the

before

export

error

can

never

be

eliminated

since

there

is

always

a

non-zero

network

delay,

but

can

be

significantly

reduced

using

our

technique

Henceforth

we

assume

that

the

players

use

such

a

technique

which

results

in

unavoidable

but

small

overall

export

error.

In

this

paper

we

consider

the

problem

of

different

and

varying

network

delays

between

each

sender-receiver

pair

of

a

DR

vector,

and

consequently,

the

different

and

varying

export

errors

at

the

receivers.

Due

to

the

difference

in

the

export

errors

among

the

receivers,

the

same

entity

is

rendered

at

different

physical

time

at

different

receivers.

This

brings

in

unfairness

in

game

playing.

For

instance

a

player

with

a

large

delay

would

always

see

an

entity

late

in

physical

time

compared

to

the

other

players

and,

therefore,

her

action

on

the

entity

would

be

delayed

(in

physical

time)

even

if

she

reacted

instantaneously

after

the

entity

was

rendered.

Our

goal

in

this

paper

is

to

improve

the

fairness

of

these

games

in

spite

of

the

varying

network

delays

by

equalizing

the

export

error

at

the

players.

We

explore

whether

the

time-average

of

the

export

errors

(which

is

the

cumulative

export

error

over

a

period

of

time

averaged

over

the

time

period)

at

all

the

players

can

be

made

the

same

by

scheduling

the

sending

of

the

DR

vectors

appropriately

at

the

sender.

We

propose

two

algorithms

to

achieve

this.

Both

the

algorithms

are

based

on

delaying

(or

dropping)

the

sending

of

DR

vectors

to

some

players

on

a

continuous

basis

to

try

and

make

the

export

error

the

same

at

all

the

players.

At

an

abstract

level,

the

algorithm

delays

sending

DR

vectors

to

players

whose

accumulated

error

so

far

in

the

game

is

smaller

than

others;

this

would

mean

that

the

export

error

due

to

this

DR

vector

at

these

players

will

be

larger

than

that

of

the

other

players,

thereby

making

them

the

same.

The

goal

is

to

make

this

error

at

least

approximately

equal

at

every

DR

vector

with

the

deviation

in

the

error

becoming

smaller

as

time

progresses.

The

first

algorithm

(which

we

refer

to

as

the

scheduling

algorithm)

is

based

on

estimating

the

delay

between

players

and

refining

the

sending

of

DR

vectors

by

scheduling

them

to

be

sent

to

different

players

at

different

times

at

every

DR

generation

point.

Through

an

implementation

of

this

algorithm

using

the

open

source

game

BZflag,

we

show

that

this

algorithm

makes

the

game

very

fair

(we

measure

fairness

in

terms

of

the

standard

deviation

of

the

error).

The

drawback

of

this

algorithm

is

that

it

tends

to

push

the

error

of

all

the

players

towards

that

of

the

player

with

the

worst

error

(which

is

the

error

at

the

farthest

player,

in

terms

of

delay,

from

the

sender

of

the

DR).

To

alleviate

this

effect,

we

propose

a

budget

based

algorithm

which

budgets

how

the

DRs

are

sent

to

different

players.

At

a

high

level,

the

algorithm

is

based

on

the

idea

of

sending

more

DRs

to

players

who

are

farther

away

from

the

sender

compared

to

those

who

are

closer.

Experimental

results

from

BZflag

illustrates

that

the

budget

based

algorithm

follows

a

more

balanced

approach.

It

improves

the

fairness

of

the

game

but

at

the

same

time

does

so

without

pushing

up

the

mean

error

of

the

players

thereby

maintaining

the

accuracy

of

the

game.

In

addition,

the

budget

based

algorithm

is

shown

to

achieve

the

same

level

of

accuracy

of

game

playing

as

the

current

implementation

of

BZflag

using

much

less

number

of

DR

vectors.

PREVIOUS

WORK

Earlier

work

on

network

games

to

deal

with

network

latency

has

mostly

focussed

on

compensation

techniques

for

packet

delay

and

loss

These

methods

are

aimed

at

making

large

delays

and

message

loss

tolerable

for

players

but

does

not

consider

the

problems

that

may

be

introduced

by

varying

delays

from

the

server

to

different

players

or

from

the

players

to

one

another.

For

example,

the

concept

of

local

lag

has

been

used

in

where

each

player

delays

every

local

operation

for

a

certain

amount

of

time

so

that

remote

players

can

receive

information

about

the

local

operation

and

execute

the

same

operation

at

the

about

same

time,

thus

reducing

state

inconsistencies.

The

online

multi-player

game

MiMaze

for

example,

takes

a

static

bucket

synchronization

approach

to

compensate

for

variable

network

delays.

In

MiMaze,

each

player

delays

all

events

by

ms

regardless

of

whether

they

are

generated

locally

or

remotely.

Players

with

a

network

delay

larger

than

ms

simply

cannot

participate

in

the

game.

In

general,

techniques

based

on

bucket

synchronization

depend

on

imposing

a

worst

case

delay

on

all

the

players.

There

have

been

a

few

papers

which

have

studied

the

problem

of

fairness

in

a

distributed

game

by

more

sophisticated

message

delivery

mechanisms.

But

these

works

assume

the

existence

of

a

global

view

of

the

game

where

a

game

server

maintains

a

view

(or

state)

of

the

game.

Players

can

introduce

objects

into

the

game

or

delete

objects

that

are

already

part

of

the

game

(for

example,

in

a

first-person

shooter

game,

by

shooting

down

the

object).

These

additions

and

deletions

are

communicated

to

the

game

server

using

action

messages.

Based

on

these

action

messages,

the

state

of

the

game

is

changed

at

the

game

server

and

these

changes

are

communicated

to

the

players

using

update

messages.

Fairness

is

achieved

by

ordering

the

delivery

of

action

and

update

messages

at

the

game

server

and

players

respectively

based

on

the

notion

of

a

fair-order

which

takes

into

account

the

delays

between

the

game

server

and

the

different

players.

Objects

that

are

part

of

the

game

may

move

but

how

this

information

is

communicated

to

the

players

seems

to

be

beyond

the

scope

of

these

works.

In

this

sense,

these

works

are

very

limited

in

scope

and

may

be

applicable

only

to

firstperson

shooter

games

and

that

too

to

only

games

where

players

are

not

part

of

the

game.

DR

vectors

can

be

exchanged

directly

among

the

players

(peerto-peer

model)

or

using

a

central

server

as

a

relay

(client-server

model).

It

has

been

shown

in

that

multi-player

games

that

use

DR

vectors

together

with

bucket

synchronization

are

not

cheatproof

unless

additional

mechanisms

are

put

in

place.

Both

the

scheduling

algorithm

and

the

budget-based

algorithm

described

in

our

paper

use

DR

vectors

and

hence

are

not

cheat-proof.

For

example,

a

receiver

could

skew

the

delay

estimate

at

the

sender

to

make

the

sender

believe

that

the

delay

between

the

sender

and

the

receiver

is

high

thereby

gaining

undue

advantage.

We

emphasize

that

the

focus

of

this

paper

is

on

fairness

without

addressing

the

issue

of

cheating.

In

the

next

section,

we

describe

the

game

model

that

we

use

and

illustrate

how

senders

and

receivers

exchange

DR

vectors

and

how

entities

are

rendered

at

the

receivers

based

on

the

time-stamp

augmented

DR

vector

exchange

as

described

in

In

Section

we

describe

the

DR

vector

scheduling

algorithm

that

aims

to

make

the

export

error

equal

across

the

players

with

varying

delays

from

the

sender

of

a

DR

vector,

followed

by

experimental

results

obtained

from

instrumentation

of

the

scheduling

algorithm

on

the

open

source

game

BZFlag.

Section

describes

the

budget

based

algorithm

that

achieves

improved

fairness

but

without

reducing

the

level

accuracy

of

game

playing.

Conclusions

are

presented

in

Section

GAME

MODEL

The

game

architecture

is

based

on

players

distributed

across

the

Internet

and

exchanging

DR

vectors

to

each

other.

The

DR

vectors

could

either

be

sent

directly

from

one

player

to

another

(peerto-peer

model)

or

could

be

sent

through

a

game

server

which

receives

the

DR

vector

from

a

player

and

forwards

it

to

other

players

(client-server

model).

As

mentioned

before,

we

assume

synchronized

clocks

among

the

participating

players.

Each

DR

vector

sent

from

one

player

to

another

specifies

the

trajectory

of

exactly

one

player/entity.

We

assume

a

linear

DR

vector

in

that

the

information

contained

in

the

DR

vector

is

only

enough

at

the

receiving

player

to

compute

the

trajectory

and

render

the

entity

in

a

straight

line

path.

Such

a

DR

vector

contains

information

about

the

starting

position

and

velocity

of

the

player/entity

where

the

velocity

is

Thus,

the

DR

vectors

sent

by

a

player

specifies

the

current

time

at

the

player

when

the

DR

vector

is

computed

(not

the

time

at

which

this

DR

vector

is

sent

to

the

other

players

as

we

will

explain

later),

the

current

position

of

the

player/entity

in

terms

of

the

x,

y,

z

coordinates

and

the

velocity

vector

in

the

direction

of

x,

y

and

z

coordinates.

Specifically,

the

ith

DR

vector

sent

by

player

j

about

the

kth

entity

is

denoted

by

DRj

ik

and

is

represented

by

the

following

tuple

(Tj

ik,

xj

ik,

yj

ik,

zj

ik,

vxj

ik,

vyj

ik,

vzj

ik).

Without

loss

of

generality,

in

the

rest

of

the

discussion,

we

consider

a

sequence

of

DR

vectors

sent

by

only

one

player

and

for

only

one

entity.

For

simplicity,

we

consider

a

two

dimensional

game

space

rather

than

a

three

dimensional

one.

Hence

we

use

DRi

to

denote

the

ith

such

DR

vector

represented

as

the

tuple

(Ti,

xi,

yi,

vxi,

vyi).

The

receiving

player

computes

the

starting

position

for

the

entity

based

on

xi,

yi

and

the

time

difference

between

when

the

DR

vector

is

received

and

the

time

Ti

at

which

it

was

computed.

Note

that

the

computation

of

time

difference

is

feasible

since

all

the

clocks

are

synchronized.

The

receiving

player

then

uses

the

velocity

components

to

project

and

render

the

trajectory

of

the

entity.

This

trajectory

is

followed

until

a

new

DR

vector

is

received

which

changes

the

position

and/or

velocity

of

the

entity.

Real

Exported

Placed

A

B

C

D

computed

at

time

and

sent

to

the

receiver

computed

at

time

and

sent

to

the

receiver

Placed

E

Figure

Trajectories

and

deviations.

Based

on

this

model,

Figure

illustrates

the

sending

and

Other

type

of

DR

vectors

include

quadratic

DR

vectors

which

specify

the

acceleration

of

the

entity

and

cubic

spline

DR

vectors

that

consider

the

starting

position

and

velocity

and

the

ending

position

and

velocity

of

the

entity.

ing

of

DR

vectors

and

the

different

errors

that

are

encountered.

The

figure

shows

the

reception

of

DR

vectors

at

a

player

(henceforth

called

the

receiver).

The

horizontal

axis

shows

the

time

which

is

synchronized

among

all

the

players.

The

vertical

axis

tries

to

conceptually

capture

the

two-dimensional

position

of

an

entity.

Assume

that

at

time

a

DR

vector

is

computed

by

the

sender

and

immediately

sent

to

the

receiver.

Assume

that

is

received

at

the

receiver

after

a

delay

of

time

units.

The

receiver

computes

the

initial

position

of

the

entity

as

(shown

as

point

E).

The

thick

line

EBD

represents

the

projected

and

rendered

trajectory

at

the

receiver

based

on

the

velocity

components

and

(placed

path).

At

time

a

DR

vector

is

computed

for

the

same

entity

and

immediately

sent

to

the

Assume

that

is

received

at

the

receiver

after

a

delay

of

time

units.

When

this

DR

vector

is

received,

assume

that

the

entity

is

at

point

D.

A

new

position

for

the

entity

is

computed

as

and

the

entity

is

moved

to

this

position

(point

C).

The

velocity

components

and

are

used

to

project

and

render

this

entity

further.

Let

us

now

consider

the

error

due

to

network

delay.

Although

was

computed

at

time

and

sent

to

the

receiver,

it

did

not

reach

the

receiver

until

time

This

means,

although

the

exported

path

based

on

at

the

sender

at

time

is

the

trajectory

AC,

until

time

at

the

receiver,

this

entity

was

being

rendered

at

trajectory

BD

based

on

Only

at

time

did

the

entity

get

moved

to

point

C

from

which

point

onwards

the

exported

and

the

placed

paths

are

the

same.

The

deviation

between

the

exported

and

placed

paths

creates

an

error

component

which

we

refer

to

as

the

export

error.

A

way

to

represent

the

export

error

is

to

compute

the

integral

of

the

distance

between

the

two

trajectories

over

the

time

when

they

are

out

of

sync.

We

represent

the

integral

of

the

distances

between

the

placed

and

exported

paths

due

to

some

DR

DRi

over

a

time

interval

as

Err(DRi,

In

the

figure,

the

export

error

due

to

is

computed

as

the

integral

of

the

distance

between

the

trajectories

AC

and

BD

over

the

time

interval

Note

that

there

could

be

other

ways

of

representing

this

error

as

well,

but

in

this

paper,

we

use

the

integral

of

the

distance

between

the

two

trajectories

as

a

measure

of

the

export

error.

Note

that

there

would

have

been

an

export

error

created

due

to

the

reception

of

at

which

time

the

placed

path

would

have

been

based

on

a

previous

DR

vector.

This

is

not

shown

in

the

figure

but

it

serves

to

remind

the

reader

that

the

export

error

is

cumulative

when

a

sequence

of

DR

vectors

are

received.

Starting

from

time

onwards,

there

is

a

deviation

between

the

real

and

the

exported

paths.

As

we

discussed

earlier,

this

export

error

is

unavoidable.

The

above

figure

and

example

illustrates

one

receiver

only.

But

in

reality,

DR

vectors

and

are

sent

by

the

sender

to

all

the

participating

players.

Each

of

these

players

receives

and

after

varying

delays

thereby

creating

different

export

error

values

at

different

players.

The

goal

of

the

DR

vector

scheduling

algorithm

to

be

described

in

the

next

section

is

to

make

this

(cumulative)

export

error

equal

at

every

player

independently

for

each

of

the

entities

that

make

up

the

game.

SCHEDULING

ALGORITHM

FORSENDING

DR

VECTORS

In

Section

we

showed

how

delay

from

the

sender

of

a

new

DR

Normally,

DR

vectors

are

not

computed

on

a

periodic

basis

but

on

an

on-demand

basis

where

the

decision

to

compute

a

new

DR

vector

is

based

on

some

threshold

being

exceeded

on

the

deviation

between

the

real

path

and

the

path

exported

by

the

previous

DR

vector.

vector

to

the

receiver

of

the

DR

vector

could

lead

to

export

error

because

of

the

deviation

of

the

placed

path

from

the

exported

path

at

the

receiver

until

this

new

DR

vector

is

received.

We

also

mentioned

that

the

goal

of

the

DR

vector

scheduling

algorithm

is

to

make

the

export

error

equal

at

all

receivers

over

a

period

of

time.

Since

the

game

is

played

in

a

distributed

environment,

it

makes

sense

for

the

sender

of

an

entity

to

keep

track

of

all

the

errors

at

the

receivers

and

try

to

make

them

equal.

However,

the

sender

cannot

know

the

actual

error

at

a

receiver

till

it

gets

some

information

regarding

the

error

back

from

the

receiver.

Our

algorithm

estimates

the

error

to

compute

a

schedule

to

send

DR

vectors

to

the

receivers

and

corrects

the

error

when

it

gets

feedbacks

from

the

receivers.

In

this

section

we

provide

motivations

for

the

algorithm

and

describe

the

steps

it

goes

through.

Throughout

this

section,

we

will

use

the

following

example

to

illustrate

the

algorithm.

Exported

path

Placed

path

at

receiver

A

B

C

D

E

F

sent

to

receiver

sent

to

receiver

G

H

I

J

K

L

N

M

estimated

to

be

received

by

receiver

estimated

to

be

received

by

receiver

actually

received

by

receiver

actually

received

by

receiver

sent

to

both

receivers

computed

by

sender

Placed

path

at

receiver

Figure

DR

vector

flow

between

a

sender

and

two

receivers

and

the

evolution

of

estimated

and

actual

placed

paths

at

the

receivers.

sent

at

time

to

both

receivers.

sent

at

time

to

receiver

and

sent

at

time

δ2

to

receiver

Consider

the

example

in

Figure

The

figure

shows

a

single

sender

sending

DR

vectors

for

an

entity

to

two

different

receivers

and

computed

at

is

sent

and

received

by

the

receivers

sometime

between

and

at

which

time

they

move

the

location

of

the

entity

to

match

the

exported

path.

Thus,

the

path

of

the

entity

is

shown

only

from

the

point

where

the

placed

path

matches

the

exported

path

for

Now

consider

At

time

is

computed

by

the

sender

but

assume

that

it

is

not

immediately

sent

to

the

receivers

and

is

only

sent

after

time

δ1

to

receiver

(at

time

and

after

time

δ2

to

receiver

(at

time

Note

that

the

sender

includes

the

sending

timestamp

with

the

DR

vector

as

shown

in

the

figure.

Assume

that

the

sender

estimates

(it

will

be

clear

shortly

why

the

sender

has

to

estimate

the

delay)

that

after

a

delay

of

receiver

will

receive

it,

will

use

the

coordinate

and

velocity

parameters

to

compute

the

entity"s

current

location

and

move

it

there

(point

C)

and

from

this

time

onwards,

the

exported

and

the

placed

paths

will

become

the

same.

However,

in

reality,

receiver

receives

after

a

delay

of

(which

is

less

than

sender"s

estimates

of

and

moves

the

corresponding

entity

to

point

H.

Similarly,

the

sender

estimates

that

after

a

delay

of

receiver

will

receive

will

compute

the

current

location

of

the

entity

and

move

it

to

that

point

(point

E),

while

in

reality

it

receives

after

a

delay

of

and

moves

the

entity

to

point

N.

The

other

points

shown

on

the

placed

and

exported

paths

will

be

used

later

in

the

discussion

to

describe

different

error

components.

Computation

of

Relative

Export

Error

Referring

back

to

the

discussion

from

Section

from

the

sender"s

perspective,

the

export

error

at

receiver

due

to

is

given

by

δ1

(the

integral

of

the

distance

between

the

trajectories

AC

and

DB

over

the

time

interval

δ1

of

Figure

This

is

due

to

the

fact

that

the

sender

uses

the

estimated

delay

to

compute

this

error.

Similarly,

the

export

error

from

the

sender"s

perspective

at

received

due

to

is

given

by

δ2

(the

integral

of

the

distance

between

the

trajectories

AE

and

DF

over

the

time

interval

δ2

Note

that

the

above

errors

from

the

sender"s

perspective

are

only

estimates.

In

reality,

the

export

error

will

be

either

smaller

or

larger

than

the

estimated

value,

based

on

whether

the

delay

estimate

was

larger

or

smaller

than

the

actual

delay

that

experienced.

This

difference

between

the

estimated

and

the

actual

export

error

is

the

relative

export

error

(which

could

either

be

positive

or

negative)

which

occurs

for

every

DR

vector

that

is

sent

and

is

accumulated

at

the

sender.

The

concept

of

relative

export

error

is

illustrated

in

Figure

Since

the

actual

delay

to

receiver

is

the

export

error

induced

by

at

receiver

is

δ1

This

means,

there

is

an

error

in

the

estimated

export

error

and

the

sender

can

compute

this

error

only

after

it

gets

a

feedback

from

the

receiver

about

the

actual

delay

for

the

delivery

of

i.e.,

the

value

of

We

propose

that

once

receiver

receives

it

sends

the

value

of

back

to

the

sender.

The

receiver

can

compute

this

information

as

it

knows

the

time

at

which

was

sent

which

is

appended

to

the

DR

vector

as

shown

in

Figure

and

the

local

receiving

time

(which

is

synchronized

with

the

sender"s

clock).

Therefore,

the

sender

computes

the

relative

export

error

for

receiver

represented

using

as

δ1

δ1

δ1

δ1

Similarly

the

relative

export

error

for

receiver

is

computed

as

δ2

δ2

δ2

δ2

Note

that

as

and

as

Relative

export

errors

are

computed

by

the

sender

as

and

when

it

receives

the

feedback

from

the

receivers.

This

example

shows

the

relative

export

error

values

after

is

sent

and

the

corresponding

feedbacks

are

received.

Equalization

of

Error

Among

Receivers

We

now

explain

what

we

mean

by

making

the

errors

equal

at

all

the

receivers

and

how

this

can

be

achieved.

As

stated

before

the

sender

keeps

estimates

of

the

delays

to

the

receivers,

and

in

the

example

of

Figure

This

says

that

at

time

when

is

computed,

the

sender

already

knows

how

long

it

may

take

messages

carrying

this

DR

vector

to

reach

the

receivers.

The

sender

uses

this

information

to

compute

the

export

errors,

which

are

δ1

and

δ2

for

receivers

and

respectively.

Note

that

the

areas

of

these

error

components

are

a

function

of

δ1

and

δ2

as

well

as

the

network

delays

and

If

we

are

to

make

the

exports

errors

due

to

the

same

at

both

receivers,

the

sender

needs

to

choose

δ1

and

δ2

such

that

δ1

δ2

But

when

was

computed

there

could

already

have

been

accumulated

relative

export

errors

due

to

previous

DR

vectors

and

the

ones

before).

Let

us

represent

the

accumulated

relative

error

up

to

DRi

for

receiver

j

as

Ri

j.

To

accommodate

these

accumulated

relative

errors,

the

sender

should

now

choose

δ1

and

δ2

such

that

δ1

δ2

The

δi

determines

the

scheduling

instant

of

the

DR

vector

at

the

sender

for

receiver

i.

This

method

of

computation

of

δ"s

ensures

that

the

accumulated

export

error

(i.e.,

total

actual

error)

for

each

receiver

equalizes

at

the

transmission

of

each

DR

vector.

In

order

to

establish

this,

assume

that

the

feedback

for

DR

vector

Di

from

a

receiver

comes

to

the

sender

before

schedule

for

is

computed.

Let

Si

m

and

Ai

m

denote

the

estimated

error

for

receiver

m

used

for

computing

schedule

for

Di

and

accumulated

error

for

receiver

m

computed

after

receiving

feedback

for

Di,

respectively.

Then

Ri

m

Ai

m

−Si

m.

In

order

to

compute

the

schedule

instances

(i.e.,

δ"s)

for

Di,

for

any

pair

of

receivers

m

and

n,

we

do

m

Si

m

n

Si

n.

The

following

theorem

establishes

the

fact

that

the

accumulated

export

error

is

equalized

at

every

scheduling

instant.

THEOREM

When

the

schedule

instances

for

sending

Di

are

computed

for

any

pair

of

receivers

m

and

n,

the

following

condition

is

satisfied:

Ak

m

Si

m

Ak

n

Si

n.

Proof:

By

induction.

Assume

that

the

premise

holds

for

some

i.

We

show

that

it

holds

for

The

base

case

for

i

holds

since

initially

m

n

and

the

m

n

is

used

to

compute

the

scheduling

instances.

In

order

to

compute

the

schedule

for

the

we

first

compute

the

relative

errors

as

Ri

m

Ai

m

Si

m,

and

Ri

n

Ai

n

Si

n.

Then

to

compute

δ"s

we

execute

Ri

m

m

Ri

n

n

Ai

m

Si

m

m

Ai

n

Si

n

n

Adding

the

condition

of

the

premise

on

both

sides

we

get,

i

Ak

m

m

i

Ak

n

n

Computation

of

the

Export

Error

Let

us

now

consider

how

the

export

errors

can

be

computed.

From

the

previous

section,

to

find

δ1

and

δ2

we

need

to

find

and

Note

that

the

values

of

and

are

already

known

at

the

sender.

Consider

the

computation

of

This

is

the

integral

of

the

distance

between

the

trajectories

AC

due

to

and

BD

due

to

From

and

point

A

is

and

point

B

is

The

trajectory

AC

can

be

represented

as

a

function

of

time

as

t,

t)

and

the

trajectory

of

BD

can

be

represented

as

t,

t).

The

distance

between

the

two

trajectories

as

a

function

of

time

then

becomes,

dist(t)

Let

a

b

c

Then

dist(t)

can

be

written

as

dist(t)

a

b

t

c.

Then

for

some

time

interval

becomes

dist(t)

dt

a

b

t

c

dt.

A

closed

form

solution

for

the

indefinite

integral

a

b

t

c

dt

b)

bt

c

ln

at

a

bt

c

c

a

ln

at

a

bt

c

a−

and

can

then

be

calculated

by

applying

the

appropriate

limits

to

the

above

solution.

In

the

next

section,

we

consider

the

computation

of

the

δ"s

for

N

receivers.

Computation

of

Scheduling

Instants

We

again

look

at

the

computation

of

δ"s

by

referring

to

Figure

The

sender

chooses

δ1

and

δ2

such

that

δ1

If

and

both

are

zero,

then

δ1

and

δ2

should

be

chosen

such

that

δ1

This

equality

will

hold

if

δ1

δ2

Thus,

if

there

is

no

accumulated

relative

export

error,

all

that

the

sender

needs

to

do

is

to

choose

the

δ"s

in

such

a

way

that

they

counteract

the

difference

in

the

delay

to

the

two

receivers,

so

that

they

receive

the

DR

vector

at

the

same

time.

As

discussed

earlier,

because

the

sender

is

not

able

to

a

priori

learn

the

delay,

there

will

always

be

an

accumulated

relative

export

error

from

a

previous

DR

vector

that

does

have

to

be

taken

into

account.

To

delve

deeper

into

this,

consider

the

computation

of

the

export

error

as

illustrated

in

the

previous

section.

To

compute

the

δ"s

we

require

that

δ1

δ2

That

is,

dist(t)

dt

dist(t)

dt.

That

is

dist(t)

dt

dist(t)

dt

dist(t)

dt

dist(t)

dt.

The

components

are

already

known

to

(or

estimated

by)

the

sender.

Further,

the

error

components

dist(t)

dt

and

dist(t)

dt

can

be

a

priori

computed

by

the

sender

using

estimated

values

of

and

Let

us

use

to

denote

dist(t)

dt

and

to

denote

dist(t)

dt.

Then,

we

require

that

dist(t)

dt

dist(t)

dt.

Assume

that

Then,

for

the

above

equation

to

hold,

we

require

that

dist(t)

dt

dist(t)

dt.

To

make

the

game

as

fast

as

possible

within

this

framework,

the

δ

values

should

be

made

as

small

as

possible

so

that

DR

vectors

are

sent

to

the

receivers

as

soon

as

possible

subject

to

the

fairness

requirement.

Given

this,

we

would

choose

δ1

to

be

zero

and

compute

δ2

from

the

equation

dist(t)

dt.

In

general,

if

there

are

N

receivers

N,

when

a

sender

generates

a

DR

vector

and

decides

to

schedule

them

to

be

sent,

it

first

computes

the

Ei

values

for

all

of

them

from

the

accumulated

relative

export

errors

and

estimates

of

delays.

Then,

it

finds

the

smallest

of

these

values.

Let

Ek

be

the

smallest

value.

The

sender

makes

δk

to

be

zero

and

computes

the

rest

of

the

δ"s

from

the

equality

Ei

dist(t)

dt

Ek,

∀i

i

N,

i

k.

The

δ"s

thus

obtained

gives

the

scheduling

instants

of

the

DR

vector

for

the

receivers.

Steps

of

the

Scheduling

Algorithm

For

the

purpose

of

the

discussion

below,

as

before

let

us

denote

the

accumulated

relative

export

error

at

a

sender

for

receiver

k

up

until

DRi

to

be

Ri

k.

Let

us

denote

the

scheduled

delay

at

the

sender

before

DRi

is

sent

to

receiver

k

as

δi

k.

Given

the

above

discussion,

the

algorithm

steps

are

as

follows:

The

sender

computes

DRi

at

(say)

time

Ti

and

then

computes

δi

k,

and

k

∀k,

k

N

based

on

the

estimation

of

delays

dtk,

∀k,

k

N

as

per

Equation

It

schedules,

DRi

to

be

sent

to

receiver

k

at

time

Ti

δi

k.

The

DR

vectors

are

sent

to

the

receivers

at

the

scheduled

times

which

are

received

after

a

delay

of

dak,

∀k,

k

N

where

dak

or

dtk.

The

receivers

send

the

value

of

dak

back

to

the

sender

(the

receiver

can

compute

this

value

based

on

the

time

stamps

on

the

DR

vector

as

described

earlier).

The

sender

computes

Ri

k

as

described

earlier

and

illustrated

in

Figure

The

sender

also

recomputes

(using

exponential

averaging

method

similar

to

round-trip

time

estimation

by

TCP

the

estimate

of

delay

dtk

from

the

new

value

of

dak

for

receiver

k.

Go

back

to

Step

to

compute

when

it

is

required

and

follow

the

steps

of

the

algorithm

to

schedule

and

send

this

DR

vector

to

the

receivers.

Handling

Cases

in

Practice

So

far

we

implicity

assumed

that

DRi

is

sent

out

to

all

receivers

before

a

decision

is

made

to

compute

the

next

DR

vector

and

the

receivers

send

the

value

of

dak

corresponding

to

DRi

and

this

information

reaches

the

sender

before

it

computes

so

that

it

can

compute

k

and

then

use

it

in

the

computation

of

k

Two

issues

need

consideration

with

respect

to

the

above

algorithm

when

it

is

used

in

practice.

It

may

so

happen

that

a

new

DR

vector

is

computed

even

before

the

previous

DR

vector

is

sent

out

to

all

receivers.

How

will

this

situation

be

handled?

What

happens

if

the

feedback

does

not

arrive

before

is

computed

and

scheduled

to

be

sent?

Let

us

consider

the

first

scenario.

We

assume

that

DRi

has

been

scheduled

to

be

sent

and

the

scheduling

instants

are

such

that

δi

δi

δi

N

Assume

that

is

to

be

computed

(because

the

real

path

has

deviated

exceeding

a

threshold

from

the

path

exported

by

DRi)

at

time

where

Ti

δi

k

Ti

δi

This

means,

DRi

has

been

sent

only

to

receivers

up

to

k

in

the

scheduled

order.

In

our

algorithm,

in

this

case,

the

scheduled

delay

ordering

queue

is

flushed

which

means

DRi

is

not

sent

to

receivers

still

queued

to

receive

it,

but

a

new

scheduling

order

is

computed

for

all

the

receivers

to

send

For

those

receivers

who

have

been

sent

DRi,

assume

for

now

that

daj,

j

k

has

been

received

from

all

receivers

(the

scenario

where

daj

has

not

been

received

will

be

considered

as

a

part

of

the

second

scenario

later).

For

these

receivers,

Ei

j,

j

k

can

be

computed.

For

those

receivers

j,

k

j

N

to

whom

DRi

was

not

sent

Ei

j

does

not

apply.

Consider

a

receiver

j,

k

j

N

to

whom

DRi

was

not

sent.

Refer

to

Figure

For

such

a

receiver

j,

when

is

to

be

scheduled

and

timeTi

Exported

path

dtj

A

B

C

D

Gi

j

computed

by

sender

and

DRi

for

receiver

to

N

is

removed

from

queue

scheduled

for

receiver

G

H

E

F

DRi

scheduled

for

receiver

j

DRi

computed

by

sender

Placed

path

at

receiver

j

Figure

Schedule

computation

when

DRi

is

not

sent

to

receiver

j,

k

j

N.

j

needs

to

be

computed,

the

total

export

error

is

the

accumulated

relative

export

error

at

time

Ti

when

schedule

for

DRi

was

computed,

plus

the

integral

of

the

distance

between

the

two

trajectories

AC

and

BD

of

Figure

over

the

time

interval

[Ti,

j

dtj].

Note

that

this

integral

is

given

by

Err(DRi,

Ti,

j

dtj).

Therefore,

instead

of

Ei

j

of

Equation

we

use

the

value

j

Err(DRi,

Ti,

j

dtj)

where

j

is

relative

export

error

used

when

the

schedule

for

DRi

was

computed.

Now

consider

the

second

scenario.

Here

the

feedback

dak

corresponding

to

DRi

has

not

arrived

before

is

computed

and

scheduled.

In

this

case,

Ri

k

cannot

be

computed.

Thus,

in

the

computation

of

δk

for

this

will

be

assumed

to

be

zero.

We

do

assume

that

a

reliable

mechanism

is

used

to

send

dak

back

to

the

sender.

When

this

information

arrives

at

a

later

time,

Ri

k

will

be

computed

and

accumulated

to

future

relative

export

errors

(for

example

k

if

dak

is

received

before

is

computed)

and

used

in

the

computation

of

δk

when

a

future

DR

vector

is

to

be

scheduled

(for

example

Experimental

Results

In

order

to

evaluate

the

effectiveness

and

quantify

benefits

obtained

through

the

use

of

the

scheduling

algorithm,

we

implemented

the

proposed

algorithm

in

BZFlag

(Battle

Zone

Flag)

game.

It

is

a

first-person

shooter

game

where

the

players

in

teams

drive

tanks

and

move

within

a

battle

field.

The

aim

of

the

players

is

to

navigate

and

capture

flags

belonging

to

the

other

team

and

bring

them

back

to

their

own

area.

The

players

shoot

each

other"s

tanks

using

shooting

bullets.

The

movement

of

the

tanks

as

well

as

that

of

the

shots

are

exchanged

among

the

players

using

DR

vectors.

We

have

modified

the

implementation

of

BZFlag

to

incorporate

synchronized

clocks

among

the

players

and

the

server

and

exchange

time-stamps

with

the

DR

vector.

We

set

up

a

testbed

with

four

players

running

the

instrumented

version

of

BZFlag,

with

one

as

a

sender

and

the

rest

as

receivers.

The

scheduling

approach

and

the

base

case

where

each

DR

vector

was

sent

to

all

the

receivers

concurrently

at

every

trigger

point

were

implemented

in

the

same

run

by

tagging

the

DR

vectors

according

to

the

type

of

approach

used

to

send

the

DR

vector.

NISTNet

was

used

to

introduce

delays

across

the

sender

and

the

three

receivers.

Mean

delays

of

and

were

introduced

between

the

sender

and

first,

second

and

the

third

receiver,

respectively.

We

introduce

a

variance

of

msec

(to

the

mean

delay

of

each

receiver)

to

model

variability

in

delay.

The

sender

logged

the

errors

of

each

receiver

every

milliseconds

for

both

the

scheduling

approach

and

the

base

case.

The

sender

also

calculated

the

standard

deviation

and

the

mean

of

the

accumulated

export

error

of

all

the

receivers

every

milliseconds.

Figure

plots

the

mean

and

standard

deviation

of

the

accumulated

export

error

of

all

the

receivers

in

the

scheduling

case

against

the

base

case.

Note

that

the

x-axis

of

these

graphs

(and

the

other

graphs

that

follow)

represents

the

system

time

when

the

snapshot

of

the

game

was

taken.

Observe

that

the

standard

deviation

of

the

error

with

scheduling

is

much

lower

as

compared

to

the

base

case.

This

implies

that

the

accumulated

errors

of

the

receivers

in

the

scheduling

case

are

closer

to

one

another.

This

shows

that

the

scheduling

approach

achieves

fairness

among

the

receivers

even

if

they

are

at

different

distances

(i.e,

latencies)

from

the

sender.

Observe

that

the

mean

of

the

accumulated

error

increased

multifold

with

scheduling

in

comparison

to

the

base

case.

Further

exploration

for

the

reason

for

the

rise

in

the

mean

led

to

the

conclusion

that

every

time

the

DR

vectors

are

scheduled

in

a

way

to

equalize

the

total

error,

it

pushes

each

receivers

total

error

higher.

Also,

as

the

accumulated

error

has

an

estimated

component,

the

schedule

is

not

accurate

to

equalize

the

errors

for

the

receivers,

leading

to

the

DR

vector

reaching

earlier

or

later

than

the

actual

schedule.

In

either

case,

the

error

is

not

equalized

and

if

the

DR

vector

reaches

late,

it

actually

increases

the

error

for

a

receiver

beyond

the

highest

accumulated

error.

This

means

that

at

the

next

trigger,

this

receiver

will

be

the

one

with

highest

error

and

every

other

receiver"s

error

will

be

pushed

to

this

error

value.

This

flip-flop

effect

leads

to

the

increase

in

the

accumulated

error

for

all

the

receivers.

The

scheduling

for

fairness

leads

to

the

decrease

in

standard

deviation

(i.e.,

increases

the

fairness

among

different

players),

but

it

comes

at

the

cost

of

higher

mean

error,

which

may

not

be

a

desirable

feature.

This

led

us

to

explore

different

ways

of

equalizing

the

accumulated

errors.

The

approach

discussed

in

the

following

section

is

a

heuristic

approach

based

on

the

following

idea.

Using

the

same

amount

of

DR

vectors

over

time

as

in

the

base

case,

instead

of

sending

the

DR

vectors

to

all

the

receivers

at

the

same

frequency

as

in

the

base

case,

if

we

can

increase

the

frequency

of

sending

the

DR

vectors

to

the

receiver

with

higher

accumulated

error

and

decrease

the

frequency

of

sending

DR

vectors

to

the

receiver

with

lower

accumulated

error,

we

can

equalize

the

export

error

of

all

receivers

over

time.

At

the

same

time

we

wish

to

decrease

the

error

of

the

receiver

with

the

highest

accumulated

error

in

the

base

case

(of

course,

this

receiver

would

be

sent

more

DR

vectors

than

in

the

base

case).

We

refer

to

such

an

algorithm

as

a

budget

based

algorithm.

BUDGET

BASED

ALGORITHM

In

a

game,

the

sender

of

an

entity

sends

DR

vectors

to

all

the

receivers

every

time

a

threshold

is

crossed

by

the

entity.

Lower

the

threshold,

more

DR

vectors

are

generated

during

a

given

time

period.

Since

the

DR

vectors

are

sent

to

all

the

receivers

and

the

network

delay

between

the

sender-receiver

pairs

cannot

be

avoided,

the

before

export

error

with

the

most

distant

player

will

always

Note

that

after

export

error

is

eliminated

by

using

synchronized

clock

among

the

players.

MeanAccumulatedError

Time

in

Seconds

Base

Case

Scheduling

Algorithm

StandardDeviationofAccumulatedError

Time

in

Seconds

Base

Case

Scheduling

Algorithm

Figure

Mean

and

standard

deviation

of

error

with

scheduling

and

without

(i.e.,

base

case).

be

higher

than

the

rest.

In

order

to

mitigate

the

imbalance

in

the

error,

we

propose

to

send

DR

vectors

selectively

to

different

players

based

on

the

accumulated

errors

of

these

players.

The

budget

based

algorithm

is

based

on

this

idea

and

there

are

two

variations

of

it.

One

is

a

probabilistic

budget

based

scheme

and

the

other,

a

deterministic

budget

base

scheme.

Probabilistic

budget

based

scheme

The

probabilistic

budget

based

scheme

has

three

main

steps:

a)

lower

the

dead

reckoning

threshold

but

at

the

same

time

keep

the

total

number

of

DRs

sent

the

same

as

the

base

case,

b)

at

every

trigger,

probabilistically

pick

a

player

to

send

the

DR

vector

to,

and

c)

send

the

DR

vector

to

the

chosen

player.

These

steps

are

described

below.

The

lowering

of

DR

threshold

is

implemented

as

follows.

Lowering

the

threshold

is

equivalent

to

increasing

the

number

of

trigger

points

where

DR

vectors

are

generated.

Suppose

the

threshold

is

such

that

the

number

of

triggers

caused

by

it

in

the

base

case

is

t

and

at

each

trigger

n

DR

vectors

sent

by

the

sender,

which

results

in

a

total

of

nt

DR

vectors.

Our

goal

is

to

keep

the

total

number

of

DR

vectors

sent

by

the

sender

fixed

at

nt,

but

lower

the

number

of

DR

vectors

sent

at

each

trigger

(i.e.,

do

not

send

the

DR

vector

to

all

the

receivers).

Let

n

and

t

be

the

number

of

DR

vectors

sent

at

each

trigger

and

number

of

triggers

respectively

in

the

modified

case.

We

want

to

ensure

n

t

nt.

Since

we

want

to

increase

the

number

of

trigger

points,

i.e,

t

t,

this

would

mean

that

n

n.

That

is,

not

all

receivers

will

be

sent

the

DR

vector

at

every

trigger.

In

the

probabilistic

budget

based

scheme,

at

each

trigger,

a

probability

is

calculated

for

each

receiver

to

be

sent

a

DR

vector

and

only

one

receiver

is

sent

the

DR

(n

This

probability

is

based

on

the

relative

weights

of

the

receivers"

accumulated

errors.

That

is,

a

receiver

with

a

higher

accumulated

error

will

have

a

higher

probability

of

being

sent

the

DR

vector.

Consider

that

the

accumulated

error

for

three

players

are

and

respectively.

Then

the

probability

of

player

receiving

the

DR

vector

would

be

Similarly

for

the

other

players.

Once

the

player

is

picked,

the

DR

vector

is

sent

to

that

player.

To

compare

the

probabilistic

budget

based

algorithm

with

the

base

case,

we

needed

to

lower

the

threshold

for

the

base

case

(for

fair

comparison).

As

the

dead

reckoning

threshold

in

the

base

case

was

already

very

fine,

it

was

decided

that

instead

of

lowering

the

threshold,

the

probabilistic

budget

based

approach

would

be

compared

against

a

modified

base

case

that

would

use

the

normal

threshold

as

the

budget

based

algorithm

but

the

base

case

was

modified

such

that

every

third

trigger

would

be

actually

used

to

send

out

a

DR

vector

to

all

the

three

receivers

used

in

our

experiments.

This

was

called

as

the

base

case

as

it

resulted

in

number

of

DR

vectors

being

sent

as

compared

to

the

base

case.

The

budget

per

trigger

for

the

probability

based

approach

was

calculated

as

one

DR

vector

at

each

trigger

as

compared

to

three

DR

vectors

at

every

third

trigger

in

the

base

case;

thus

the

two

cases

lead

to

the

same

number

of

DR

vectors

being

sent

out

over

time.

In

order

to

evaluate

the

effectiveness

of

the

probabilistic

budget

based

algorithm,

we

instrumented

the

BZFlag

game

to

use

this

approach.

We

used

the

same

testbed

consisting

of

one

sender

and

three

receivers

with

delays

of

and

from

the

sender

and

with

low

delay

variance

and

moderate

delay

variance

The

results

are

shown

in

Figures

and

As

mentioned

earlier,

the

x-axis

of

these

graphs

represents

the

system

time

when

the

snapshot

of

the

game

was

taken.

Observe

from

the

figures

that

the

standard

deviation

of

the

accumulated

error

among

the

receivers

with

the

probabilistic

budget

based

algorithm

is

less

than

the

base

case

and

the

mean

is

a

little

higher

than

the

base

case.

This

implies

that

the

game

is

fairer

as

compared

to

the

base

case

at

the

cost

of

increasing

the

mean

error

by

a

small

amount

as

compared

to

the

base

case.

The

increase

in

mean

error

in

the

probabilistic

case

compared

to

the

base

case

can

be

attributed

to

the

fact

that

the

even

though

the

probabilistic

approach

on

average

sends

the

same

number

of

DR

vectors

as

the

base

case,

it

sometimes

sends

DR

vectors

to

a

receiver

less

frequently

and

sometimes

more

frequently

than

the

base

case

due

to

its

probabilistic

nature.

When

a

receiver

does

not

receive

a

DR

vector

for

a

long

time,

the

receiver"s

trajectory

is

more

and

more

off

of

the

sender"s

trajectory

and

hence

the

rate

of

buildup

of

the

error

at

the

receiver

is

higher.

At

times

when

a

receiver

receives

DR

vectors

more

frequently,

it

builds

up

error

at

a

lower

rate

but

there

is

no

way

of

reversing

the

error

that

was

built

up

when

it

did

not

receive

a

DR

vector

for

a

long

time.

This

leads

the

receivers

to

build

up

more

error

in

the

probabilistic

case

as

compared

to

the

base

case

where

the

receivers

receive

a

DR

vector

almost

periodically.

MeanAccumulatedError

Time

in

Seconds

Base

Case

Deterministic

Algorithm

Probabilistic

Algorithm

StandardDeviationofAccumulatedError

Time

in

Seconds

Base

Case

Deterministic

Algorithm

Probabilistic

Algorithm

Figure

Mean

and

standard

deviation

of

error

for

different

algorithms

(including

budget

based

algorithms)

for

low

delay

variance.

MeanAccumulatedError

Time

in

Seconds

Base

Case

Deterministic

Algorithm

Probabilistic

Algorithm

StandardDeviationofAccumulatedError

Time

in

Seconds

Base

Case

Deterministic

Algorithm

Probabilistic

Algorithm

Figure

Mean

and

standard

deviation

of

error

for

different

algorithms

(including

budget

based

algorithms)

for

moderate

delay

variance.

Deterministic

budget

based

scheme

To

bound

the

increase

in

mean

error

we

decided

to

modify

the

budget

based

algorithm

to

be

deterministic.

The

first

two

steps

of

the

algorithm

are

the

same

as

in

the

probabilistic

algorithm;

the

trigger

points

are

increased

to

lower

the

threshold

and

accumulated

errors

are

used

to

compute

the

probability

that

a

receiver

will

receiver

a

DR

vector.

Once

these

steps

are

completed,

a

deterministic

schedule

for

the

receiver

is

computed

as

follows:

If

there

is

any

receiver(s)

tagged

to

receive

a

DR

vector

at

the

current

trigger,

the

sender

sends

out

the

DR

vector

to

the

respective

receiver(s).

If

at

least

one

receiver

was

sent

a

DR

vector,

the

sender

calculates

the

probabilities

of

each

receiver

receiving

a

DR

vector

as

explained

before

and

follows

steps

to

else

it

does

not

do

anything.

For

each

receiver,

the

probability

value

is

multiplied

with

the

budget

available

at

each

trigger

(which

is

set

to

as

explained

below)

to

give

the

frequency

of

sending

the

DR

vector

to

each

receiver.

If

any

of

the

receiver"s

frequency

after

multiplying

with

the

budget

goes

over

the

receiver"s

frequency

is

set

as

and

the

surplus

amount

is

equally

distributed

to

all

the

receivers

by

adding

the

amount

to

their

existing

frequencies.

This

process

is

repeated

until

all

the

receivers

have

a

frequency

of

less

than

or

equal

to

This

is

due

to

the

fact

that

at

a

trigger

we

cannot

send

more

than

one

DR

vector

to

the

respective

receiver.

That

will

be

wastage

of

DR

vectors

by

sending

redundant

information.

gives

us

the

schedule

at

which

the

sender

should

send

DR

vectors

to

the

respective

receiver.

Credit

obtained

previously

(explained

in

step

if

any

is

subtracted

from

the

schedule.

Observe

that

the

resulting

value

of

the

schedule

might

not

be

an

integer;

hence,

the

value

is

rounded

off

by

taking

the

ceiling

of

the

schedule.

For

example,

if

the

frequency

is

this

implies

that

we

would

like

to

have

a

DR

vector

sent

every

triggers.

However,

we

are

constrained

to

send

it

at

the

trigger

giving

us

a

credit

of

When

we

do

send

the

DR

vector

next

time,

we

would

be

able

to

send

it

on

the

trigger

because

of

the

credit.

The

difference

between

the

schedule

and

the

ceiling

of

the

schedule

is

the

credit

that

the

receiver

has

obtained

which

is

remembered

for

the

future

and

used

at

the

next

time

as

explained

in

step

For

each

of

those

receivers

who

were

sent

a

DR

vector

at

the

current

trigger,

the

receivers

are

tagged

to

receive

the

next

DR

vector

at

the

trigger

that

happens

exactly

schedule

(the

ceiling

of

the

schedule)

number

of

times

away

from

the

current

trigger.

Observe

that

no

other

receiver"s

schedule

is

modified

at

this

point

as

they

all

are

running

a

schedule

calculated

at

some

previous

point

of

time.

Those

schedules

will

be

automatically

modified

at

the

trigger

when

they

are

scheduled

to

receive

the

next

DR

vector.

At

the

first

trigger,

the

sender

sends

the

DR

vector

to

all

the

receivers

and

uses

a

relative

probability

of

for

each

receiver

and

follows

the

steps

to

to

calculate

the

next

schedule

for

each

receiver

in

the

same

way

as

mentioned

for

other

triggers.

This

algorithm

ensures

that

every

receiver

has

a

guaranteed

schedule

of

receiving

DR

vectors

and

hence

there

is

no

irregularity

in

sending

the

DR

vector

to

any

receiver

as

was

observed

in

the

budget

based

probabilistic

algorithm.

We

used

the

testbed

described

earlier

(three

receivers

with

varying

delays)

to

evaluate

the

deterministic

algorithm

using

the

budget

of

DR

vector

per

trigger

so

as

to

use

the

same

number

of

DR

vectors

as

in

the

base

case.

Results

from

our

experiments

are

shown

in

Figures

and

It

can

be

observed

that

the

standard

deviation

of

error

in

the

deterministic

budget

based

algorithm

is

less

than

the

base

case

and

also

has

the

same

mean

error

as

the

base

case.

This

indicates

that

the

deterministic

algorithm

is

more

fair

than

the

base

case

and

at

the

same

time

does

not

increase

the

mean

error

thereby

leading

to

a

better

game

quality

compared

to

the

probabilistic

algorithm.

In

general,

when

comparing

the

deterministic

approach

to

the

probabilistic

approach,

we

found

that

the

mean

accumulated

error

was

always

less

in

the

deterministic

approach.

With

respect

to

standard

deviation

of

the

accumulated

error,

we

found

that

in

the

fixed

or

low

variance

cases,

the

deterministic

approach

was

generally

lower,

but

in

higher

variance

cases,

it

was

harder

to

draw

conclusions

as

the

probabilistic

approach

was

sometimes

better

than

the

deterministic

approach.

CONCLUSIONS

AND

FUTURE

WORK

In

distributed

multi-player

games

played

across

the

Internet,

object

and

player

trajectory

within

the

game

space

are

exchanged

in

terms

of

DR

vectors.

Due

to

the

variable

delay

between

players,

these

DR

vectors

reach

different

players

at

different

times.

There

is

unfair

advantage

gained

by

receivers

who

are

closer

to

the

sender

of

the

DR

as

they

are

able

to

render

the

sender"s

position

more

accurately

in

real

time.

In

this

paper,

we

first

developed

a

model

for

estimating

the

error

in

rendering

player

trajectories

at

the

receivers.

We

then

presented

an

algorithm

based

on

scheduling

the

DR

vectors

to

be

sent

to

different

players

at

different

times

thereby

equalizing

the

error

at

different

players.

This

algorithm

is

aimed

at

making

the

game

fair

to

all

players,

but

tends

to

increase

the

mean

error

of

the

players.

To

counter

this

effect,

we

presented

budget

based

algorithms

where

the

DR

vectors

are

still

scheduled

to

be

sent

at

different

players

at

different

times

but

the

algorithm

balances

the

need

for

fairness

with

the

requirement

that

the

error

of

the

worst

case

players

(who

are

furthest

from

the

sender)

are

not

increased

compared

to

the

base

case

(where

all

DR

vectors

are

sent

to

all

players

every

time

a

DR

vector

is

generated).

We

presented

two

variations

of

the

budget

based

algorithms

and

through

experimentation

showed

that

the

algorithms

reduce

the

standard

deviation

of

the

error

thereby

making

the

game

more

fair

and

at

the

same

time

has

comparable

mean

error

to

the

base

case.

REFERENCES

S.Aggarwal,

H.

Banavar,

A.

Khandelwal,

S.

Mukherjee,

and

S.

Rangarajan,

Accuracy

in

Dead-Reckoning

based

Distributed

Multi-Player

Games,

Proceedings

of

ACM

SIGCOMM

Workshop

on

Network

and

System

Support

for

Games

(NetGames

Aug.

L.

Gautier

and

C.

Diot,

Design

and

Evaluation

of

MiMaze,

a

Multiplayer

Game

on

the

Internet,

in

Proc.

of

IEEE

Multimedia

M.

Mauve,

Consistency

in

Replicated

Continuous

Interactive

Media,

in

Proc.

of

the

ACM

Conference

on

Computer

Supported

Cooperative

Work

pp.

S.K.

Singhal

and

D.R.

Cheriton,

Exploiting

Position

History

for

Efficient

Remote

Rendering

in

Networked

Virtual

Reality,

Presence:

Teleoperators

and

Virtual

Environments,

vol.

no.

pp.

C.

Diot

and

L.

Gautier,

A

Distributed

Architecture

for

Multiplayer

Interactive

Applications

on

the

Internet,

in

IEEE

Network

Magazine,

vol.

pp.

L.

Pantel

and

L.C.

Wolf,

On

the

Impact

of

Delay

on

Real-Time

Multiplayer

Games,

in

Proc.

of

ACM

May

Y.

Lin,

K.

Guo,

and

S.

Paul,

Sync-MS:

Synchronized

Messaging

Service

for

Real-Time

Multi-Player

Distributed

Games,

in

Proc.

of

IEEE

International

Conference

on

Network

Protocols

(ICNP),

Nov

K.

Guo,

S.

Mukherjee,

S.

Rangarajan,

and

S.

Paul,

A

Fair

Message

Exchange

Framework

for

Distributed

Multi-Player

Games,

in

Proc.

of

May

N.

E.

Baughman

and

B.

N.

Levine,

Cheat-Proof

Playout

for

Centralized

and

Distributed

Online

Games,

in

Proc.

of

IEEE

April

M.

Allman

and

V.

Paxson,

On

Estimating

End-to-End

Network

Path

Properties,

in

Proc.

of

ACM

Sept.

BZFlag

Forum,

BZFlag

Game,

URL:

Nation

Institute

of

Standards

and

Technology,

NIST

Net,

URL:

dead-reckoning vector
distributed multi-player game
distribute multi-player game
dead-reckon
clock synchronization

fairness
budget based algorithm
mean error
accuracy
scheduling algorithm
quantization
export error
bucket synchronization
network delay