# The $\sigma$ law of evolutionary dynamics in community-structured
populations

$Changbing~{}Tang^{a},~{}Xiang~{}Li^{a},~{}Lang~{}Cao^{b},~{}Jingyuan~{}Zhan^{a}$111
_Email address_<EMAIL_ADDRESS>aAdaptive Networks and Control Lab,
Department of Electronic Engineering Fudan University, Shanghai 200433, China
bDepartment of Mathematical Engineering and Information Physics, University of
Tokyo, Tokyo 153-8505, Japan

###### Abstract

Evolutionary game dynamics in finite populations provides a new framework to
understand the selection of traits with frequency-dependent fitness. Recently,
a simple but fundamental law of evolutionary dynamics, which we call $\sigma$
law, describes how to determine the selection between two competing
strategies: in most evolutionary processes with two strategies, $A$ and $B$,
strategy $A$ is favored over $B$ in weak selection if and only if $\sigma
R+S>T+\sigma P$. This relationship holds for a wide variety of structured
populations with mutation rate and weak selection under certain assumptions.
In this paper, we propose a model of games based on a community-structured
population and revisit this law under the Moran process. By calculating the
average payoffs of $A$ and $B$ individuals with the method of effective
sojourn time, we find that $\sigma$ features not only the structured
population characteristics but also the reaction rate between individuals.
That’s to say, an interaction between two individuals are not uniform, and we
can take $\sigma$ as a reaction rate between any two individuals with the same
strategy. We verify this viewpoint by the modified replicator equation with
non-uniform interaction rates in a simplified version of the prisoner’s
dilemma game (PDG).

###### keywords:

Evolutionary game theory, Sojourn time, Non-uniform interaction rate,
Simplified PDG.

††journal: Theoretical Biology

##  1 Introduction

Evolutionary game theory was originally introduced as a tool for studying
animal behavior (Maynard and Price,, 1973; Maynard,, 1982) but has become a
general approach that transcends almost every aspect of evolutionary biology
(Nowak and Sigmund, 2004a, ), and provides a framework to understand the
dynamics of frequency-dependent selection (Hofbauer and Sigmund,, 1998;
Gintis,, 2000; Cressman,, 2003; Nowak and Sigmund, 2004a, ; Nowak,, 2006;
Antal et al., 2009a, ; Wu et al.,, 2010; Gokhale and Traulsen,, 2011; Tarnita
et al.,, 2011).

The traditional approach of evolutionary game theory uses deterministic
dynamics to describe infinitely large, well-mixed populations (Hofbauer and
Sigmund,, 1988, 2003). To understand evolutionary game dynamics in finite-
sized populations, a stochastic approach is developed (Schaffer,, 1988; Fogel
et al.,, 1998; Ficici et al.,, 2000; Alos-Ferrer,, 2003; Perc,, 2007; Traulsen
and Nowak,, 2007). A crucial quantity in the stochastic approach is the
fixation probability of strategies which relies on the individual’s
reproduction ability (Nowak et al., 2004b, ; Taylor et al.,, 2004; Imhof and
Nowak,, 2006; Nowak,, 2006; Traulsen et al., 2006a, ; Lessard and Ladret,,
2007; Bomze and Pawlowitsch,, 2008). In the limit of weak selection, for a
neutral mutant, probability $\rho_{A}$, that an individual using strategy $A$
will invade and take over the whole population of individuals using strategy
$B$, is equal to the reciprocal of the population size, i.e., $1/N$.
Therefore, the selection favors the fixation of invading strategy $A$ if
$\rho_{A}>1/N$ under the weak selection.

Evolutionary game dynamics are also affected by the structure of population.
Lots of early works focused on spatial games with regular lattices (Nowak and
May, 1992; Nakamaru et al., 1997; Hauert and Doebeli, 2004; Szabó and Fath,
1998; Szabó et al., 2000), which recently, have been expanded to general
structured populations, such as graphs (Lieberman et al.,, 2005; Ohtsuki et
al., 2006a, ; Ohtsuki and Nowak, 2006b, ; Ohtsuki et al., 2007a, ; Taylor et
al.,, 2007; Cao and Li,, 2008; Fu et al.,, 2009; Li and Cao,, 2009; Perc and
Wang,, 2010; Wang et al.,, 2011), phenotype space (Antal et al., 2009b, ) and
set structured populations (Tarnita et al., 2009a, ). In a word, a population
structure specifies who interacts with whom, and it greatly affects the
outcome of an evolution. If the fitness of an individual is determined by its
interactions with others, then we are in the world of evolutionary game theory
(Nathanson et al.,, 2009).

Consider a game of two strategies, $A$ and $B$, in a population of fixed size
$N$. Mutual cooperation is rewarded by $R$ for each player, whereas mutual
defection pays each player a punishment $P$. If a cooperator plays against a
defector, the former gets the sucker s payoff $S$ and the latter gets the
temptation to defect $T$. Thus, the interactions are given by the payoff
matrix

$\begin{array}[]{ccc}{}\hfil&A&B\\\ A&R&S\\\ B&T&P\\\ \end{array}$ (1)

Each individual obtains a payoff by interacting with others according to the
population structure, and reproduces either genetically or culturally with a
rate proportional to its payoff. Of course, reproduction is subject to
mutation. Whenever an individual reproduces, the offspring adopts the parent’s
strategy with probability $1-\nu$ and a random strategy with probability
$\nu$. To make the evolutionary dynamics of the process well approximated by
an embedded Markov chain on the pure states (Wu et al.,, 2011), it’s
considered that mutation rate $\nu<(NlnN)^{-1}\ll 1$.

It is said that strategy $A$ is selected over strategy $B$ if it is more
abundant in the stationary distribution of the mutation-selection process.
Tarnita et al., 2009b  showed that with the weak selection, the condition that
strategy $A$ is more abundant than strategy $B$ in the stationary distribution
of the mutation-selection process is

$\begin{array}[]{l}\sigma R+S>T+\sigma P\end{array}$ (2)

From this inequality, we know that the condition specifying which strategy is
more abundant is a linear inequality of the payoff values, $R$, $S$, $T$, $P$.
It’s pointed out that parameter $\sigma$, reflects how the structured
populations influence the evolutionary dynamics (Nowak et al.,, 2010), and
this inequality holds for a wide variety of population structures, including
well mixed populations, graphs, phenotype space and set structured populations
(Nathanson et al.,, 2009). Conveniently, we call this simple but fundamental
law as the $\sigma$ law.

Recently, attention are focused on the population structures allowing
selection at multiple levels (Girvan and Newman,, 2002; Newman,, 2006;
Traulsen et al., 2005a, ; Traulsen et al., 2006b, ; Traulsen et al.,, 2008;
Wang et al.,, 2011; Hauert and Imhof,, 2011; Wang et al.,, 2012). In this
case, a population is divided into groups (or communities), and the selection
between individuals can be either in a community or among several communities.
Following this line, we propose a model of games on a community-structured
population based on the graph theory with a finite size. We verify that
evolutionary games on the community-structured population obeys the $\sigma$
law from the viewpoint of interaction under the Moran process with sojourn
time. Along the path of an invasion attempt, we find that the probability of
an interaction between any two individuals with the same strategy is $\sigma$
times of that between different strategies. In other words, $\sigma$
represents the reaction rate between two individuals with the same strategy,
and quantifies the degree that individuals using the same strategy are more
($\sigma>1$) or less ($\sigma<1$) likely to interact than individuals using
different strategies.

##  2 $\sigma$ law in community-structured populations

###  2.1 Model of community structures

Consider a game played in a finite population of the fixed size $H$
distributed over $M$ communities (each community has the same size $N$, and
any two of which have none common member) with strategies $A$ and $B$. Each
individual obtains a payoff according to the payoff matrix $(1)$. At each time
step, a single individual is selected proportional to its fitness for
reproduction, and the offspring either replaces a randomly chosen individual
in this community or moves to another community at any time step to replaces a
randomly chosen individual. In Hauert and Imhof, (2011), the authors discussed
the evolutionary dynamics in a deme structured population, and assumed that
the migration rate is proportional to deme’s fitness. In this paper, we
suppose that there exists a graph in the structured population connecting
communities, and the migration occurs between two communities if there is an
edge between them (See Fig. 1).

Label all communities in the structured populations with $l=1,2,\cdots,M$. The
probability that a selected individual belongs to community $l$ migrates to
community $m$ is given by $\lambda_{lm}$. Hence the migration process is
determined by an $M\times M$ matrix, $\Lambda=[\lambda_{lm}]$, $\lambda_{ll}$
is the probability that an individual stays in community $l$. Note that
$\lambda_{ll}+\sum_{m\in\Omega_{l}}^{M}\lambda_{lm}=1$, where $\Omega_{l}$
denotes the set of all the communities that individuals in community $l$ can
migrate to. Thus $\Lambda$ is a stochastic matrix with an eigenvalue $1$, and
no eigenvalue has an absolute value greater than $1$ (Godsil and Royle,,
2001).

Assume that a mutant $A$-individual (individual using strategy $A$) is
produced from a community $l$ with pure strategy $B$. This mutant replaces a
randomly chosen individual in community $l$ with probability $\lambda_{ll}$,
or migrates to community $m$ and replaces a randomly chosen individual in
community $m$ with probability $\lambda_{lm},m\in\Omega_{l}$. Since the graph
connects all the communities as its nodes, there exists a path along which a
mutant beginning in community $l,l\in\\{1,2,\cdots,M\\}$ can be fixed in the
whole population (note that the path is not unique). An example that the
fixation process of a single mutant in the whole community-structured
population is shown in Fig. 2, where the fixation of a mutant from community
$1$ to take over the whole population corresponds to a path.

###  2.2 $\sigma$ law in community-structured population with sojourn time

Assume that a migration occurs very rarely and much less than mutation
($\lambda_{lm}\ll\nu<(NlnN)^{-1}\ll 1,m\in\Omega_{l})$. In the limit of
$\lambda_{lm}\rightarrow 0$ and $\nu\rightarrow 0$, the population spends
almost all the time in the homogeneous states with all individuals of type $A$
or type $B$. Since mutations occur much more frequently than migrations, a
migrant has either invaded the whole community or been eliminated before the
next migration takes place (Traulsen et al., 2006b, ). Due to this special
structure, the interactions within a community is more frequent than those
between communities. This allows us to use the method of sojourn time in the
Moran process which was applied to explain the one-third law of evolutionary
dynamics in Ohtsuki et al., 2007b .

Suppose that a single individual from $l$-community is selected proportional
to its fitness for reproduction. This newborn has to stay or migrate to
another community and replaces a randomly selected individual, and the total
number of the population keeps unchanged. Based on the migration matrix
$\Lambda$ constructed previously, we know that the newborn stays in community
$l$ with probability $\lambda_{ll}$, and migrate to community $m$ with
probability $\lambda_{lm}$, where $\lambda_{ll}\gg\lambda_{lm},m\neq l$.

First, we discuss the Moran process that happens in $l$-community with
probability $\lambda_{ll}$. Among the total $N$ individuals in community $l$,
$j$ of them follow strategy $A$, and $N-j$ follow strategy $B$. Introduce a
parameter $\omega$ to measure the intensity of selection. The corresponding
fitness of A and B individuals are given by

$\begin{array}[]{l}f_{j}=1-\omega+\omega F_{j}\\\ g_{j}=1-\omega+\omega
G_{j}\end{array}$

where $F_{j}$ and $G_{j}$ are the expected payoffs for $A$ and $B$,
respectively. In this paper, we consider the weak selection, i.e.,
$\omega\rightarrow 0$. In the state space $\\{0,\cdots,N\\}$, there are two
absorbing states, $j=0$ (which means $A$ has become extinct) and $j=N$ (which
means $A$ has reached fixation). Then the transition probability $p_{j,j\pm
1}$ is given by

$\begin{array}[]{l}p_{j,j+1}=(N-j)/N\cdot jf_{j}/[jf_{j}+(N-j)g_{j}]\\\
p_{j,j-1}=j/N\cdot[(N-j)g_{j}]/[jf_{j}+(N-j)g_{j}]\\\
p_{j,j}=1-p_{j,j+1}-p_{j,j-1}\end{array}$ (3)

Consider a path along which a mutant invades a resident population from state
$j$ $(1\leq j\leq N-1)$. Since the mutant eventually gets absorbed into either
state $0$ or state $N$, we are interested in how much time is spent at state
$j$ along the path. In Ohtsuki et al., 2007b , the authors called it the
sojourn time. Given all paths that start at state $i$, let $\bar{t}_{ij}$ be
the mean sojourn time at state $j$ before the absorption into either state $0$
or state $N$, which is described as (see, for example, Ewens, (2004)).

$\bar{t}_{ij}=\sum_{k=0}^{k^{l}}p_{ik}\bar{t}_{kj}+\delta_{ij},~{}~{}~{}\bar{t}_{0j}=\bar{t}_{Nj}=0$
(4)

In particular, if we start from a single mutant with no loss of generality,
i.e., $i=1$, then Eq. (4) can be simplified as

$\overline{t}_{1j}=N/j.$ (5)

This means that the stochastic process spends most of the time around the
absorbing state, $j=0$. Besides, the mean effective sojourn time
$\overline{\tau}_{ij}$ at state $j$ for paths that start at state $i$ is given
by $\overline{\tau}_{ij}=(p_{j,j+1}+p_{j,j-1})\overline{t_{ij}}$. Now, we only
concern the mean effective sojourn time $\overline{\tau}_{1j}$ which starts
from a single mutant under the neutral drift. Since the fitness $f_{j}$ and
$g_{j}$ are very close to $1$ under weak selection, with Eqs. (3) and (5), we
obtain

$\begin{array}[]{l}\overline{\tau}_{1j}=(p_{j,j+1}+p_{j,j-1})\overline{t_{1j}}\\\
=\\{(N-j)/N\cdot
jf_{j}/[jf_{j}+(N-j)g_{j}]+j/N\cdot[(N-j)g_{j}]/[jf_{j}+(N-j)g_{j}]\\}N/j\\\
\approx 2(N-j)/N\end{array}$ (6)

We now can count the average number of games played by $A$-individual and
$B$-individual along a path of invasion. At state $j$, an $A$-individual meets
$(j-1)$ $A$-individuals and $(N-j)$ $B$-individuals, while a $B$-individual
meets $j$ $A$-individuals and $(N-j-1)$ $B$-individuals. The average number of
games played by $A$-individual and $B$-individual at state $j$ can be
summarized as (Ohtsuki et al., 2007b, )

$\left(\begin{array}[]{cc}A\rightarrow A&A\rightarrow B\\\ B\rightarrow
A&B\rightarrow B\\\ \end{array}\right)_{j}=\left(\begin{array}[]{cc}j-1&N-j\\\
j&N-j-1\\\ \end{array}\right)$ (7)

Therefore, the effective number of encounters are

$\sum_{j=1}^{N-1}\overline{\tau}_{1j}\left(\begin{array}[]{cc}j-1&N-j\\\
j&N-j-1\\\ \end{array}\right)=$
$2\left(\begin{array}[]{cc}\sum_{j=1}^{N-1}\frac{1}{N}(j-1)(N-j)&\sum_{j=1}^{N-1}\frac{1}{N}(N-j)(N-j)\\\
\sum_{j=1}^{N-1}\frac{1}{N}j(N-j)&\sum_{j=1}^{N-1}\frac{1}{N}(N-j-1)(N-j)\\\
\end{array}\right)$
$=2\cdot\left(\begin{array}[]{cc}\sum_{j=1}^{N-1}\frac{1}{N}(j-1)N_{B}^{l}&\sum_{j=1}^{N-1}\frac{1}{N}(N-j)N_{B}^{l}\\\
\sum_{j=1}^{N-1}\frac{1}{N}(N-j)N_{A}^{l}&\sum_{j=1}^{N-1}\frac{1}{N}(j-1)N_{A}^{l}\\\
\end{array}\right)$ (8)

where $N_{A}^{l}$ is the number of $A$-individuals in community $l$ at state
$j$, $N_{B}^{l}=N-N_{A}^{l}$ is the number of $B$-individuals in community $l$
at state $j$. Furthermore, denote $I_{XX}^{l}$ the total number of
interactions that an $X$-individual interacts with other $X$-individuals, and
$I_{XY}^{l}(X\neq Y)$ is the total number of interactions that an
$X$-individual interacts with $Y$-individuals, where $X,Y\in\\{A,B\\}$.
Symmetrically, we have the following equations $\langle
I_{AA}^{l}N_{B}^{l}\rangle_{0}=\langle I_{BB}^{l}N_{A}^{l}\rangle_{0}$,
$\langle I_{AB}^{l}N_{B}^{l}\rangle_{0}=\langle
I_{BA}^{l}N_{A}^{l}\rangle_{0}$ at neutrality (Nathanson et al.,, 2009). The
notation $\langle\cdot\rangle_{0}$ denotes the quantity averaged over all
states of the stochastic process under neutral drift, $w=0$. Thus, Eq. (8) can
be simplified as

$2\left(\begin{array}[]{cc}\langle I_{AA}^{l}N_{B}^{l}\rangle_{0}&\langle
I_{AB}^{l}N_{B}^{l}\rangle_{0}\\\ \langle
I_{BA}^{l}N_{A}^{l}\rangle_{0}&\langle I_{BB}^{l}N_{A}^{l}\rangle_{0}\\\
\end{array}\right)$
$~{}~{}~{}\rightarrow~{}\left(\begin{array}[]{cc}\sigma^{l}&1\\\
1&\sigma^{l}\\\ \end{array}\right)$ (9)

where $\sigma^{l}=\langle I_{AA}^{l}N_{B}^{l}\rangle_{0}/\langle
I_{AB}^{l}N_{B}^{l}\rangle_{0}$. On the other hand, the newborn reproduced in
community $l$ migrates to community $m$ with probability $\lambda_{lm}$.
Suppose that in community $m$ there are $h$ $A$-individuals and $N-h$
$B$-individuals, where $N$ is the size of community $m$. Similarly, we can get
the effective sojourn time $\overline{\tau}_{1h}\approx 2(N-h)/N$. And the
effective number of encounters can be written as

$\left(\begin{array}[]{cc}\sigma^{m}&1\\\ 1&\sigma^{m}\\\ \end{array}\right)$
(10)

where $\sigma^{m}=\langle I_{AA}^{m}N_{B}^{m}\rangle_{0}/\langle
I_{AB}^{m}N_{B}^{m}\rangle_{0}$.

Since we only discuss the number of interactions along a path that the mutant
invades the whole population, the total number of interactions is a linear
composition of the number of interactions in every community. So the total
effective number of interactions can be finally written as

$\left(\begin{array}[]{cc}\sigma&1\\\ 1&\sigma\\\ \end{array}\right)$ (11)

where $\sigma=c_{l}\sigma^{l}+\sum_{m\in\Omega_{l}}^{M}c_{m}\sigma^{m}$. Here,
the coefficient $c_{n}(n\in l\cup\Omega_{l})$, reflects the strength of effect
from effective number of interactions in community $n$ to total effective
number of interactions.

Eq. (11) suggests that both types of individuals with the same strategy
interact $\sigma$ times as often as they interact with the other individuals.
Therefore the average payoffs of $A$ and $B$ individuals along an invasion-
path in a community-structured population are $\sigma R+S$ and $T+\sigma P$,
respectively. From this result we can get the $\sigma$ law under the weak
selection in a community-structured population, i.e., the condition that
strategy $A$ is more abundant than strategy $B$ in the stationary distribution
is $\sigma R+S>T+\sigma P$.

##  3 $\sigma$ law in the case of non-uniform interaction

The $\sigma$ law provides a fundamental criterion to specify which strategy is
more abundant in a structured population. But, what’s the $\sigma$? In Nowak
et al., (2010), the authors said that $\sigma$ is the ‘structure coefficient’,
and it reflects how the structured population influences the evolutionary
dynamics. In this section, we point out that $\sigma$ also features the
reaction rate between two individuals with the same strategy.

To verify this point, we will use the evolutionary game theory with non-
uniform interaction rates discussed in Taylor and Nowak, (2006). They assumed
that the probability of interaction between two individuals is dependent of
their strategies, and described the interaction as

$\begin{array}[]{l}A+A^{~{}\underrightarrow{~{}~{}r_{1}~{}~{}}~{}}AA\\\
A+B^{~{}\underrightarrow{~{}~{}r_{2}~{}~{}}~{}}AB\\\
B+B^{~{}\underrightarrow{~{}~{}r_{3}~{}~{}}~{}}BB\end{array}$

Let us consider a simplified version of the prisoner’s dilemma game (PDG) with
the payoff matrix as below

$\begin{array}[]{ccc}{}\hfil&C&D\\\ C&b-c&-c\\\ D&b&0\\\ \end{array}$ (12)

If we discuss the game (12) in a population without community-structure, i.e.
$M=1$, strategy $D$ dominates strategy $C$ with uniform interaction rates
$r_{1}=r_{2}=r_{3}$. Eventually, the entire population consists of defectors.
However, if individuals only interact with opponents of the same strategy,
cooperators cannot be exploited by defectors. In this case, when $r_{2}=0$ and
$r_{1},r_{3}>0$, cooperation is the dominant strategy due to $b-c>0$.
Therefore, $r_{2}\neq 0$ means that cooperators and defectors do interact.

Without loss of generality, we assume that $r_{1}=r_{3}=r>0$, $r_{2}=1$ in a
population with community-structure. Let $x^{l}$ and $y^{l}$ be the
frequencies of individuals adopting strategy $C$ and $D$ in community $l$,
respectively, and $x^{l}+y^{l}=1$. The fitness of individuals are determined
by the average payoffs over a large number of interactions. Therefore, the
fitness of $A$ and $B$ individuals in community $l$ are

$\begin{array}[]{c}f_{C}^{l}=[r(b-c)x^{l}-cy^{l}]/(rx^{l}+y^{l}),~{}~{}~{}~{}f_{D}^{l}=bx^{l}/(x^{l}+ry^{l}),\end{array}$
(13)

In our model, an individual is randomly selected at each time step as a new
vacancy, which is replaced either with the offspring of an individual from the
same community or with an individual migrated from other communities. In
community $l$, the probability of an $X$-individual ($X=C$, or $D$) filling a
new vacancy due to local reproduction is proportional to the product of the
number of $X$-individuals and their fitness, i.e., $N_{X}^{l}f_{X}^{l}$. And,
the probability of an $X$-individual filling in a new vacancy due to global
migration is proportional to the product of the averaged number of
$X$-individuals and the migration rate $\lambda_{l}$, i.e.,
$\lambda_{l}\langle N_{X}\rangle$. Here, we suppose
$\lambda_{lm}=\lambda_{l}=(1-\lambda_{ll})/N_{\Omega_{l}},m\in\Omega_{l}$,
where $N_{\Omega_{l}}$ is the number of set $\Omega_{l}$. Thus, we obtain the
transition probabilities as

$\begin{array}[]{l}T_{C\rightarrow
D}^{l}=(N_{C}^{l}/N)\cdot(N_{D}^{l}f_{D}^{l}+\lambda_{l}\langle
N_{D}\rangle)/[\sum_{C,D}(N_{D}^{l}f_{D}^{l}+\lambda_{l}\langle
N_{D}\rangle)]\\\
~{}~{}~{}~{}~{}~{}~{}~{}~{}=(N_{C}^{l}/N)\cdot[\overline{f}^{l}/(\overline{f}^{l}+\lambda_{l})\cdot(N_{D}^{l}f_{D}^{l})/(N\overline{f}^{l})+\lambda_{l}/(\overline{f}^{l}+\lambda_{l})\cdot\langle
N_{D}\rangle/N]\end{array}$ $\begin{array}[]{l}T_{D\rightarrow
C}^{l}=(N_{C}^{l}/N)\cdot(N_{C}^{l}f_{C}^{l}+\lambda_{l}\langle
N_{C}\rangle)/[\sum_{C,D}(N_{C}^{l}f_{C}^{l}+\lambda_{l}\langle
N_{C}\rangle)]\\\
~{}~{}~{}~{}~{}~{}~{}~{}~{}=(N_{D}^{l}/N)\cdot[\overline{f}^{l}/(\overline{f}^{l}+\lambda_{l})\cdot(N_{C}^{l}f_{C}^{l})/(N\overline{f}^{l})+\lambda_{l}/(\overline{f}^{l}+\lambda_{l})\cdot\langle
N_{C}\rangle/N]\end{array}$ (14)

where
$\overline{f}^{l}=f_{C}^{l}\cdot(N_{C}^{l}/N)+f_{D}^{l}\cdot(N_{D}^{l}/N)$,
and $\langle N_{X}\rangle$=$\sum_{l=1}^{M}N_{X}^{l}/M$. From Eq. (14), we know
that after a vacancy appears, either local reproduction occurs with
probability $\overline{f}^{l}/(\overline{f}^{l}+\lambda_{l})$, or global
migration occurs with probability
$\lambda_{l}/(\overline{f}^{l}+\lambda_{l})$.

For sufficiently large but finite populations, Traulsen et al. have shown that
the stochastic process can be well approximated by a set of stochastic
differential equations combining deterministic dynamics and diffusion referred
to as Langevin dynamics (Traulsen et al., 2005b, ; Traulsen et al., 2006c, ).
Hence from Eq. (14), we may derive the Langevin equation to describe the
evolutionary dynamics as

$\begin{array}[]{l}\dot{x}^{l}=a^{l}(x^{l})+b^{l}(x^{l})\xi\end{array}$ (15)

where $a^{l}=T_{D\rightarrow C}^{l}-T_{C\rightarrow D}^{l}$, $b^{l}$ is the
effective terms, and $\xi$ is the uncorrelated Gaussian noise. As
$N\rightarrow\infty$, the diffusion term tends to zero and a deterministic
equation is obtained

$\begin{array}[]{l}\dot{x}^{l}=a^{l}(x^{l})=(N_{D}^{l}/N)\cdot[\overline{f}^{l}/(\overline{f}^{l}+\lambda_{l})\cdot(N_{C}^{l}f_{C}^{l})/(N\overline{f}^{l})+\lambda_{l}/(\overline{f}^{l}+\lambda_{l})\cdot\langle
N_{C}\rangle/N]\\\
~{}~{}~{}~{}~{}~{}-(N_{C}^{l}/N)\cdot[\overline{f}^{l}/(\overline{f}^{l}+\lambda_{l})\cdot(N_{D}^{l}f_{D}^{l})/(N\overline{f}^{l})+\lambda_{l}/(\overline{f}^{l}+\lambda_{l})\cdot\langle
N_{D}\rangle/N]\end{array}$ (16)

Therefore, the equilibria of Eq. (16) which satisfy $T_{D\rightarrow
C}^{l}=T_{C\rightarrow D}^{l}$ are

$\begin{array}[]{l}[-(\alpha-2\beta)(x^{l})^{2}+(\alpha-2\beta)x^{l}+\beta]/[-(r-1)^{2}(x^{l})^{2}+(r-1)^{2}x^{l}+r]=\gamma/N\end{array}$
(17)

where $\alpha=r^{2}(b-c)-(b+c)$, $\beta=-rc$, and
$\gamma=4(N_{C}^{l}-\overline{N}_{C})M\lambda_{l}$ with
$\overline{N}_{C}=\sum_{i=1}^{M}N_{C}^{i}$.

Denote $\Delta N_{C}=N_{C}^{l}-\overline{N}_{C}$. When $\Delta N_{C}=0$, i.e.,
each community has the same numbers of $C$-individuals, Eq. (17) can be
simplified as

$\begin{array}[]{l}-(\alpha-2\beta)(x^{l})^{2}+(\alpha-2\beta)x^{l}+\beta=0\end{array}$
(18)

When $\alpha>-2\beta$, i.e., $r>(b+c)/(b-c)$, there are two interior
equilibria. Specifically, the stable interior equilibrium is

$\begin{array}[]{c}x^{l*}_{s}=1/2+\sqrt{\alpha^{2}-4\beta^{2}}/(2\alpha-4\beta)\end{array}$

and the unstable interior equilibrium is

$\begin{array}[]{c}x^{l*}_{u}=1/2-\sqrt{\alpha^{2}-4\beta^{2}}/(2\alpha-4\beta)\end{array}$

As $r$ increases, the two interior equilibria move symmetrically away from the
bifurcation point $x^{l*}=1/2$, the interior stable equilibrium moves toward
1, and the unstable equilibrium moves toward 0. Therefore, the proportion of
cooperators tends to increase monotonically with $r$ increases, and
$(b+c)/(b-c)$ is the critical value which specifies whether cooperators are
more abundant than defectors or not. That’s to say, cooperators are more
abundant than defectors when

$r(b-c)-c>b$ (19)

This condition coincides with Eq. (2), for $R=b-c,S=-c,T=b,P=0$, and the
interaction rate $r$ plays the same role as $\sigma$ in section 2.

Since the frequencies of individuals adopting strategy $C$ in each community
become stable at the steady state of the evolutionary dynamics, $\Delta
N_{C}\rightarrow 0$. Hence, when $\Delta N_{C}\neq 0$, we write $\Delta
N_{C}=\Delta N_{C0}\cdot\varepsilon$ for some fixed $\Delta N_{C0}>0$, and Eq.
(17) can be simplified as

$\begin{array}[]{l}[\gamma(r-1)^{2}-N(\alpha-2\beta)](x^{l})^{2}-[\gamma(r-1)^{2}-N(\alpha-2\beta)]x^{l}+(N\beta-\gamma
r)=0\end{array}$ (20)

When $N\alpha+2N\beta>\gamma(r+1)^{2}/N$, i.e.,
$r>(b+c+\gamma/N)/(b-c-\gamma/N)$, we obtain the stable interior equilibrium

$\begin{array}[]{c}x^{l*}_{s}=1/2+1/2\sqrt{[\gamma(r+1)^{2}-N(\alpha+2\beta)]/[\gamma(r-1)^{2}-N(\alpha-2\beta)]}\end{array}$

and the unstable interior equilibrium is

$\begin{array}[]{c}x^{l*}_{u}=1/2-1/2\sqrt{[\gamma(r+1)^{2}-N(\alpha+2\beta)]/[\gamma(r-1)^{2}-N(\alpha-2\beta)]}\end{array}$

This conclusion is similar to the case of $\Delta N_{C}=0$, that is, the two
interior equilibria move symmetrically away from the bifurcation point
$x^{l*}=1/2$ and the proportion of cooperators tends to increase monotonically
with $r$ increases. Thus, $(b+c+\gamma/N)/(b-c-\gamma/N)$ is the critical
value which specifies whether cooperators are more abundant than defectors or
not. With simplification, we have the condition that cooperators are more
abundant than defectors

$r(b-c)-c>b+(\gamma/N)(r+1)$ (21)

When $r>1$, a small $\varepsilon>0$ leads to $\gamma\rightarrow 0$. Eq. (21)
is discriminated from Eq. (19) only with the infinitesimal term, and the
second term in the r.h.s of Eq. (21) is negligible in the meaning of
mathematic. Therefore, Eq. (21) also coincides with Eq. (2), for
$R=b-c,S=-c,T=b,P=0$.

To sum up, Eq. (19) and Eq. (21) reflect that the interaction rate $r$ plays
the similar role as $\sigma$ in section 2, and we verify that $\sigma$
features the reaction rate between two individuals with the same strategy.

##  4 Conclusion and Discussion

In this paper, we have proposed a model of games on a community-structured
population to understand the $\sigma$ law with the Moran process and the
ergodic theory. By calculating the average payoffs of $A$ and $B$ individuals
with the effective sojourn time method, we find that, $\sigma$ features not
only the structured populations characteristics, but also the reaction rate
between individuals. Parameter $\sigma$, reflects that individuals using the
same strategy may interact with themselves $\sigma$ times than the
interactions with individuals of the other strategy. That’s to say, an
interaction between two individuals are not uniform with the verification in a
simplified PDG.

In practice, an overlap of communities usually exists in social networks for
example, yet the case with overlap doesn’t be discussed in this paper. Here we
simplify an overlapped community-structured population to a none-overlap
community-structured population. For a simple example without loss of
generality, suppose that a game is played in a finite population with the
fixed size $H$ distributed over $2$ communities, which have one common
individual, as shown in Fig. 3(a). In this case, the overlap individual can be
regarded as two parts, itself and a virtual reproduction which exist in the
two corresponding separated communities, respectively. Therefore, the
overlapped community-structured population with size $H$ becomes a none-
overlap one with size $H+1$ (see Fig. 3(b)), which is discussed in our paper.
Hence, we provide a simplified yet alternative approach to study the
evolutionary games in a community-structured population with overlap.

## Acknowledgment

We were grateful to Lin Wang and Yu Wang for their helpful discussions and
valuable suggestions, and the anonymous reviewers for their constructive
comments to help improve this paper. This work is supported partly by NSFC
(Grant No. 60874089), the National Key Basic Research and Development Program
(Grant No. 2010CB731403), and the NCET program of China (Grant No. 09-0317).

## References

## References

  * Alos-Ferrer, (2003) Alos-Ferrer, C. 2003. Finite population dynamics and mixed equilibria. Int. Game Theory Review. 5, 263-290.
  * (2) Antal, T., Traulsen, A., Ohtsuki, H., Tarnita, C.E., Nowak, M.A., 2009a. Mutation-selection equilibrium in games with multiple strategies. J. Theor. Biol. 258, 614-622.
  * (3) Antal, T., Ohtsuki, H., Wakeley, J., Taylor, P.D., Nowak, M.A., 2009b. Evolution of cooperation by phenotypic similarity. Proc. Natl. Acad. Sci. USA. 106, 8597-8600.
  * Bomze and Pawlowitsch, (2008) Bomze, I., Pawlowitsch, C., 2008. One-third rules with equality: second-order evolutionary stability conditions in finite populations. J. Theor. Biol. 254, 616-620.
  * Cao and Li, (2008) Cao. L., Li. X., 2008. Mixed evolutionary strategies imply coexisting opinions on networks. Phys. Rev. E. 77, 016108.
  * Cressman, (2003) Cressman, R., 2003. Evolutionary Dynamics and Extensive Form Games. MIT Press, Cambridge.
  * Ewens, (2004) Ewens, W. J., 2004. Mathematical Population Genetics, vol. 1. Theoretical Introduction. Springer, New York.
  * Ficici et al., (2000) Ficici, S. G., Pollack, J. B., 2000. Effects of finite populations on evolutionary stable strategies. In: Whitley, D.(ED.), Proceedings of the 2000 Genetic and Evolutionary Computation Conference, San Francisco, CA. Morgan-Kaufmann; 2000, 927-934.
  * Fogel et al., (1998) Fogel, G., Andrews, P., Fogel, D., 1998. On the instability of evolutionary stable strategies in small populations. Ecol, Model. 109, 283-294.
  * Fu et al., (2009) Fu, F., Wang, L., Nowak ,M. A., Hauert, C., 2009. Evolutionary dynamics on graphs: efficient method for weak selection. Phys. Rev. E. 79, 046707.
  * Gintis, (2000) Gintis, H., 2000. Game Theory Evolving. Princeton University Press, Princeton, USA.
  * Girvan and Newman, (2002) Girvan, M. and Newman, M. E., 2002. Community structure in social and biological networks. J., Proc. Natl. Acad. Sci. U.S.A., 99, 7821-7826.
  * Godsil and Royle, (2001) Godsil, C. and Royle, G., 2001. Algebraic Graph Theory. vol. 207: Springer-Verlag.
  * Gokhale and Traulsen, (2011) Gokhale, C.S., Traulsen, A., 2011. Strategy abundance in evolutionary many-player games with multiple strategies J. Theor. Biol. 238, 180-191.
  * Hauert and Doebeli, (2004) Hauert, C., Doebeli, M., 2004. Spatial structure often inhibits the evolution of cooperation in the snowdrift game. Nature 428, 643-646.
  * Hauert and Imhof, (2011) Hauert, C., Imhof, L.A, 2011. Evolutionary games in deme structured, finite populations. J. Theor. Biol. DOI:10.1016/j.jtbi.2011.06.010.
  * Hofbauer and Sigmund, (1988) Hofbauer, J., Sigmund, K., 1988. The Theory of Evolution and Dynamical Systems (Cambridge Univ Press, Cambridge, UK).
  * Hofbauer and Sigmund, (1998) Hofbauer, J., Sigmund, K., 1998. Evolutionary Games and Population Dynamics. Cambridge University Press, Cambridge, UK.
  * Hofbauer and Sigmund, (2003) Hofbauer, J., Sigmund, K., 2003. Evolutionary game dynamics. Bull. Am. Math. Soc. 40, 479-519.
  * Imhof and Nowak, (2006) Imhof, L. A, Nowak, M. A., 2006. Evolutionary game dynamics in a Wright-Fisher process. J. Math. Biol. 52, 667-681.
  * Lessard and Ladret, (2007) Lessard, S., Ladret, V., 2007. The probability of a single mutant in an exchangeable selection model. J. Math. Biol. 54, 721-744.
  * Lieberman et al., (2005) Lieberman, E., Hauert, C., Nowak, M.A., 2005. Evolutionary dynamics on graphs. Nature 433, 312-316.
  * Li and Cao, (2009) Li. X., Cao. L., 2009. Largest Laplacian eigenvalue predicts the emergence of costly punishment in the evolutionary ultimatum game on networks. Phys. Rev. E. 80, 066101.
  * Maynard and Price, (1973) Maynard, Smith. J., Price, G. R.,1973. The logic of animal conflict. Nature 246, 15-18.
  * Maynard, (1982) Maynard, Smith. J., 1982. Evolution and the Theory of Games (Cambridge Univ Press, Cambridge, UK).
  * Nakamaru et al., (1997) Nakamaru, M., Matsuda, H,, Iwasa, Y., 1997. The evolution of cooperation in a lattice-structured population. J. Theor. Biol. 184, 65-81.
  * Nathanson et al., (2009) Nathanson, C.G., Tarnita, C.E., Nowak, M.A., 2009. Calculating Evolutionary Dynamics in Structured Populations. PLoS Computational Biology. Vol5(12), e1000615.
  * Newman, (2006) Newman M. E. J., 2006. Modularity and community structure in networks. Proc. Natl. Acad. Sci. U.S.A., 103, 8577-8582.
  * Nowak and May, (1992) Nowak, M. A., May, R. M., 1992. Evolutionary games and spatial chaos. Nature 359(6398), 826-829.
  * (30) Nowak, M. A., Sigmund, K., 2004a. Evolutionary dynamics of biological games. Science 303,793-799.
  * (31) Nowak, M. A., Sasaki, A., Taylor, C., Fudenberg, D., 2004b. Emergence of cooperation and evolutionary stability in finite populations. Nature 428, 646-650.
  * Nowak, (2006) Nowak, M.A., 2006. Evolutionary Dynamics. Harvard University Press, MA.
  * Nowak et al., (2010) Nowak, M.A., Tarnita, C.E., Antal, T., 2010. Evolutionary dynamics in structured populations. Phil. Trans. R. Soc. B, 365, 19-30.
  * (34) Ohtsuki, H., Hauert, C., Lieberman, E., Nowak, M.A., 2006a. A simple rule for the evolution of cooperation on graphs and social networks. Nature 441, 502-505.
  * (35) Ohtsuki, H., Nowak, M.A., 2006b. Evolutionary games on cycles. Proc. R. Soc B. 273, 2249-2256.
  * (36) Ohtsuki, H., Nowak, M.A., Pacheco, J.M., 2007a. Breaking the symmetry between interaction and replacement in evolutionary dynamics on graphs. Phys Rev Lett. 98, 108106.
  * (37) Ohtsukia H., Bordalob, P., Nowak, M.A., 2007b. The one-third law of evolutionary dynamics. J. Theor. Biol. 249, 289-295.
  * Perc, (2007) Perc, M., 2007. Transition from Gaussian to levy distributions of stochastic payoff variations in the spatial prisoner s dilemma game. Phys. Rev. E. 75, 022101.
  * Perc and Wang, (2010) Perc. M., Wang. Z., 2010. Heterogeneous Aspirations Promote Cooperation in the Prisoner s Dilemma Game. PLoS ONE 5(12), e15117.
  * Schaffer, (1988) Schaffer, M. 1988. Evolutionarily stable strategies for a finite population and variable contest size. J. Theor. Biol. 132, 469-478.
  * Szabo and Fath, (1998) Szabó, G., Töke, C., 1998. Evolutionary prisoner’s dilemma game on a square lattice. Phys. Rev. E. 58, 69-73.
  * Szabo et al., (2000) Szabó, G., Antal, T., Szabó, P., Droz, M., 2000. Spatial evolutionary prisoner’s dilemma game with three strategies and external constraints. Phys. Rev. E. 62, 1095-103.
  * Taylor et al., (2004) Taylor, C., Fudenberg, D., Sasaki, A., Nowak, M.A., 2004. Evolutionary game dynamics in finite populations. B. Math. Biol. 66, 1621-1644.
  * Taylor and Nowak, (2006) Taylor. C., Nowak, M.A., 2006. Evolutionary game dynamics with non-uniform interaction rates. Theor. Popul. Biol. 69, 243-252.
  * Taylor et al., (2007) Taylor, P.D., Day, T., Wild, G., 2007. Evolution of cooperation in a finite homogeneous graph. Nature 447, 469-472.
  * (46) Tarnita, C.E., Antal, T., Ohtsuki, H., Nowak, M.A., 2009a. Evolutionary dynamics in set structured populations. Proc. Natl. Acad. Sci. USA. 106, 8601-8604.
  * (47) Tarnita, C. E., Ohtsuki, H., Antal, T., Fu, F., Nowak, M. A., 2009b. Strategy selection in structured populations. J. Theor. Biol. 259, 570-581.
  * Tarnita et al., (2011) Tarnita, C. E., Nicholas W., Nowak, M. A., 2011. Multiple strategies in structured populations. Proc. Natl. Acad. Sci. USA. 108(6), 2334-2337.
  * (49) Traulsen, A., Sengupta, A. M., Nowak, M. A., 2005a. Stochastic evolutionary dynamics on two levels. J. Theor. Biol. 235, 393-401.
  * (50) Traulsen, A., Claussen, J. C. and Hauert, C., 2005b. Coevolutionary Dynamics: From Finite to Infinite Populations. Phys. Rev. Lett. 95, 238701.
  * (51) Traulsen, A., Pacheco, J.M., Imhof, L., 2006a. Stochasticity and evolutionary stability. Phys. Rev. E. 74, 021905.
  * (52) Traulsen, A., and Nowak, M. A., 2006b. Evolution of cooperation by multilevel selection. Proc. Natl. Acad. Sci. U.S.A., 103, 10952-10955.
  * (53) Traulsen, A., Claussen, J. C. and Hauert, C., 2006c. Coevolutionary dynamics in large, but finite populations. Phys. Rev. E 74, 011901.
  * Traulsen and Nowak, (2007) Traulsen, A., Nowak, M. A., 2007. Chromodynamics of cooperation in finite populations. PLoS ONE 2, e270.
  * Traulsen et al., (2008) Traulsen, A., Shoresh, N. and Nowak, M. A., 2008. Analytical results for individual and group selection of any intensity. Bull. Math. Biol., 70, 1410 C1424.
  * Wu et al., (2010) Wu, B., Zhou, D., Fu, F., Luo, Q. j., Wang, L., Traulsen, A., 2010. Evolution of cooperation on stochastic dynamical networks. PLoS ONE 5(6), e11187.
  * Wu et al., (2011) Wu, B., Gokhale, C.S, Wang, L., Traulsen, A., 2011. How small are mutation rates? J.Math.Biol. DOI 10.1007/s00285-011-0430-8.
  * Wang et al., (2011) Wang, Z., Murks, A., Du, W.B., Rong, Z.H., Perc, M. 2011. Coveting thy neighbors fitness as a means to resolve social dilemmas. J. Theor. Biol. 277, 19-26.
  * Wang et al., (2012) Wang, Z., Szolnoki, A. and Perc, M. 2012. Evolution of public cooperation on interdependent networks: The impact of biased utility functions. EPL, 97, 48001.
  * Wang et al., (2011) Wang, J., Wu, B., Daniel, W. C. Ho., Wang, L., 2011. Evolution of cooperation in multilevel public goods games with community structures. Eur. Phy. L., 93, 58001.

---

Figure 1.

An illustrative community-structured population with strategies $A$ and $B$.
At each time step, a single individual from the entire population is selected
proportional to its fitness for reproduction, and the offspring either stays
in this community or migrates to another community. There exists a graph
connecting all the communities in the structured population. The migration
occurs between two communities if there is an edge connecting them both.

Figure 2.

An illustrative fixation process of a single mutant in the whole community-
structured population. (a) A single mutant is produced in community $1$ (white
node); (b) This mutant successfully takes over community $1$ with probability
$\lambda_{11}$, and then migrates to community $2$ to replace a randomly
chosen individual with probability $\lambda_{12}$; (c) The mutant takes over
community $2$ with probability $\lambda_{22}$, and then migrates to community
$3$ to replace a randomly chosen individual with probability $\lambda_{23}$;
(d) The mutant takes over community $4$ with probability $\lambda_{44}$, and
then migrates to community $5$ to replace a randomly chosen individual with
probability $\lambda_{45}$.

---

Figure 3.

(a) An overlapped community-structured population with one common individual;
(b) A population with two none-common communities.