One example of deterministic two-action cooperative games is shown in Fig.1(a), in which there exist two optimal Nash equilibria. In this game, agents are desired to coordinate their actions towards a consistent Nash equilibrium to maximize their benefits.

Previous works [@panait2006lenient; @matignon2012independent; @hao2013dynamics; @hao2014reinforcement; @hao2015multiagent] have investigated the problem of how a population of agents could achieve coordination under the same cooperative environments (games) to maximize the system’s overall benefits. However, in general, the payoff matrix between any pair of agents may be different depending on the agents’ interacting environments. Following the setting in [@damer2008achieving], we assume that each game is drawn independently from certain probability distribution to model agents’ interaction environments in a generalized way. Without loss of generality, a general form of 2-action cooperative game $G_i^j$ between agent $i$ and $j$ is shown in Fig.1(b), where the value of $u_a$ (or $u_b$) is sampled from a stochastic variable $x_a$ (or $x_b$) following certain cumulative probability distribution $F_a(x)$ (or $F_b(x)$), and $\alpha < x_a (x_b)$ represents the payoff under mis-coordination. The payoff matrix of any pair of agents is not known as a prior, which can be learned through repeated interactions. In our settings, we assume that each agent 