------------------- -------------------------------------- -------------------------
      1              1                  0                   1+$\alpha \exp(-1)$              1+$\alpha \exp(-1)$
    2,3,4            0                  1                            0                                0
    Total                                                                                   $1+ \alpha \exp(-1)$

  : List of the agent decisions and associated costs in the Nash Equilibrium with shaped cost functions in the four-agent example.

\[fig: example cost improved\]

Multi-Agent Reinforcement Learning
==================================

We have shown how to compute the Nash Equilibrium of the multi-agent epidemic model in a single stage. However, it is analytically intractable to compute the Nash Equilibrium when we consider repeated games . The complexity will further grow when the number of agents increases and when there are information asymmetry. Nonetheless, we can apply multi-agent reinforcement learning[@bucsoniu2010multi] to numerically compute the Nash Equilibrium. Then the evolution of the pandemic can be predicted by simulating the system under the Nash Equilibrium.

Q Learning
----------

As evident from , the optimal action for agent $i$ at time $k$ is a function of $x_{i,k}$ and $m_k$. Hence we can define a Q function (action value function) for agent $i$ as $$Q_i: x_{i,k}\times m_k\times u_{i,k}\mapsto \mathbb{R}.$$ According to the assumptions made in the observation model, all agents can observe $m_k$ at time $k$. For a single stage game, we have derived in that $Q_i (x,m,u)= 