1-(1-u)^{m}(1-x) + \alpha_i p(u)$. For repeated games , we can learn the Q function using temporal different learning. At every time $k$, agent $i$ chooses its action as $$\label{eq: optimal action}
u_{i,k} = \arg\min_u Q_i(x_{i,k}, m_k, u).$$ After taking the action $u_{i,k}$, agent $i$ observes $x_{i,k+1}$ and $m_{k+1}$ and receives the cost $l_{i,k}$ at time $k+1$. Then agent $i$ updates its Q function: $$\begin{aligned}
&Q_i (x_{i,k},m_k,u_{i,k}) \leftarrow Q_i (x_{i,k},m_k,u_{i,k}) + \eta \delta_{i,k},\\
&\delta_{i,k} = l_{i,k} + \gamma \min_u Q_i(x_{i,k+1},m_{k+1},u) - Q_i(x_{i,k},m_k,u_{i,k}),\end{aligned}$$ where $\eta$ is the learning gain and $\delta_{i,k}$ is the temporal difference error.

All agents can run the above algorithm to learn their $Q$ functions during the interaction with others. However, the algorithm introduced above has several problems:

-   Exploration and limited rationality.

    There is no exploration in . Indeed, Q-learning is usually applied together with $\epsilon$-greedy where with probability $1-\epsilon$, the action $u_{i,k}$ is chosen to be the optimal action in , and with probability $\epsilon$, the action is randomly chosen with a uniform distribution over the action space. The $\epsilon$-greedy approach is introduced mainly from an algorithmic perspective to improve convergence of the learning process. When applied to the epidemic model, it has a unique societal implication. 