The results are shown in \[fig: marl\].

-   Case 1: discount $\gamma = 0$ with runtime cost $l_{i,k}$.

    With $\gamma = 0$, this case reduces to a single stage game as discussed in the *distributed optimal control* section. The result should align with the analytical Nash Equilibrium in . As shown in the left plot in \[fig: marl\](a), the optimal action for a healthy agent is always *low* (solid green), while the optimal action for an infected agent is always *high* (dashed magenta). The Q values for infected agents do not depend on $m_k$. The Q values for healthy agents increase when $m_k$ increases if the activity level is not zero, due to the fact that: for a fixed activity level, the chance to get infected is higher when there are more infected agents in the system. All these results align with our previous theoretical analysis. Moreover, as shown in the right plot in \[fig: marl\](a), the agents are learning to flatten the curve across different episodes.

-   Case 2: discount $\gamma = 0.5$ with runtime cost $l_{i,k}$.

    Since the agents are now computing cumulative costs as in , the corresponding Q values are higher than those in case 1. 