of $\mtbfA\otimes\mtbfB$ [@Langville_2004_JCAM_KroneckerProduct], where $\otimes$ is the Kronecker Product, are $$\lambda_{i}\nu_{j},\qquad\mtfor\ i\in\left\{ 1,\cdots,n\right\} ,\ j\in\left\{ 1,\cdots,m\right\} .$$

  -------- ----------------- ---------------- --------------------- ----------------- ---------------- ---------------------
                                                                                                       
              Separate-RL     Cohesion-RL\#1     Cohesion-RL\#2        Separate-RL     Cohesion-RL\#1     Cohesion-RL\#2
    $0$     1238.4$\pm$81.6                    **1394.4$\pm$68.5**   1239.5$\pm$83.9                    **1397.4$\pm$68.6**
   $0.2$    1272.9$\pm$83.7                    **1428.1$\pm$64.1**   1279.5$\pm$83.4                    **1429.2$\pm$64.3**
   $0.4$    1286.7$\pm$85.3                    **1472.4$\pm$58.7**   1316.4$\pm$77.5                    **1472.4$\pm$59.0**
   $0.6$    1346.3$\pm$75.9                    **1505.1$\pm$55.4**   1388.5$\pm$70.8                    **1515.3$\pm$55.9**
   $0.8$    1373.9$\pm$66.2                    **1556.7$\pm$54.0**   1440.7$\pm$63.5                    **1570.0$\pm$54.0**
   $0.95$                                      **1556.5$\pm$49.6**   1315.2$\pm$68.6                    **1577.8$\pm$51.3**
    Avg.        1297.2                             **1485.5**            1330.0                             **1493.7**
  -------- ----------------- ---------------- --------------------- ----------------- ---------------- ---------------------

  -------- ----------------- ---------------- --------------------- ----------------- ----------------- ---------------------
                                                                                                        
              Separate-RL     Cohesion-RL\#1     Cohesion-RL\#2        Separate-RL     Cohesion-RL\#1      Cohesion-RL\#2
    $0$     1194.0$\pm$86.8                    **1396.7$\pm$68.5**                     1358.9$\pm$63.6   **1399.1$\pm$68.0**
   $0.2$    1183.8$\pm$87.8                    **1427.4$\pm$64.5**                     1408.0$\pm$57.8   **1430.2$\pm$63.6**
   $0.4$    1200.4$\pm$81.1                    **1469.2$\pm$58.2**   1423.7$\pm$63.9                     **1471.3$\pm$59.3**
   $0.6$    1254.1$\pm$75.7                    **1515.2$\pm$55.0**   1463.6$\pm$58.1                     **1516.8$\pm$56.1**
   $0.8$    1291.8$\pm$76.8                    **1562.3$\pm$53.8**   1519.8$\pm$53.9                     **1552.9$\pm$54.0**
   $0.95$                                      **1568.5$\pm$51.0**   1434.4$\pm$53.3                     **1574.6$\pm$50.0**
    Avg.        1228.2                             **1489.9**            1438.7                              **1490.8**
  -------- ----------------- ---------------- --------------------- ----------------- ----------------- ---------------------

*The value of $\gamma$ specifies different RL methods: (a) $\gamma=0$ means the contextual bandit [@huitian_2014_NIPS_ActCriticBandit4JITAI]; (b) $0<\gamma<1$ is the discounted reward RL, which is first compared in the online actor-critic setting for mHealth. In each comparision, the* ***bold value*** *is the best, and the* ** *is the 2nd best.*

Algorithm\#2 for the Critic update
==================================

In this section, we provide another updating rule for the critic update (i.e. policy improvement). Note that to prevent the 