each user. However, there are too few of samples to support the separate online learning, which leads to unsatisfactory interventions (policies) for the users. (**3**) Our method doesn’t require the given network cohesion. We propose a method to learn the network intentionally for the mHealth study. It makes use of the warm start trajectories in the online learning, which are expected to represent the users’ properties. (**4**) Compared with [@Bianchi_2013_NIPS_GangOfBandit], the proposed method has a tuning parameter, which allows us to control how much information we should share with similar users. It is worth mentioning that our method may not be limited to mHealth. It can be applied to other health-related tasks. Extensive experiment results on the HeartSteps dataset verifies that our method can achieve clear improvement over the Separate-RL.

Preliminaries
=============

Markov Decision Process (MDP)
-----------------------------

We assume the mHealth intervention is a Markov Decision Process (MDP) [@Geist_2013_TNNLS_RL_valueFunctionApproximation; @Grondman_2012_IEEEts_surveyOfActorCriticRL; @Pednault_2002_SIGKDD_CostSensitiveRL; @Michail_2003_JMLR_LSPI_LSTDQ] that consists of a 5-tuple $\left\{ \mtcalS,\mtcalA,\mtcalP,\mtcalR,\gamma\right\} $, where $\mtcalS$ is the state space and $\mtcalA$ is the action space. $\mtcalP\!\!:\!\mtcalS\!\times\!\mtcalA\!\times\!\mtcalS\!\mapsto\!\left[0,1\right]$ is the state transition model in which $\mtcalP\left(s,a,s'\right)$ indicates the probability of transiting from one state $s$ to another $s'$ after taking action $a$; $\mtcalR\left(s,a,s'\right)$ is the 