propose a cohesion-based online RL method for the mHealth study. We aim to understand how to share information across similar users in order to improve the performance.

Construct the network cohesion by using the warm start trajectory (WST) {#sub:ConstructCohesionNetwork}
-----------------------------------------------------------------------

We assume there is an undirected network cohesion connecting similar users, i.e. $\mtcalG=\left(\mtcalV,\mtcalE\right)$, where $\mtcalV=\left\{ 1,2,\cdots,N\right\} $ is the set of nodes (representing users) and $\mtcalE\subset\mtcalV\times\mtcalV$ is the edge set. Altough advanced social medias, like Facebook, Twitter and Linkedin, could provide us ith various network information, they are not designed for the mHealth. There is noisy and misleading relational information in the network for mHealth WS[@Bianchi_2013_NIPS_GangOfBandit; @guangliangCheng_2016_JStars_robustHyperClassification; @Alexandra_2016_AISTATS_GraphBandits; @Claudio_2014_ICML_olineClusteringBandits; @guangliangCheng_2016_TGRSL; @fyzhu_2014_TIP_DgS_NMF; @guangliangCheng_2014_ICIP; @fyzhu_2014_AAAI_ARSS; @guangliangCheng_2015_ICIP; @yingWang_2015_TIP_RobustUnmixing; @xiaoping_2017_ICASSP; @guangliangCheng_2016_neurocomputing; @fyzhu_2014_JSTSP_RRLbS]. Thus, we want to learn the network cohesion intentionally for the mHealth by measuring the similarities between the related behaviors of users.

In RL, the MDP provides a mathematical tool to describe the property of users in a specific mHealth study[^3]. By measuring the similarities among the users’ MDPs , we could learn the network cohesion targeted to that mHealth study. However, the MDP models are unknown to the RL problem. Instead, the warm start trajectories (WSTs) of all the $N$ users 