his or her data. Such assumption works well in the ideal condition where the sample drawn from each user is large in size to support the separate online learning. However, though the data for all users is abundant, the data for each user is limited in size. For example at the beginning of online learning, there are $t=5$ tuples, which is hardly enough to support a separate learning and likely to result in unstable policies. From the perspective of optimization, the problem of lack of samples badly affects the actor-critic updating not only at the beginning of online learning but also along the whole learning process. This is because the actor-critic objective functions are non-convex and nonlinear; the bad solution at the beginning of online learning would bias the optimization to sub-optimal directions. Besides, the policy achieved at the early stage of online learning is of bad user experience, which is likely for the users to be inactive with or even to abandon the mHealth.

Different from current methods, we consider the phenomenon that a user is similar to some (but not all) users, and similar users behave similar but not completely identical to each other. To this end, we 