use $\widehat{J}\left(\theta_{n}\right)$ rather than $J\left(\theta_{n}\right)$ in  to indicate that the objective function for the actor updating is defined based on the $Q$-value estimation.

Since the critic updating results in a closed-form solution , we could substitute the expression  into the objective for the actor updating . Such case, however, leads to a very complex optimization problem. In the case of large feature space, one can recursively update $\mhbfw_{\theta}$ and $\widehat{\theta}_{n}$ to reduce the computational cost.

Network Cohesion based online Actor-Critic RL
=============================================

It is a famous phenomenon observed in lots of social behavior studies [@Haynie_2001_AJOS_NetworkStructure; @Fujimoto_2012_SocialScience_NetworkInfluence] that people are widely connected in a network and linked users tend to have similar behaviors. Advances in social media help a lot to record the relational information among users, which ensures the availability of network information for health-related studies. Besides, individuals are widely connected due to the similar features, such as age, gender, race, religion, education level, work, income, other socioeconomic status, medical records and genetics features etc [@Tianxi_2016_CORR_PredictModels4NetworkLinkedData]. However, for simple study, current online methods for the mHealth simply assume that users are completely different; they share no information among users and learn a separate RL for each user by only using 