are available, which provide the observation of users. Thus, we use the WSTs for the graph learning, i.e. $\mtcalD^{\left(0\right)}=\left\{ \mtcalD_{n}^{\left(0\right)}\mid n=1,\cdots,N\right\} $, where $\mtcalD_{n}^{\left(0\right)}=\left\{ \left(s_{i,n},a_{i,n,}r_{i,n}\right)\right\} _{i=1}^{T_{0}}$ is the WST for the $n$-th user. Since an MDP consists of the state transistion and immediate reward model, the feature for the cohesion network learning is constructed by stacking the states and rewards in the WST as follows $$\mtbfv_{n}=\left[s_{1,n}^{\intercal},r_{1,n},\cdots,s_{T_{0},n}^{\intercal},r_{T_{0},n}\right]^{\intercal}\in\mtbbR^{pT_{0}+T_{0}},\label{eq:feature_4_GraphLearning}$$ for $n\in\left\{ 1,\cdots,N\right\} $. Note that the action or policy is not part of an MDP. To reduce the influence of random actions in the WST, we get rid of the temporal order by sorting all the elements in $\mtbfv_{n}$ . Then the benchmark method, i.e. $K$-nearest neighbor ($K$NN), is used to learn the neighboring information among users $$c_{ij}=\begin{cases}
1, & \mtif\ \mtbfv_{i}\in\mtcalN\left(\mtbfv_{j}\right)\ \text{or}\ \mtbfv_{j}\in\mtcalN\left(\mtbfv_{i}\right)\\
0, & \text{otherwise}
\end{cases},\label{eq:KNN_4_GraphLearning}$$ where $\mtbfv_{i}\in\mtcalN\left(\mtbfv_{j}\right)$ indicates that $i$-th node is the $K$NN of the $j$-th node [@Luxburg_2007_SC_SpectralClustering];  is an undirected Graph. The value of $K$ controls how widely the users are connected. A large $K$ indicates a wide connection among users and vice versa.

Model of cohesion based Actor-Critic RL \[sub:Objective\_4\_GraphRL\]
---------------------------------------------------------------------

The underlying assumption throughout this paper is that if two users are connected, their values and 