KGTrust: Evaluating Trustworthiness of SIoT via Knowledge Enhanced Graph
  Neural Networks
Social Internet of Things (SIoT), a promising and emerging paradigm that
injects the notion of social networking into smart objects (i.e., things),
paving the way for the next generation of Internet of Things. However, due to
the risks and uncertainty, a crucial and urgent problem to be settled is
establishing reliable relationships within SIoT, that is, trust evaluation.
Graph neural networks for trust evaluation typically adopt a straightforward
way such as one-hot or node2vec to comprehend node characteristics, which
ignores the valuable semantic knowledge attached to nodes. Moreover, the
underlying structure of SIoT is usually complex, including both the
heterogeneous graph structure and pairwise trust relationships, which renders
hard to preserve the properties of SIoT trust during information propagation.
To address these aforementioned problems, we propose a novel knowledge-enhanced
graph neural network (KGTrust) for better trust evaluation in SIoT.
Specifically, we first extract useful knowledge from users' comment behaviors
and external structured triples related to object descriptions, in order to
gain a deeper insight into the semantics of users and objects. Furthermore, we
introduce a discriminative convolutional layer that utilizes heterogeneous
graph structure, node semantics, and augmented trust relationships to learn
node embeddings from the perspective of a user as a trustor or a trustee,
effectively capturing multi-aspect properties of SIoT trust during information
propagation. Finally, a trust prediction layer is developed to estimate the
trust relationships between pairwise nodes. Extensive experiments on three
public datasets illustrate the superior performance of KGTrust over
state-of-the-art methods.
Zhizhi Yu, Di Jin, Cuiying Huo, Zhiqiang Wang, Xiulong Liu, Heng Qi, Jia Wu, Lingfei Wu
# KGTrust: Evaluating Trustworthiness of SloT via Knowledge Enhanced Graph Neural Networks

###### Abstract.

Social Internet of Things (SIoT), a promising and emerging paradigm that injects the notion of social networking into smart objects (i.e., things), paving the way for the next generation of Internet of Things. However, due to the risks and uncertainty, a crucial and urgent problem to be settled is establishing reliable relationships within SIoT, that is, trust evaluation. Graph neural networks for trust evaluation typically adopt a straightforward way such as one-hot or node2vec to comprehend node characteristics, which ignores the valuable semantic knowledge attached to nodes. Moreover, the underlying structure of SIoT is usually complex, including both the heterogeneous graph structure and pairwise trust relationships, which renders hard to preserve the properties of SIoT trust during information propagation. To address these aforementioned problems, we propose a novel knowledge-enhanced graph neural network (KGTrust) for better trust evaluation in SIoT. Specifically, we first extract useful knowledge from users' comment behaviors and external structured triples related to object descriptions, in order to gain a deeper insight into the semantics of users and objects. Furthermore, we introduce a discriminative convolutional layer that utilizes heterogeneous graph structure, node semantics, and augmented trust relationships to learn node embeddings from the perspective of a user as a trustor or a trustee, effectively capturing multi-aspect properties of SIoT trust during information propagation. Finally, a trust prediction layer is developed to estimate the trust relationships between pairwise nodes. Extensive experiments on three public datasets illustrate the superior performance of KGTrust over state-of-the-art methods.

Graph Neural Networks, Trust Evaluation, Social Internet of Things +
Footnote †: journal: Computer graphics & neural networks

+
Footnote †: journal: Computer graphics & neural networks
straightforward ways such as one-hot or node2vec. More importantly, the observed trust relationships in real-world are often very sparse (Koren et al., 2017), which makes it difficult for these methods to fully model the multi-aspect properties of SIoT trust during information propagation, thus negatively influencing the prediction of trust relationships.

So an interesting yet important question is how to effectively design a GNN-based method for more accurate trust evaluation within SIoT. Particularly, two challenges need to be addressed. First, the rich node semantics within SIoT should be taken into consideration. As nodes in SIoT are generally associated with textual information such as comments or descriptions, it is crucial to deeply mine and encode these useful data to essentially embody the inherent characteristics of nodes. Second, multi-aspect properties of SIoT trust should be effectively preserved. The underlying structure of SIoT is usually complex, which contains not only the heterogeneous graph structure but also the pairwise trust relationships. Therefore, a reliable GNN-based method ought to make allowance for flexibly preserving the properties of SIoT trust (including asymmetric, propagative, and composable nature) when information propagates along the heterogeneous graph structure, especially when the observed trust connections are very sparse.

In light of the aforementioned challenges, we propose KGTrust, a knowledge enhanced graph neural network model for trust evaluation in SIoT. Specifically, we first design an embedding layer to fully model the semantics of users and objects by extracting useful and relevant knowledge from users' comment behaviors and external structured triples, respectively. We then introduce a personalized PageRank-based neighbor sampling strategy to augment the trust structure, alleviating the sparsity of user-specific trust relationships. After that, we employ a discriminative convolutional mechanism to learn node embeddings from the perspective of a user as a trustor or a trustee, and adaptively integrate them with a learnable gating mechanism. In this way, multi-aspect properties of SIoT trust, including asymmetric, propagative and composable nature, can be effectively preserved. Finally, the learned embeddings for pairwise users are concatenated and fed into a prediction layer to estimate their trust relationships.

We summarize our main contributions as follows:

* To the best of our knowledge, we are the first to gain a deeper insight into the trust evaluation within SIoT via jointly considering three key ingredients, that is, heterogeneous graph structure, node semantics and associated trust relationships.
* We present a novel knowledge enhanced graph neural network, named KGTrust, which innovatively mines and models the intrinsic characteristics of users and objects with the guidance of external knowledge and multi-aspect trust properties, for assessing trustworthiness in SIoT.
* Extensive experiments across three public datasets demonstrate the superior performance of the new approach KGTrust over state-of-the-art baselines.

## 2. Preliminaries

We first give the notations and problem definition, then introduce properties of SIoT trust, and finally discuss graph neural networks as the base of our proposed KGTrust.

### Notations and Problem Definition

**Definition 1. Social Trust Internet of Things.** A Social Trust Internet of Things, defined as \(G=(V,E,\mathcal{A},\mathcal{R},\psi,\varphi)\), is a form of heterogeneous directed network, where \(V=\{v_{1},\ldots,v_{n}\}\) and \(E=\{e_{ij}\}\subseteq V\times V\) represent the sets of nodes and edges, respectively. It is also associated with a node type mapping function \(\psi:V\rightarrow\mathcal{A}\), and an edge type mapping function \(\varphi:E\rightarrow\mathcal{R}\), where \(\mathcal{A}\in\{\text{user, object}\}\) and \(\mathcal{R}\in\{\text{user, user}\}\), \(\langle\text{user, object}\rangle\), \(\langle\text{object, user}\rangle\), \(\langle\text{object}\rangle\), \(\langle\text{object}\rangle\}\) denote the sets of node and edge types. All edges formulate an original adjacency matrix \(\mathbf{A}=(a_{ij})_{m\times n}\), where \(a_{ij}\) denotes the relation between nodes \(v_{i}\) and \(v_{j}\). Notice that, the edges representing the trust relationships between user nodes are asymmetric while the others are symmetric.

**Definition 2. Trust Evaluation.** Given a Social Trust Internet of Things \(G\), let \(T=\{(v_{i},v_{j}),t_{ij}|e_{ij}\in E\}\) be the set of observed trust relationships of user nodes, where nodes \(v_{i}\) and \(v_{j}\) denote trustor and trustee, respectively, and \(t_{ij}\) measures the trustworthiness from nodes \(v_{i}\) to \(v_{j}\), which is typically application specific. For example, in SIGCOMM-2009 dataset1, trustworthiness is simply divided into two types, that is, trust or distrust. Trust evaluation is to design a mapping \(\mathcal{F}\) to evaluate the trustworthiness of unobserved/missing trust relationship of the trustor-trustee pair \(\tilde{t}_{ij}\), where \(v_{i},v_{j}\in V\), \(v_{i}\neq v_{j}\), and \(e_{ij}\notin E\). Frequently used notations are summarized in Appendix A.

Footnote 1: [https://crawdad.org/thlab/sigcomm2009](https://crawdad.org/thlab/sigcomm2009)

### Properties of SIoT Trust

To essentially establish the trustworthiness between pairwise user nodes within SIoT, multi-aspect trust properties, including asymmetric, propagative and aggregative, should be considered.

**Asymmetric Nature.** The trust relationship between nodes is unequal, that is, node \(v_{i}\) trusts node \(v_{j}\) does not mean node \(v_{j}\) trusts node \(v_{i}\), shown as Figure 1(a). Formally, let \(t_{ij}\) be the trustworthiness of trustor-trustee pair \(\langle v_{i},v_{j}\rangle\), the asymmetry nature of trust is expressed as:

\[t_{ij}\neq t_{ji}. \tag{1}\]

**Propagative Nature.** It indicates that trust may be propagated from one node to another, creating a trust chain for two nodes that are not explicitly connected. As shown in Figure 1(b), assuming that node \(v_{i}\) trusts node \(v_{j}\) and node \(v_{j}\) trusts node \(v_{k}\), it can then

Figure 1. The illustration of properties of SIoT trust.
be inferred that node \(v_{i}\) may trust node \(v_{k}\) to a certain extent. The propagation nature of trust is defined as:

\[t_{ij}\wedge t_{jk}\Rightarrow t_{ik}. \tag{2}\]

**Composable Nature.** It refers to the fact that the trustworthiness propagated by a node from different trust chains to another non-neighbor node can be aggregated. For example, in Figure 1(e), there are two trust chains among node \(v_{i}\) and node \(v_{j}\), that is, \(v_{i}\to v_{k}\to v_{j}\) and \(v_{i}\to v_{m}\to v_{n}\to v_{j}\). As a result, it is necessary to aggregate the trustworthiness from these two trust chains to determine the trust relationship from node \(v_{i}\) to node \(v_{j}\).

### Graph Neural Networks

Graph neural networks (GNNs) are a kind of neural networks that directly operate on graph-structured data (Garf et al., 2017; He et al., 2017). They typically follow the message passing framework, which learns embedding of each node through iteratively propagating and aggregating feature information from its topological neighbors. Mathematically, let \(\mathbf{h}_{i}^{(l)}\) be the latent embedding of node \(v_{i}\) at the \(l\)-th layer, the message passing process is defined as:

\[\begin{split}\mathbf{m}_{i}^{(l)}=\text{AGG}^{(l)}((\mathbf{h}_ {j}^{(l-1)}:v_{j}\in\mathcal{N}(v_{i}))),\\ \mathbf{h}_{i}^{(l)}=\text{UPD}^{(l)}(\mathbf{h}_{i}^{(l-1)}, \mathbf{m}_{i}^{(l)}),\end{split} \tag{3}\]

where AGG and UPD denote the functions to aggregate and update the message, \(\mathbf{h}_{i}^{(0)}\) represents the node's attributes, and \(\mathcal{N}(v_{i})\) denotes the set of neighbors of node \(v_{i}\).

## 3. Methodology

We first give a brief overview of the proposed method, and then introduce three key components in detail.

### Overview

To effectively assess the potential trust relationships within SIoT, we propose a novel knowledge enhanced graph neural network that can fully mine the inherent characteristics of nodes and model the multi-aspect properties of SIoT trust, namely KGTrust. The whole structure of KGTrust is illustrated in Figure 2, which consists of three main components, that is, embedding layer, heterogeneous convolutional layer as well as predictor layer. Specifically, for the embedding layer, we initialize the embeddings of users and objects by extracting useful and related knowledge from users' comment behaviors and external structured triples, so as to fully explore the intrinsic characteristics of users and objects. For the heterogeneous convolutional layer, we utilize the propagative nature of SIoT trust to augment the trust structure, and then introduce a discriminative convolutional mechanism to learn both node and object embeddings by considering the role of a user as a trustor or a trustee, respectively. After that, we leverage a learnable gating mechanism to adaptively integrate these two types of user embeddings, capturing the asymmetric nature of SIoT trust. For the prediction layer, a single-layer multiple perceptron is introduced to predict the trust relationships between user pairs based on user embeddings.

Figure 2. The architecture of KGTrust, which is constituted of three key components: 1) Embedding Layer: a comprehensive user and object modeling by integrating user comments and external knowledge triples; 2) Heterogeneous Convolutional Layer: a knowledge enhanced graph neural network to further mine and learn node latent embeddings; as well as 3) Prediction Layer: measuring the trust relationships between user pairs.
### Embedding Layer

To gain a deeper insight into the inherent characteristics of users and objects within SloT, we initialize user and object embeddings, which is the cornerstone of GNNs, by taking account of the users' comment behaviors and external knowledge related to object descriptions, respectively.

**User Embedding.** In SloT, users typically deliver their opinions on objects by providing comments in the form of text, which reflects the characteristics of users to a certain extent. Based on this, for each user, we employ Doc2vec (Deng et al., 2019), an unsupervised algorithm to learn fixed-length node embeddings for texts with variable lengths, to initialize its embedding. Specifically, for a user node \(v_{i}\), let \(d_{i}\) be the set containing all the comments delivered by \(v_{i}\), the user embedding \(\mathbf{h}_{i}\) can then be calculated as:

\[\mathbf{h}_{i}=\text{Doc2vec}(d_{i}). \tag{4}\]

**Object Embedding.** For object nodes, we capture their characteristics by integrating structured knowledge associated with object descriptions (i.e., head-predicate-tail triplets) from the knowledge graph. Here we employ TransE (Deng et al., 2019), a simple and effective approach, to parameterize triplets to learn object embeddings. It encodes the head (or tail) node as a low-dimensional embedding and the relation as algebraic operations between head and tail embeddings. Given a triplet \((h,r,t)\), let \(\mathbf{r}\) be the embedding of relation \(r\), \(\mathbf{h}\) and \(\mathbf{t}\) be the embeddings of objects \(h\) and \(t\), respectively. TransE aims to embed each object and relation by optimizing the translation principle \(\mathbf{h}+\mathbf{r}\approx\mathbf{t}\), if \((h,r,t)\) holds. The score function is formulated as:

\[f(h,r,t)=-||\mathbf{h}+\mathbf{r}-\mathbf{t}||_{2}^{2}, \tag{5}\]

where \(\mathbf{h}\) and \(\mathbf{t}\) are subject to the normalization constraint that the magnitude of each vector is 1. Intuitively, a large score of \(f(h,r,t)\) indicates that the triplet is more likely to be a true fact in real-world, and vice versa. Note that we only consider triplets where object nodes within the SloT are head instead of tail. In this way, the object embeddings can be effectively enriched at a semantic level.

**Embedding Transformation.** Considering that the generated user and object embeddings may have unequal dimensions, or even be lied in different embedding spaces, we need to project these two types of embeddings into the same embedding space. For a node \(v_{i}\) with type \(\psi_{i}\), we project its embeddings into the same latent embedding space using a type-specific linear transformation \(\mathbf{W}_{\psi_{i}}\):

\[\mathbf{h}_{i}^{\prime}=\mathbf{W}_{\psi_{i}}\cdot\mathbf{h}_{i}, \tag{6}\]

where \(\mathbf{h}_{i}\) and \(\mathbf{h}_{i}^{\prime}\) are the original and projected embedding of node \(v_{i}\), respectively.

### Heterogeneous Convolutional Layer

After initializing the user and object embeddings, we further design a heterogeneous convolutional layer, which takes the multi-aspect trust properties into consideration, so as to better assess trustworthiness among users in SloT. It mainly consists of three modules: personalized PageRank (PPR)-based neighbor sampling, information propagation and information fusion.

**PPR-Based Neighbor Sampling.** Generally, the available user-specified trust relationships within SloT are often very sparse, that is, a limited number of user pairs with trust relationships are buried in a large proportion of user pairs without trust relationships, making trust evaluation an arduous task (Kang et al., 2019). To this end, we consider employing personalized PageRank, which shows effectiveness in graph neural networks (Kipf and Welling, 2017), to augment the trust structure.

Personalized PageRank (Papnik et al., 2017) adopts a random walk with restart based strategy that uses the propagative nature of SloT trust to calculate the correlation between nodes. It takes the graph structure as input, and computes a ranking score \(p_{ij}\) from source node \(v_{i}\) to target node \(v_{j}\), where the larger \(p_{ij}\), the more similar these two nodes. Formally, given a SloT \(G=(V,E)\), let \(\hat{\mathbf{A}}=\hat{\mathbf{D}}^{-\frac{1}{2}}\hat{\mathbf{A}}\hat{\mathbf{D }}^{-\frac{1}{2}}\) be the normalized adjacency matrix, where \(\hat{\mathbf{A}}=\mathbf{A}+\mathbf{I}\) stands for the adjacency matrix with self-loops, the PPR matrix \(\mathbf{P}\) is calculated as:

\[\mathbf{P}=(1-\lambda)\hat{\mathbf{A}}\mathbf{P}+\lambda\mathbf{I}, \tag{7}\]

where \(\lambda\) is the reset probability. It is worth noting that we use a push iteration method to compute PPR scores according to the existing work (Beng et al., 2019), which can be approximated effectively even for very large networks (Kipf and Welling, 2017).

Then, the augmented trust relationships of each user node \(v_{i}\) can be constructed by choosing its top \(k\) PPR neighbors:

\[N_{i}=\operatorname*{arg\,max}_{V:<V_{U},|V^{*}|=k}\sum_{\psi_{j}\in V^{*}}p_{ ij}, \tag{8}\]

where \(V_{U}\) represents the set of user nodes in SloT. In this way, several long-range but informative trust relationships can be captured, and further promote the modeling of user nodes.

**Information Propagation.** Due to the asymmetry of trust relationships in SloT, each user may have dual roles as a trustor or trustee. For this purpose, we consider propagating node embeddings over both trustor role and trustee role, so as to extract two specific embeddings in these two roles.

Specifically, from the perspective of trustor, we learn the embeddings of users and objects through users' augmented outgoing trust relationships, objects' connections, and interactions between user-object pairs. Mathematically, given a target node \(v_{i}\), as different types of neighbor nodes (user or object) may have different impacts on it, we employ type-level attention (Kipf and Welling, 2017) to learn the importance of different types of neighbor nodes. Let \(\hat{\mathbf{A}}_{O}=[\hat{a}_{ij}]\) be the normalized adjacency matrix which is related to the trustor role, \(\mathbf{h}_{\psi}\) be the embedding of type \(\psi\), which is defined as the sum of the neighbor node embedding \(\mathbf{h}_{j}^{\prime}\) with node \(v_{j}\in\mathcal{N}_{i}\) under type \(\psi\), that is:

\[\mathbf{h}_{\psi}=\sum_{\psi_{j}}\hat{a}_{ij}\mathbf{h}_{j}^{\prime}. \tag{9}\]

Based on the target node embedding \(\mathbf{h}_{i}^{\prime}\) and its corresponding type embedding \(\mathbf{h}_{\psi}\), the type-level attention weights can then be calculated as:

\[a_{\psi}=\text{softmax}_{\psi}(\sigma(\eta_{\psi}^{T}[\mathbf{h}_{i}^{\prime},\mathbf{h}_{\psi}])), \tag{10}\]

where \(\eta_{\psi}\) is the attention vector for the type \(\psi\), and softmax is adopted to normalize across all the types.

In addition, considering that different neighbor nodes of the same type could also have different importance, we further apply node-level attention (Kipf and Welling, 2017) to learn the weights between nodes of the same type. Formally, given a target node \(v_{i}\) with type \(\psi\), let \(v_{j}\) be its neighbor node with type \(\psi^{\prime}\), the node-level attention weights
can then be computed as:

\[\beta_{ij}=\text{softmax}_{v_{j}}(\sigma(\mathbf{y}^{T}\cdot\alpha_{\psi^{ \prime}}[\mathbf{h}_{i}^{\prime},\mathbf{h}_{j}^{\prime}])), \tag{11}\]

where \(\gamma\) is the attention vector, and softmax is applied to normalize across all the neighbor nodes of the target node \(v_{i}\).

By integrating the above process, the matrix form of the layer-wise propagation rule can be defined as follows:

\[\mathbf{H}^{(l)}=\sigma(\sum_{i\neq\sigma,\mathbf{\overline{n}}}\mathbf{B}_{ \psi}\cdot\mathbf{H}_{\psi}^{(l-1)}\cdot\mathbf{W}_{\psi}^{(l-1)}), \tag{12}\]

where \(\mathcal{A}\) is the set of node types in SloT, and \(\mathbf{B}_{\psi}=(\beta_{ij})_{m\times n}\) represents the attention matrix. In this way, the specific information about trustor role can be obtained.

As for the trustee role, we learn the node embeddings via users' augmented incoming trust relationships, objects' connections, and interactions between user-object pairs, which can be calculated in the same way as in trustor role. Therefore, the specific information about trustee role can be captured by generating the node embeddings \(\mathbf{\overline{H}}\).

**Information Fusion.** In order to achieve the optimal combination of embeddings of a user in different roles (trustee or trustor) for downstream trust evaluation, we introduce a learnable gating mechanism (Wang et al., 2017) to determine how much the joint embedding depends upon the role of trustor or trustee. Given a user node \(v_{i}\), let \(\mathbf{h}_{i}\) and \(\mathbf{\overline{h}}_{i}\) represent its embeddings as trustor role and trustee role, respectively, the joint representation \(\mathbf{z}_{i}\) can be calculated as:

\[\mathbf{z}_{i}=\mathbf{g}_{e}\odot\mathbf{h}_{i}+(1-\mathbf{g}_{e})\odot \mathbf{\overline{h}}_{i}, \tag{13}\]

where \(\mathbf{g}_{e}\) is a gating vector with elements in \([0,1]\) to balance embeddings, and \(\odot\) represents element-wise multiplication. Obviously, the joint embedding with gate closer to \(0\) tends to use the embedding of a user as a trustee; whereas the joint embedding with gate closer to \(1\) utilizes the embedding of a user as a trustor. More importantly, to constrain the value of each element in \([0,1]\), we apply sigmoid function to calculate the gate \(\mathbf{g}_{e}\) as:

\[\mathbf{g}_{e}=\text{sigmoid}(\mathbf{\tilde{g}}_{e}), \tag{14}\]

where \(\mathbf{\tilde{g}}_{e}\) is a real-value vector that is learned during training.

### Predictor Layer

To convert the learned user embeddings into the latent factor of trust relationship in SloT, for a given user pair \(\langle v_{i},v_{j}\rangle\), we first concatenate the embeddings of nodes \(v_{i}\) and \(v_{j}\), and then feed them to a multiple perceptron (MLP) followed by a softmax function as:

\[\tilde{y}_{ij}=\text{softmax}(\text{MLP}(\mathbf{z}_{i}\parallel\mathbf{z}_ {j})), \tag{15}\]

where \(\parallel\) is the concatenation operator, and \(\tilde{y}_{ij}\) is the predicted probability that the user pair \(\langle v_{i},v_{j}\rangle\) belongs to a trusted pair or a distrusted pair.

Finally, we define the trust evaluation loss function by using cross entropy as:

\[\mathcal{L}=-\sum_{v_{i}v_{j}}y_{ij}\ln\tilde{y}_{ij}, \tag{16}\]

where \(y_{ij}\) denotes the ground truth of trust relationship of user pair \(\langle v_{i},v_{j}\rangle\). In particular, we employ the back propagation algorithm and Adam optimizer to train the model.

## 4. Experiments

We first introduce the experimental setup, and then compare the new approach KGTrust with state-of-the-arts in terms of effectiveness and robustness. We finally present an in-depth analysis of different components of KGTrust and give the parameter analysis.

### Experimental Setup

**Datasets.** We conduct experiments on three widely used SloT datasets, namely FilmTrust2, Ciao3 and Epinions3, where the basic information is summarized in Table 1. More details of datasets are provided in Appendix B.1.

Footnote 2: [http://www.librec.net/datasets.html](http://www.librec.net/datasets.html)

Footnote 3: [http://www.cse.msu.edu/~tanggill/trust.html](http://www.cse.msu.edu/~tanggill/trust.html)

**Baselines.** We compare KGTrust with eight state-of-art methods. They include: 1) the network embedding methods GAT (Kip
specific, in terms of accuracy, the improvement of KGTrust over different baselines ranges from 2.08% to 26.77%, 0.39% to 22.39%, and 0.57% to 23.01% on FilmTrust, Ciao, and Epinions, respectively. In terms of F1-Score, the improvement of KGTrust over different baselines ranges from 1.14% to 16.29%, 0.80% to 7.78%, and 0.73% to 17.04% on these three datasets. These results not only demonstrate the superiority of enriching node semantics with node-related knowledge, but also validate the effectiveness of flexibly preserving the multi-aspect properties of SIoT trust during information propagation. Particularly, the performance of KGTrust is much better

\begin{table}
\begin{tabular}{c|c|c c c c c c c c|c} \hline Datasets & Metrics & GAT & SGC & SLF & STNE & SNEA & DeepTrust & AtNE-Trust & Guardian & KGTrust \\ \hline \multirow{4}{*}{FilmTrust} & Accuracy & 68.29 & 75.61 & 65.55 & 72.87 & 63.91 & 53.05 & 63.11 & 77.74 & **79.82** \\ \cline{2-13}  & F1-Score & 71.74 & 77.14 & 65.65 & 73.27 & 66.67 & 64.63 & 65.13 & 79.78 & **80.92** \\ \hline \multirow{4}{*}{Ciao} & Accuracy & 64.28 & 69.93 & 72.17 & 71.33 & 68.97 & 50.17 & 68.23 & 72.17 & **72.56** \\ \cline{2-13}  & F1-Score & 71.36 & 70.34 & 73.39 & 71.38 & 70.83 & 66.52 & 71.50 & 73.50 & **74.30** \\ \hline \multirow{4}{*}{ Epinions} & Accuracy & 72.05 & 78.62 & 80.83 & 79.51 & 74.63 & 58.38 & 74.35 & 80.82 & **81.39** \\ \cline{2-13}  & F1-Score & 75.57 & 78.76 & 80.95 & 78.57 & 74.92 & 64.80 & 74.88 & 81.11 & **81.84** \\ \hline \end{tabular}
\end{table}
Table 2. Performance comparions on three SIoT datasets in terms of Accuracy (%) and F1-Score (%). (bold: best)

\begin{table}
\begin{tabular}{c|c|c|c c c c c c c c|c} \hline Datasets & Metrics & Training & GAT & SGC & SLF & STNE & SNEA & DeepTrust & AtNE-Trust & Guardian & KGTrust \\ \hline \multirow{4}{*}{FilmTrust} & \multirow{4}{*}{Accuracy} & 50\% & 60.36 & 71.26 & 54.96 & 69.42 & 60.14 & 49.51 & 60.17 & 74.14 & **74.94** \\  & & 60\% & 62.79 & 72.21 & 55.51 & 69.98 & 61.01 & 50.08 & 60.72 & 74.81 & **76.11** \\  & & 70\% & 64.39 & 73.16 & 61.22 & 72.14 & 62.90 & 50.20 & 62.14 & 75.51 & **78.16** \\  & & 80\% & 67.28 & 74.01 & 63.61 & 72.78 & 63.30 & 51.68 & 63.00 & 76.45 & **79.66** \\  & & 90\% & 68.29 & 75.61 & 65.55 & 72.87 & 63.91 & 53.05 & 63.11 & 77.74 & **79.82** \\ \cline{2-13}  & \multirow{4}{*}{F1-Score (\%)} & 50\% & 62.35 & 71.52 & 56.48 & 69.44 & 62.45 & 60.11 & 60.27 & 75.52 & **75.98** \\  & & 60\% & 63.51 & 72.40 & 57.37 & 70.17 & 62.68 & 60.38 & 61.69 & 76.52 & **76.73** \\  & & 70\% & 66.80 & 73.83 & 62.23 & 72.17 & 64.83 & 62.20 & 63.42 & 78.56 & **78.94** \\  & & 80\% & 68.34 & 74.40 & 65.00 & 72.33 & 65.12 & 63.41 & 63.88 & 79.08 & **80.47** \\  & & 90\% & 71.74 & 77.14 & 65.65 & 73.27 & 66.67 & 64.63 & 65.13 & 79.78 & **80.92** \\ \hline \multirow{4}{*}{Ciao} & \multirow{4}{*}{Accuracy} & 50\% & 59.76 & 67.40 & 71.32 & 70.69 & 66.88 & 49.80 & 62.24 & 71.27 & **71.72** \\  & & 60\% & 61.03 & 68.29 & 71.66 & 70.87 & 67.82 & 50.01 & 62.66 & 71.62 & **72.11** \\ \cline{1-1}  & & 70\% & 62.17 & 68.39 & 71.89 & 70.92 & 68.15 & 50.03 & 63.52 & 71.90 & **72.34** \\ \cline{1-1}  & & 80\% & 63.01 & 68.81 & 72.08 & 71.05 & 68.53 & 50.07 & 66.58 & 71.94 & **72.36** \\ \cline{1-1}  & & 90\% & 64.28 & 69.93 & 72.17 & 71.33 & 68.97 & 50.17 & 68.23 & 72.17 & **72.56** \\ \cline{1-1} \cline{2-13}  & \multirow{4}{*}{F1-Score (\%)} & 50\% & 66.47 & 67.53 & 71.87 & 70.83 & 67.68 & 61.30 & 62.76 & 71.84 & **72.85** \\ \cline{1-1}  & & 60\% & 68.08 & 68.58 & 72.68 & 70.85 & 68.87 & 61.38 & 63.03 & 72.28 & **73.11** \\ \cline{1-1}  & & 70\% & 70.61 & 68.78 & 72.88 & 71.07 & 69.45 & 61.77 & 65.37 & 72.67 & **73.23** \\ \cline{1-1}  & & 80\% & 70.85 & 69.76 & 73.00 & 71.32 & 70.15 & 63.63 & 69.92 & 73.32 & **74.06** \\ \cline{1-1}  & & 90\% & 71.36 & 70.34 & 73.39 & 71.38 & 70.83 & 66.52 & 71.50 & 73.50 & **74.30** \\ \hline \multirow{4}{*}{ Epinions} & \multirow{4}{*}{Accuracy} & 50\% & 61.70 & 77.22 & 79.99 & 79.04 & 73.84 & 55.53 & 71.90 & 80.15 & **80.59** \\  & & 60\% & 61.92 & 77.57 & 80.05 & 79.13 & 74.12 & 56.25 & 73.01 & 80.22 & **80.65** \\ \cline{1-1}  & & 70\% & 64.76 & 77.82 & 80.44 & 79.32 & 74.36 & 56.71 & 73.40 & 80.31 & **80.96** \\ \cline{1-1}  & & 80\% & 70.79 & 78.17 & 80.60 & 79.45 & 74.59 & 58.23 & 73.59 & 80.55 & **81.14** \\ \cline{1-1}  & & 90\% & 72.05 & 78.62 & 80.83 & 79.51 & 74.63 & 58.38 & 74.35 & 80.82 & **81.39** \\ \cline{1-1} \cline{2-13}  & \multirow{4}{*}{F1-Score (\%)} & 50\% & 65.60 & 77.63 & 80.08 & 78.18 & 73.28 & 61.27 & 72.87 & 80.41 & **81.05** \\ \cline{1-1}  & & 60\% & 66.64 & 77.92 & 80.15 & 78.22 & 73.73 & 63.93 & 73.74 & 80.51 & **81.11** \\ \cline
than that of vanilla GAT (i.e., 11.53%, 8.28%, 9.34% relative improvements in accuracy, and 9.18%, 2.94%, 6.27% relative improvements in F1-Score), which further shows the significance of jointly considering three key ingredients within SIoT, namely heterogeneous graph structure, node semantics and associated trust relationships. Neither DeepTrust nor AtNE-Trust is so competitive here, which is mainly because they fail to make better use of the information propagation over graph structure, seriously affecting their performance for trust evaluation.

**Robustness.** To further measure the stable ability of our KGTrust and baselines, we conduct experiments across all training and testing set ratios. The ratio of training set is set as \(x\%\) and the remaining \((1-x)\%\) as the testing set, where \(x\) belongs to [50, 60, 70, 80, 90] over three datasets. We run each method 10 times and report the average performance in terms of accuracy and F1-Score.

The results are shown in Table 3. As shown, the proposed method KGTrust always performs the best across different training ratios and datasets. Specifically, when fewer observed trust relationships are provided, the performances of baselines are surprisingly reduced, especially the classical GNN-based methods such as GAT, while our model still achieves relatively high performance. This demonstrates that our method can better assess trust relationships by validly alleviating data sparsity with personalized PageRank-based neighbor sampling. Moreover, as the ratio of observed trust relationships increases, KGTrust consistently maintains superior and achieves the best performance when the training set ratio is 90%, which validates the effectiveness and robustness of the proposed approach. Also of note, KGTrust outperforms Guardian, which uses GNN for trust evaluation, in all cases, further indicating the rationality of fully mining the intrinsic characteristics of nodes with the guidance of node-related knowledge.

### Ablation Study

Similar to most deep learning models, our proposed KGTrust also contains some important components that may have a significant impact on the performance. To test the effectiveness of each component, we conduct experiments by comparing KGTrust with five variations. The variants are as follows: 1) KGTrust of using random vectors instead of introducing structured triples to initialize object embeddings, named as w/o Triples, 2) KGTrust of removing PPR-based neighbor sampling, named as w/o PPR, 3) KGTrust of removing the trustee role of a node, and aggregating information only from its trustor role, named as w/o Trustee, 4) KGTrust of removing the trustor role of a node, and aggregating information only from its trustee role, named as w/o Trustor, and 5) KGTrust of employing concatenation operator instead of gating mechanism to fuse the two role embeddings (truster or trustee) of a user node, named as KGTrust (Con).

From the results in Table 4, we can draw the following conclusions: 1) The results of KGTrust are consistently better than its five variants, indicating the effectiveness and necessity of jointly taking into account of heterogeneous graph structure, node semantics and associated trust relationships within SIoT. 2) Removing structured triples or PPR-based neighbor sampling strategy leads to slight performance drop, which demonstrates the usefulness of employing external knowledge to deeply mine object semantics and propagative nature of trust to enrich the trust relationships. 3) Neither KGTrust w/o Trustee nor KGTrust w/o Trustor is so competitive here, making us realize the importance of considering the trust asymmetry nature during information propagation. 4) Compared to KGTrust (Con), the improvement brought by KGTrust is more significant, which illustrates the rationality of adaptively fusing the embeddings of dual roles of a user.

### Parameter Analysis

We investigate the sensitivity of two main parameters, including the top \(k\) for PPR-based neighbor sampling and the dimension of final embeddings, on Ciao and Epinions datasets. Results on FilmTrust dataset can be found in Appendix C.

**Analysis of \(k\).** The parameter \(k\) determines the number of trust relationships augmented by each user node with PPR-based neighbor sampling. We vary its value from 10 to 50 and the corresponding results are shown in Figure 3. With the increase of augmented trust relationships, the performance shows a trend of first rising and then descending. This is probably because a small number of augmented trust relationships are not enough to obtain informative node embeddings, whereas too many augmented trust relationships may introduce noise and thus weaken the information propagation.

**Analysis of Final Embedding Dimension.** We test the effect of the dimension of final embedding, and vary it from 16 to 256. The result is shown in Figure 4. With the increase of the dimension of final embeddings, the values of metrics, including accuracy and F1-Score, increase first and then start to decrease. It is reasonable since KGTrust needs a suitable dimension to en

\begin{table}
\begin{tabular}{l|c c|c c|c c} \hline \hline \multirow{2}{*}{Datasets} & \multicolumn{2}{c|}{FlimTrust} & \multicolumn{2}{c|}{Ciao} & \multicolumn{2}{c}{Epinions} \\ \cline{2-7}  & Accuracy & F1-Score & Accuracy & F1-Score & Accuracy & F1-Score \\ \hline KGTrust & **79.82** & **80.92** & **72.56** & **74.30** & **81.39** & **81.84** \\ - w/o Triples & - & - & 71.10 & 72.48 & 80.51 & 80.86 \\ - w/o PPR & 78.29 & 78.74 & 72.12 & 72.88 & 80.71 & 81.19 \\ - w/o Trustee & 78.13 & 79.18 & 59.07 & 64.60 & 70.73 & 72.10 \\ - w/o Trustor & 77.22 & 78.37 & 60.58 & 65.67 & 70.62 & 72.01 \\ KGTrust (Con) & 76.76 & 77.51 & 59.28 & 64.74 & 70.75 & 73.10 \\ \hline \hline \end{tabular}
\end{table}
Table 4. Comparisons of our KGTrust and its five variants on three SIOT datasets in terms of Accuracy (%) and F1-Score (%). For FilmTrust, we do not introduce structured triples as no object descriptions provided, while such information is provided by Ciao and Epinions.

Figure 3. The performance with different numbers of top \(k\) for PPR-based neighbor sampling.
within SIoT, including heterogeneous graph structure, node semantics and associated trust relationships, while larger dimensions may introduce additional redundancies, affecting the performance of assessing trustworthiness.

## 5. Related Work

We briefly review some literatures that are related to our work, that is, trust evaluation and knowledge graph neural networks.

### Trust Evaluation

Trust plays a crucial role in assisting users gather reliable information and trust evaluation. predicting the unobserved pairwise trust relationships among users, draws considerable research interest. Existing trust evaluation methods can be roughly divided into three categories, including random walk-based methods, matrix factorization-based methods and deep learning-based methods.

**Random Walk-Based methods.** In the past decade, many attentions have been paid to exploit trust propagation along the path from the trustor to the trustee to assess trustworthiness. For example, ModelTrust (Zhou et al., 2018) conducts motivating experiments to analyze the difference in accuracy and coverage between local and global trust measures, and then introduces a local trust metric to predict trustworthiness of unknown users. OptimalTrust (Zhou et al., 2018) designs a new concept, namely quality of trust, to search the optimal trust path to generate the most trustworthy evaluation. After that, AssessTrust (Zhou et al., 2018) introduces three-valued subjective logic (3VSL) that effectively considers the information in trust propagation and aggregation to assess multi-hop interpersonal trust. OpinionWalk (Zhou et al., 2018) computes the trustworthiness between any two users based on 3VSL and breadth-first search strategy.

**Matrix Factorization-Based Methods.** Several recent studies have extended matrix factorization to trust evaluation (Zhou et al., 2018), where the basic idea is to learn the low-rank representations of users and their correlations by incorporating prior knowledge and node-related data. For instance, hTrust (Zhou et al., 2018) presents an unsupervised method that exploits low-rank matrix factorization and homophily effect for trust evaluation. mTrust (Zhou et al., 2018) argues that the trust relationships among users are usually multiple and heterogeneous, and accordingly designs a fine-grained representation to incorporate multi-faceted trust relationships. sTrust (Zhou et al., 2018) proposes a trust prediction model by considering social status theory that reflects users' position or level in the social community.

**Neural Network-Based Methods.** As neural networks become the most eye-catching tools for tracking graphs, several efforts have been devoted to utilizing neural networks to boost the performance of trust evaluation. For example, NeuralWalk (Zhou et al., 2018) designs a neural network architecture that models single-hop trust propagation and trust combination for assessing trust relationships. DeepTrust (Zhou et al., 2018) presents a deep trust evaluation model which effectively mines user characteristics by introducing associated user attributes. AtNETrust (Zhou et al., 2018) improves the performance of trust relationship prediction by jointly capturing the properties of trust network and multi-view user attributes. C-DeepTrust (Zhou et al., 2018) points out that the user preference may change due to the drifts of their interests, and accordingly integrates both the static and dynamic user preference to tackle the trust evaluation problem. Guardian (Guardian, 2018) estimates the trustworthiness between any two users by simultaneously capturing both social connections and trust relationships. GATrust (Zhou et al., 2018) presents a GNN-driven approach which integrates multiple node attributes and the observed trust interactions for trust evaluation.

Despite various trust evaluation algorithms or models have been developed, they still suffer from an inability to comprehensively mine and encode the node semantics within SIoT and multi-aspect properties of SIoT trust.

### Knowledge Graph Neural Networks

To enable more effective learning on graph-structured data, researchers have dedicated to employing external knowledge(Zhou et al., 2018; Zhou et al., 2018), such as large-scale knowledge bases Wikipedia and Freebase, to enrich node representations and apply them to downstream tasks. For example, KGAT (Kog et al., 2017) presents a knowledge-aware recommendation approach by explicitly modeling the high-order connectivity with semantic relations in collaborative knowledge graph. COMPGCN (Zhou et al., 2018) learns both node and relation embeddings in a multi-relational graph via using entity-relation composition operations from knowledge graph embedding. Caps-GNN (Kang et al., 2018) designs a novel personalized review generation approach with structural knowledge graph data and capsule graph neural networks. Later on, RECON (Zhou et al., 2018) points out that knowledge graph can provide valuable additional signals for short sentences, and develops a sentence relation extraction integrating external knowledge. KCGN (Kang et al., 2018) proposes an end-to-end model that jointly injects knowledge-aware user- and item-wise dependent structures for social recommendation. However, how to utilize the guidance of external knowledge to facilitate the understanding of node semantics within SIoT, and further assess trustworthiness among users is still an area that needs to be explored urgently.

## 6. Conclusion

In this paper, we present a novel knowledge enhanced graph neural network, namely KGTrust, for trust evaluation in Social Internet of Things. In specific, we comprehensively incorporate the rich node semantics within SIoT by deeply mining and encoding node-related information. Considering that the observed trust relationships are often relatively sparse, we use personalized PageRank-based neighbor sampling strategy to enrich the trust structure. To further maintain the multi-aspect properties of SIoT trust, we learn effective node embeddings by employing a discriminative convolutional mechanism that considers the propagative and composable nature from the perspective of a user as a trustor or a trustee, respectively.

Figure 4. The performance with different dimensions of final embedding.
After that, a learnable gating mechanism is introduced to adaptively integrate the information from dual roles of a user. Finally, the learned embeddings for pairwise users are concatenated for a trust relationship predictor. Extensive experimental results demonstrate the superior performance of the proposed new approach over state-of-the-arts across three benchmark datasets.
