# Self-supervised Hypergraph Representation Learning for Sociological Analysis

Xiangguo Sun, Hong Cheng, Bo Liu, Jia Li, Hongyang Chen, Guandong Xu, Hongzhi
Yin Xiangguo Sun, and Hong Cheng are with The Chinese University of Hong Kong.
<EMAIL_ADDRESS>Bo Liu is with Southeast University, China.
<EMAIL_ADDRESS>Jia Li is with Hong Kong University of Science and Technology
(Guangzhou<EMAIL_ADDRESS>Hongyang Chen is with Zhejiang Lab, China.
<EMAIL_ADDRESS>Guandong Xu is with University of Technology Sydney,
Australia<EMAIL_ADDRESS>Hongzhi Yin is with The University of
Queensland, Australia<EMAIL_ADDRESS>
###### Abstract

Modern sociology has profoundly uncovered many convincing social criteria for
behavioral analysis. Unfortunately, many of them are too subjective to be
measured and very challenging to be presented in online social networks
(OSNs). On the other hand, data mining techniques can better find data
patterns but many of them leave behind unnatural understanding to humans. In
this paper, we propose a fundamental methodology to support the further fusion
of data mining techniques and sociological behavioral criteria. First, we
propose an effective hypergraph awareness and a fast line graph construction
framework. The hypergraph can more profoundly indicate the interactions
between individuals and their environments. A line graph treats each social
environment as a super node with the underlying influence between different
environments. In this way, we go beyond traditional pair-wise relations and
explore richer patterns under various sociological criteria; Second, we
propose a novel hypergraph-based neural network to learn social influence
flowing from users to users, users to environments, environment to users, and
environments to environments. The neural network can be learned via a task-
free method, making our model very flexible to support various data mining
tasks and sociological analysis; Third, we propose both qualitative and
quantitive solutions to effectively evaluate the most common sociological
criteria like social conformity, social equivalence, environmental evolving
and social polarization. Our extensive experiments show that our framework can
better support both data mining tasks for online user behaviors and
sociological analysis.

###### Index Terms:

hypergraph, social conformity, social influence, self-supervised learning.

##  1 Introduction

Nowadays, online behavior analysis has become one of the most eye-catching
interdisciplinary areas in academic and business because of its important role
in various web-based applications [1, 2, 3]. How to fully integrate the most
classic achievements from various disciplines is a long-term concern, which
can epitomize the whole effort for human-centered computing.

The earliest studies on human behavior mostly come from sociologists [4] where
the most typical theories therein are social equivalence [5, 6, 7] and social
conformity [8, 9]. As illustrated in Figure 1a, social equivalence measures
the sociological distance of a pair of individuals such as the user’s social
roles, network positions, friendships, etc., and found that two similar
persons usually live in a similar social environment. Social conformity states
that social influence usually makes people assimilable in their groups and
thus causes group members to follow the behavior of the majority. As shown in
Figure 1b, violating the norm in a social group may yield group pressure [10]
to a person, especially when he/she looks for an informative guide to the
group.

(a) social equivalence.

(b) social conformity.

Figure 1: Social Equivalence and Social Conformity

Social equivalence and social conformity are two skeleton views for learning
users’ behaviors in the sociology discipline because the former indicates us
to distinguish/characterize two individuals via their environments and the
latter indicates us to identify different social environments via their
members. They are very natural to explain various behaviors online and
promising to guide our real-world life. Unfortunately, although being studied
for a long time by sociologists, behavior analysis still remains an open
problem, especially when it meets online social networks (OSNs) because
quantifying these criteria in OSNs is very subjective. Besides, larger data
volume from OSNs makes it more difficult to uncover these social criteria than
the offline survey.

Figure 2: Collaborative Filtering (CF): A widely used data mining technique in
online behavior analysis.

The most notable difference between sociological studies and data mining
methods for online behavior analysis is that the latter usually characterizes
people in various latent variable spaces. Data mining methods claim that
users’ vectorized representations in latent spaces should also preserve the
similarities reflected in the real world. Figure 2 presents collaborative
filtering (CF) [11], which is one of the most typical data mining paradigms
and still dominates data mining area by many frontier researches [12, 13]. It
aims at projecting users or items as low-dimensional vectors via Matrix
Factorization [14] or more advanced deep neural networks [15, 16] on their
relation table such as rating, buying, staring, etc. The underlying idea is to
find similar users or items in the latent space and then predict a close item
for a target user. Indeed it is a very beautiful and clear methodology, but
still unnatural for human understanding and less convictive than sociology
research. After all, it is far-fetched to force unknown users that have never
seen before to believe they are similar in the real world merely because their
latent vectors are close, which is far from straightforward compared with
various widely accepted sociological theories.

The key problem is: no matter how hard we try, these latent representations
only preserve partial similarities reflected in the real world and are far
from sufficient because of the limitation of data and models. We can not make
the real person fully understand why these latent vectors are meaningful
because they are only built on similarity metrics rather than more natural
sociological criteria.

Currently, data mining models still suffer from the insufficiency of natural
proof for their results although there are lots of work trying to push it
forward. Some recent researchers have started to consider sociological
concepts on various measurement definitions [17, 18]. However, these
measurements are so subjective and heavily rely on hand-craft features, which
is very hard to be widely applied. To this end, other researchers avoid
quantifying specific concepts but try to imitate social phenomena such as
social influence [19, 20] on pair-wise graphs. Although these models integrate
social observations into their models, social influence, from the perspective
of sociologists, is far more complicated than pair-wise interactions because
it is also widely reflected in various environments. Even if we learned
individual representations, we still do not understand how these environments
perform because there is no corresponding environment uncovered. There are
also some works trying to fuse social impact for specific tasks [21, 22]. But
applying them to other scenarios might be difficult because they do not
provide a task-free framework to capture more general sociological criteria.
Filling the huge gap between sociology and data mining in online behavior
analysis is never easy and needs to solve many tough challenges, including but
not limited to:

  * •

How to explore complicated social environments beyond pair-wise relations
since most online social networks have not directly indicated them?

  * •

How to design a task-free method to learn social influence flowing that
support both data mining and sociological analysis?

  * •

How to evaluate sociological criteria in a data mining model since they are
too subjective to be measured?

In this paper, we propose a fundamental methodology that can well model online
social influence flowing under sociological criteria. Specifically, to address
the first challenge, we take both sociological attributes and user connectives
with hypergraphs beyond traditional pair-wise relations. A hypergraph allows
one edge (a.k.a hyperedge) to connect more than two nodes, making our learned
social environments more flexible than classic graphs; To address the second
challenge, we propose a hypergraph-based social influence flowing model to
learn individual and environmental interaction patterns under social criteria;
To address the third challenge, we propose a package solution to evaluate some
widely concerned sociological criteria with both qualit-ative and quanti-tive
ways. In summary, our main contributions are as follows:

  * •

We propose a hypergraph-based social environment awareness method and a fast
line graph algorithm. The hypergraph naturally supports learning the
interaction patterns between individuals and their environments. The line
graph takes each environment as a new node and learns the interactions between
different environments. In this way, our perceived environments are far richer
than traditional pair-wise relations and closer to the real world.

  * •

We propose a novel hypergraph neural network to learn influence flowing under
social criteria. Besides pair-wise relations that can only model the
individual impact, our model can further learn social influence flowing from
users to environments, environments to users, and environments to
environments, making our model more expressive to reflect more profound
sociological criteria.

  * •

We propose both qualitative and quantitive methods to evaluate the most common
sociological criteria like social conformity, equivalence, environment
evolving, social polarization and et al.

  * •

We conduct extensive experiments, from which we can find that our framework
not only supports better performance in data mining tasks but also effectively
preserve rich criteria in sociological analysis.

Figure 3: Hypergraph-based Social Influence Flowing under Sociological
Criteria.

##  2 Objectives and Definitions

###### Definition 1

(Hypergraph and its Line Graph) We denote a hypergraph by
$\mathcal{G}=\\{\mathcal{V},\mathcal{E}^{h},\mathcal{E}^{p},\bm{X},\bm{W},\bm{U}\\}$
where $\mathcal{V}$ is the node set denoting all the users; $\mathcal{E}^{h}$
is the collection of hyperedges where each hyperedge can be treated as a set
of users this hyperedge contains; $\mathcal{E}^{p}$ contains pair-wise
relations between users; $\bm{X}\in\mathbb{R}^{|\mathcal{V}|\times d}$ is user
attributes matrix;
$\bm{W}\in\mathbb{R}^{|\mathcal{E}^{h}|\times|\mathcal{E}^{h}|}$ and
$\bm{U}\in\mathbb{R}^{|\mathcal{V}|\times|\mathcal{V}|}$ are two diagonal
matrices where diagonal entries denote the hyperedge weight and user weight,
respectively. The line graph of this hypergraph is denoted by
$\mathcal{G}^{\circ}=\\{\mathcal{V}^{\circ},\mathcal{E}^{\circ}\\}$ where each
node in $\mathcal{V}^{\circ}$ corresponds to one hyperedge of the hypergraph
$|\mathcal{V}^{\circ}|=|\mathcal{E}^{h}|$; $\mathcal{E}^{\circ}$ is the pair-
wise connection of different node in $\mathcal{V}^{\circ}$. Usually,
$\mathcal{E}^{\circ}$ exists because hyperedges have some overlapping users or
share similar characteristics.

In this paper, we use hyperedges to present social environments, and pair-wise
links present individual impact. Specifically, we define the social
environment as a set of people who have close social patterns and can
influence each other, which can be well undertaken by our hyperedges. The
hyperedge weights $\bm{W}$ can be pre-defined as the hyperedge size and the
user weights $\bm{U}$ can be pre-defined as their
centrality111https://wikipedia.org/wiki/Centrality in the real world. They can
also be manually assigned by the social demand or treated as free parameters
learned by downstream tasks.

Objectives: With the above concepts, the objectives of this paper are two-
fold: The first one is to design a task-free framework to learn user
representation that can reflect social criteria well. Based on this, our
second objective is to evaluate whether these representations fused with
social criteria perform better in online behavior analysis.

##  3 Influence Flowing under Social Criteria

Our framework is shown in Figure 3 where the initial online social network is
augmented by two layer graphs. The upper graph contains various hyperedges,
which contain complicated impact of social environment and users; the bottom
graph is the line-graph of the hyperedges, which can present the correlations
of different social environments. With the above networks, we propose a
hypergraph neural network following sociological influence flowing patterns.

###  3.1 Social Environment Awareness via Hypergraphs

Figure 4 presents widely used hyperedge construction methods in existing
hypergraph researches. Unfortunately, none of them are sufficient to support
more profound sociological criteria. Specifically, clustering-based methods
[23] heavily rely on users’ attributes but ignore their real world
interactions, making the hyperedges far from flexible. Community-based methods
[24] consider user’s network connectivity but in practice not every user
should be assigned to a specific community and the hyperedges are less
hierarchical; Neighbour-based methods [25] take k-hop neighbours around a
centered user as a group but this may cause huge hyperedge redundancy.

(a) cluster

(b) community

(c) neighbours

Figure 4: Commonly Used Social Environments

Here we propose a new method to reduce the hyperedge redundancy, make them
more hierarchical, and let them reflect both social attributes and social
interactions. Let $\bm{F}\in\mathbb{R}_{[0,1]}^{N\times C}$ be user
probability distributions on the hyperedges where $N$ is the number of user;
$C$ is the number of social environments (a.k.a hyperedges); and each entry
value $0\leq\bm{F}_{ij}\leq 1$. We claim that the hyperedges should both
relate to user attributes $\bm{X}\in\mathbb{R}^{N\times d}$ and their
connectivity matrix $\bm{A}\in\\{0,1\\}^{N\times N}$. Then $\bm{F}$ can be
estimated by a parameterized ($\theta$) transformation
$\overline{\bm{F}}=f(\bm{X}|\theta)$ with the following negative log-
likelihood target:

$\displaystyle\theta^{\star}=\underset{\theta}{\arg\min}\Big{\\{}$
$\displaystyle-\mathbb{E}_{p_{uv}^{+}}\left[\log\left(1-\exp(-f_{u}(\bm{X}|\theta)\cdot
f_{v}^{T}(\bm{X}|\theta))\right)\right]$ (1)
$\displaystyle+\mathbb{E}_{p_{uv}^{-}}\left[f_{u}(\bm{X}|\theta)\cdot
f_{v}^{T}(\bm{X}|\theta)\right]\Big{\\}}$

where $p^{+}$ and $p^{-}$ denote the probabilities of positive edges and
negative edges. Here we can sampled these edges from the connectivity matrix
$\bm{A}$. With the learned hyperedge probability $\bm{F}$, the social
environments can be finally denoted by a hypergraph:
$\mathcal{G}=\\{\mathcal{V},\mathcal{E}^{h},\mathcal{E}^{p},\bm{X},\bm{W},\bm{U}\\}$.

###  3.2 Hypergraph Influence Flowing under Social Criteria

Unlike most related work that only care about individual impact, more
sociological criteria are reflected in both individuals and environment level.
To this end, we first generate a line graph of the hypergraph, which can
indicate the influence among various environments, then with both hypergraph
and its line graph, we propose a hypergraph flowing model to capture richer
social criteria.

####  3.2.1 Generative Dual Network

Although our hypergraph contains the potential social patterns of individual
to individual and individual to hyperedge, it still lacks of the social
patterns of hyperedge to hyperedge, which can be well solved by hypergraph’s
line graph. Here the line graph takes each hyperedge as a new node and
calculates the correlations of different hyperrdges to decide the pair-wise
connectivity of the line graph. Let $s_{ij}$ be the similarity score between
hyperedge $i$ and $j$. Since groups with similar proposition usually cooperate
more closely than the others, we can follow the Bernoulli distribution to
reconstruct their real world connection as:

$\small\bm{A}^{\circ}_{ij}\sim\operatorname{Bernoulli}\left(s_{ij}\right)$

where $\bm{A}^{\circ}\in\mathbb{R}^{|\mathcal{E}^{h}|\times|\mathcal{E}^{h}|}$
is the estimated adjacency matrix of the line graph. To calculate the
similarity score $s_{ij}$ of all pairs of hyperedges, previous work[26, 12]
usually calculate the Jaccard Similaritiy on any hyperedge pair, or take the
transition on hypergraph incidence matrix like
$(\bm{H}^{T}\bm{H})_{|\mathcal{E}^{h}|\times|\mathcal{E}^{h}|}$.
Unfortunately, the first method is low efficient and the second takes up more
memories. Even we efficiently get all pairs similarity scores, the total
generative space still remains as $|\mathcal{E}^{h}|\times|\mathcal{E}^{h}|$,
making the generation very slow.

To solve these problems, we propose a very effective method shown in Algorithm
1 to fast estimate a line graph. Let $\mathcal{V}^{\circ}$ be a set of super
nodes where each node corresponds to a hyperedge. Then we can treat the
original hypergraph as a bipartite graph
$\mathcal{M}=\\{\mathcal{V},\mathcal{V}^{\circ},\mathcal{E}^{b}\\}$. We start
from each super node by selecting a member in this hyperedge and then visit
another hyperedge (super node) this member also belong to. We repeat this
process until there is no more appropriate nodes selected or the walk length
exceeds the max length. Then we can start a new walk and continue the above
process. Once the algorithm finished, we can immediately obtain a collection
of walking paths where each pair of adjacent nodes in a path denotes the
concurrence of two hyperedges. The frequency of a hyperedge pair $(i,j)$ on
the total collection is an estimation of $s_{ij}$. Therefore, we can directly
sampled hyperedge pairs from this collection as the edges of the line graph.

Input: Hypergraph $\\{\mathcal{V},\mathcal{E}^{h}\\}$; max length $\ell$;
repeating times $r$;

Output: line graph $\\{\mathcal{V}^{\circ},\mathcal{E}^{\circ}\\}$

1 // Initialize a path multiset

2 $\mathcal{C}=\emptyset$

3 //Denote the hypergraph as a bipartitle graph.

4 $\mathcal{M}=\\{\mathcal{V},\mathcal{V}^{\circ},\mathcal{E}^{b}\\}$

5 for  _$i$ from $1$ to $r$_ do

6 for  _$v_{0}^{\circ}\in\mathcal{V}^{\circ}$_ do

7 //random walk on $\mathcal{E}^{b}$ from $v_{0}^{\circ}$ within $2\ell$
steps.

8
$p=\\{v_{0}^{\circ},v_{0},v_{1}^{\circ},v_{1},\cdots,v_{0},v_{k}^{\circ},v_{k}\\},k\leq\ell$

9 //update the multiset

10
$\mathcal{C}=\mathcal{C}\cup\\{(v_{0}^{\circ},v_{1}^{\circ}),(v_{1}^{\circ},v_{2}^{\circ}),\cdots,(v_{k-1}^{\circ},v_{k}^{\circ})\\}$

11

12 end for

13

14 end for

15 //sample different elements from $\mathcal{C}$

16 $\mathcal{E}^{\circ}\sim
P_{(v_{i}^{\circ},v_{j}^{\circ})}=\frac{\underset{(v_{m}^{\circ},v_{n}^{\circ})\in\mathcal{C}}{\sum}I\left((v_{m}^{\circ},v_{n}^{\circ})=(v_{i}^{\circ},v_{j}^{\circ})\right)}{|\mathcal{C}|}$

return _ $\\{\mathcal{V}^{\circ},\mathcal{E}^{\circ}\\}$ _

Algorithm 1 Fast Line Graph Construction

####  3.2.2 Hypergraph Neural Network under Social Criteria

Compared to traditional models that usually consider one-way influence like
$group\\!\rightarrow\\!user$ or $user\\!\rightarrow\\!user$, we propose to
joint the hypergrpah and its line graph to provide more flexible
expressiveness on richer social criteria among individual $\Leftrightarrow$
individual, individual $\Leftrightarrow$ environments, and environments
$\Leftrightarrow$ environments. To this end, we here propose a novel
hypergraph-based neural network to capture social influence flowing.

A. Influence Flowing on the Hypergraph. Compared with traditional graph models
which seek smoothness on pair-wise links, hypergraphs are more feasible to
achieve the smoothness on various environmnets, which means we can easily
preserve some sociological criteria such as social conformity et al. and
perform better with the impact of multiple environments.

We define a binary hypergraph incidence matrix as
$\bm{H}\in\\{0,1\\}^{|\mathcal{V}|\times|\mathcal{E}^{h}|}$ where
$\bm{H}_{uk}=1$ means user $u$ is in the hyperedge $k$ and $\bm{H}_{uk}=0$
means the user $u$ does not belong to $k$. Let
$\bm{D}^{e}\in\mathbb{R}^{|\mathcal{E}^{h}|\times|\mathcal{E}^{h}|}$ be the
hyperedge degree with diagonal entries
$\bm{D}^{e}(k)=\sum_{u\in\mathcal{V}}\bm{U}_{u}\bm{H}_{uk}$. Similarly, let
$\bm{D}^{v}(u)=\sum_{k\in\mathcal{E}^{h}}\bm{W}_{k}\bm{H}_{uk}$ be node
degrees; Inspired by social influence theory, social influence flowing from
different nodes via multiple environments can be approximated in the following
way:

$\bm{\Theta}=\left(\bm{D}^{v}\right)^{-\frac{1}{2}}\bm{U}\bm{H}\bm{W}\left(\bm{D}^{e}\right)^{-1}\bm{H}^{\mathrm{T}}\bm{U}\left(\bm{D}^{v}\right)^{-\frac{1}{2}}$
(2)

Intuitively, $\bm{H}\bm{H}^{\mathrm{T}}$ measures the social influence flowing
from users to environments and then environments to users. Then $\bm{\Theta}$
can be treated as a normalized form of $\bm{H}\bm{H}^{\mathrm{T}}$ weighted by
user and environment impacts. Furthermore, we can extend the above equation to
capture farther impact as follows:

$\bm{\Theta}_{\Sigma}=\sum_{k=1}^{K}\gamma^{k-1}\bm{\Theta}^{k}$ (3)

where $\gamma$ is a decay parameter for the observation that far away social
influence usually have more feeble impact on users. With the above formulas,
user representations on a hypergraph can be affected as follows:

$\bm{X}^{\ell+1}=f\left((\bm{\Theta}_{\Sigma}-\operatorname{Diag}_{\bm{\Theta}_{\Sigma}}+\bm{I})\cdot\bm{X}^{\ell}\cdot\bm{P}^{\ell}\right)$
(4)

where $\operatorname{Diag}_{\bm{\Theta}_{\Sigma}}$ is the diagonal matrix of
$\bm{\Theta}_{\Sigma}$;
$\bm{I}\in\\{0,1\\}^{|\mathcal{V}|\times|\mathcal{V}|}$ is the identity
matrix; $\bm{P}^{\ell}\in\mathbb{R}^{|\mathcal{V}|\times d}$ is the feature
transformation matrix. $f(\cdot)$ is an activate function such as “Relu”,
“Sigmoid” et al; $\ell=1,\cdots,L$ denotes the neural network layer; The last
layer output can be denoted by $\bm{R}^{h}=\bm{X}^{L+1}$.

B. Influence Flowing on Pair-wise Relations. Besides the interactions between
users and their environment, our hypergraph notation also includes the pair-
wise links $\mathcal{E}^{p}$, which can support us build individual level
impact via simple graph neural network [27] like:

$(\bm{X}^{p})^{\ell+1}=f\left((\bm{D}^{p})^{-\frac{1}{2}}\bm{A}^{p}(\bm{D}^{p})^{-\frac{1}{2}}(\bm{X}^{p})^{\ell}(\bm{P}^{p})^{\ell}\right)$
(5)

where $\bm{A}^{p}\in\mathbb{R}^{|\mathcal{V}|\times|\mathcal{V}|}$ is a pair-
wise adjacency matrix built on $\mathcal{E}^{p}$ and self-connections;
$\bm{D}^{p}_{ii}=\sum_{j}\bm{A}^{p}_{ij}$; $\bm{P}^{p}$ is the weight matrix.
The last layer output can be denoted by $\bm{R}^{p}=(\bm{X}^{p})^{L+1}$.
Intuitively, Equation (5) measures the social impact in individual level. With
user representations both from hypergraph and pair-wise relations, we can
combine them together:

$\bm{R}^{encode}=\bm{R}^{h}\oplus\bm{R}^{p}$ (6)

where $\oplus$ is the concatenation operation.

C. Influence Flowing on the Line Graph. Besides the above social influence
flowing, social criteria are also widely reflected in mutual influence between
different environments, which can be well addressed in hypergraph’s line graph
$\mathcal{G}^{\circ}=\\{\mathcal{V}^{\circ},\mathcal{E}^{\circ}\\}$ where each
node in $\mathcal{V}^{\circ}$ corresponds to an environment entity in the
hypergraph and $\mathcal{E}^{\circ}$ are pair-wise interactions for these
environments.

Inspired by social conformity theory, we can represent the environment by
taking the summation of all its members like
$\bm{X}^{\circ}=\bm{H}^{T}\bm{R}^{encode}$ where
$\bm{X}^{\circ}\in\mathbb{R}^{|\mathcal{V}^{\circ}|\times d}$ is the hyperedge
representation matrix. Then we can measure the social influence flowing from
an environment to another environment on the line graph by a similar graph
neural network to Equation (5). Let the updated hyperedge representations be
$(\bm{X}^{\circ})^{L+1}$. Inspired by social equivalence theory, we can use
the environments to denote its members as:
$\bm{R}^{\star}=\bm{H}(\bm{X}^{\circ})^{L+1}$ where
$\bm{R}^{\star}\in\mathbb{R}^{|\mathcal{V}|\times d}$ is the reconstructed
user representations from their environments.

###  3.3 Task-free Learning by Dual and Contrastive Ideas

Contrastive learning [12] on simple graphs usually generates different graph
views (different topological structures) and then use the same encoder to get
their own representations. It tries to make the same entity over different
views as close as possible and keeps nonaligned entities away from each other.
Unfortunately, although our framework might generate several “views”
(hyperedges, pair-wise relations, and line graph edges), they can not be
modeled by the same encoder, and nodes in different views are not always
aligned (for example, nodes in line-graph are not aligned with any nodes in
other graphs). Besides, there are also rich influence between different views
neglected, leaving classic contrastive methods not applicable here. Luckily,
we can borrow recent idea of dual learning [28] in machine translation that
uses two dual translators (like ${Chinese\\!\rightarrow\\!English}$ and
${English\\!\rightarrow\\!Chinese}$) to compare the initial representations
and the reconstructed ones. In this paper, we can also obtain users’ initial
features and their reconstructed representations from the hypergraph and its
line graph. However, dual learning only cares about the same entity but
ignores the other entity pairs that also impact the final learning results.

To this end, we first obtain the reconstructed user representation by
projecting previous $\bm{R}^{\star},\bm{R}^{h},\bm{R}^{p}$ via a Multilayer
Perceptron (MLP) like:
$\widehat{\bm{X}}=\operatorname{MLP}(\bm{R}^{\star}\oplus\bm{R}^{h}\oplus\bm{R}^{p})$.
Then we compare users’ initial representation $\bm{X}$ and the reconstructed
matrix $\widehat{\bm{X}}$ with both aligned entities a.k.a positive pairs
$\mathcal{P}=\\{(u,u)|u\in\mathcal{V}\\}$ and nonaligned entity pairs a.k.a
negative pairs
$\mathcal{N}=\\{(u,v)|u\in\mathcal{V},v\in\mathcal{V},(u,v)\notin\mathcal{E}^{p}\\}$.
Our task-free training loss with both feature and structure reconstruction can
be defined as follows:

$\small\mathcal{L}\\!=\\!\\!\\!\\!\\!\\!\underset{(u,u)\in\mathcal{P}}{\sum}\\!\\!\\!\left[\|\bm{X}_{u}\\!-\\!\widehat{\bm{X}}_{u}\|^{2}_{2}\\!-\\!m_{p}\right]_{+}\\!\\!\\!\\!+\\!\\!\\!\\!\underset{(u,v)\in\mathcal{N}}{\sum}\\!\\!\\!\\!\left[m_{n}\\!\\!-\\!\|\bm{X}_{u}\\!\\!-\\!\widehat{\bm{X}}_{v}\|^{2}_{2}\right]_{+}$
(7)

where $m_{p}$ presents the margin over which positive pairs will contribute to
the loss; $m_{n}$ is the negative margin under which negative pairs will
contribute to the loss. After the learning process finished, we can use the
updated model to generate $\bm{R}^{\star},\bm{R}^{h},\bm{R}^{p}$ to support
downstream tasks, which can be vividly named as “pluggable” way. We can also
combine Equation (7) with task specific loss and then directly conduct
downstream tasks in an “unpluggable” way.

###  3.4 Evaluating Sociological Criteria

There is a long-term concern on how to reveal the sociological criteria in a
data mining model because they are usually too subjective to be measured.
Here, we wish to push forward by a package solution to some widely concerned
sociological criteria:

Social Conformity.  With the representations of users and hyperedges learned
by the model, social conformity can be easily visualized by the users who have
higher similarities to the hyperedge (section 4.8). Besides, it can be also
quantitatively measured as follows:

$\small co=\mathbb{E}_{e\sim P(\mathcal{E}^{h})}\left[\sum_{u\in
e}\frac{\operatorname{I}(s_{ue}>\rho)}{|e|}\right]$ (8)

where $co$ is the expected ratio of significant users to each hyperedge;
$\operatorname{I}(\cdot)$ is the condition indication; $s_{ue}$ is the
similarity of user $u$ and hyperedge $e$; $\rho$ is the significance
threshold. $P(\mathcal{E}^{h})$ is the probability distribution on hyperedges.

Social Equivalence. A visualization way is to draw the hyperedges and compare
whether close users in the real world also share similar hyperedges (section
4.8), which is also equivalent to a quantified measurement as follows:

$\footnotesize eq\\!=\\!\frac{\\!\\!\\!\\!\\!\\!\underset{(u,v)\sim
P(\mathcal{E}^{p})}{\mathbb{E}}\left[\underset{e\in\mathcal{E}^{h}}{\sum}\\!\\!\\!\operatorname{I}(u\\!\in\\!e\\!\land\\!v\\!\in\\!e)/\\!\operatorname{I}(u\\!\in\\!e\\!\lor\\!v\\!\in\\!e)\right]}{\\!\\!\\!\\!\\!\\!\underset{(u^{{}^{\prime}}\\!\\!,v^{{}^{\prime}})\sim
P(\\!\mathcal{V}\times\mathcal{V}\setminus\mathcal{E}^{p})}{\mathbb{E}}\left[\underset{e\in\mathcal{E}^{h}}{\sum}\\!\\!\\!\operatorname{I}(u^{{}^{\prime}}\\!\\!\in\\!e\\!\land\\!v^{{}^{\prime}}\\!\\!\in
e)/\\!\operatorname{I}(u^{{}^{\prime}}\\!\\!\in\\!e\\!\lor\\!v^{{}^{\prime}}\\!\\!\in\\!e)\right]}$
(9)

where the numerator is the expected Jaccard index of environments for the
positive user pairs; the denominator is the same thing for the negative pairs.
Intuitively, $eq>1$ means the social equivalence is significantly observed.

Social Environment Evolving. We calculate the ratio of significant users to
the group size and then we can directly see the ratio changing during
different phases (section 4.9).

Social Polarization. As a natural consequence of social influence, social
polarization [4] is the phenomenon that users may enhance their belief if
their neighbours take the same attitudes. Here we measure a user’s belief from
the perspective of information
theory222https://wikipedia.org/wiki/Information_theory. For each node $u$ in a
hyperedge $i$, we use the dot product of $u$ and $i$ to present the confidence
of this user believing his opinion, say $p_{ui}$. Then the uncertainty is
defined as $h_{ui}=-\log p_{ui}$ and the lower value indicates more radical
opinion this user holds. Then the Social Polarization within a group can be
easily visualized by the uncertainty distributions of group members (section
4.10). Furthermore, the overall polarization degree in group $i$ can be also
measured by the expectation of all group members a.k.a group entropy:

$po(i)=-\mathbb{E}_{u\sim p_{ui}}\left[\log p_{ui}\right]$ (10)

##  4 experiment

###  4.1 Datasets

We evaluate our framework on five real-world datasets. Detailed statistics are
presented in Table I. MovieLens333available at https://movielens.org is a
movie rating network where nodes include ”movie” and ”user”. Each observed
pair-wise link starts from a user and points to a movie. The link value
presents the user’s rating score (from $0$ to $5$) to a target movie; Facebook
[29] is a social network where nodes denote user accounts and pair-wise edges
are following relations; FakeNews [30] dataset contains 314 retweeting
networks where 157 of them are fake news. Nodes in each graph are users and
pair-wise links denote the retweeting relations; PersonalityCafe444available
at https://www.personalitycafe.com is an online forum where most of the users
have their personality types which are denoted by Myers–Briggs Type Indicator
(MBTI)555https://wikipedia.org/wiki/Myers-Briggs_Type_Indicator. The dataset
contains 42K users as nodes and the pair-wise links denote some users quoted
the others in the forum; KaraClub [31] is a classic small-scaled network with
34 nodes connected by 156 edges. We use it to visualize our hyperedge
structure compared with other methods.

###  4.2 Baselines

We evaluate our framework with the following baselines: metapath2vec [32]
projects network entities to multiple latent spaces with independent random
walks along different meta paths; Confluence [17] first defines social
conformity as three kinds of calculations from user’s historical actions and
then uses the relation network to learn the weights of different kinds of
conformity metrics; OCCFRP [18] quantifies social influence and introduce it
with collaborative filtering method to predict users’ retweeting actions; NCF
[33] enhances collaborative filtering model with deep neural network where
each user or item is represented by a neural encoder and the learning process
integrate both Matrix Factorization and the multi-layer perceptron; DeepInf
[19]models social influence within local sub-network uses graph convolution
network to encode instances from the sampled neighbors; SIDM [22] splits
social interaction network as influence structure and influence dynamics and
then model each of them via graph neural networks.

TABLE I: Dataset Statistics

|  Node #Number |  Pair-wise Relation #Number
---|---|---
MovieLens |  movie; user #9,742; #610 |  (user,rates, movie) #70,586
Facebook |  user # 4,039 |  (user, follows, user) #88,234
FakeNews |  user #41,054 |  (user retweet user) #40,740
PersonalityCafe |  user # 42,360 |  (user, quote, user) #889,706
KarateClub |  user #34 |  (user, follow, user) #156

###  4.3 Metrics and Experimental Settings

Tasks and Metrics: For the rating problem, we aim at predicting users’ rating
scores for specific items (such as movies). We treat this problem as value
regression and thus the adopted metrics include MAE (mean absolute error), MSE
(mean squared error), R2 (coefficient of determination), RMSE (root mean
square error), and Max Error. For the following relation awareness, we treat
it as the link prediction and evaluate on AUC (area under the ROC curve), AP
(average precision, a.k.a area under the PR curve), MRR (mean reciprocal
rank), Hits@1, and Hits@5 (hit ratio at top 1 and top 5). For the fake news
identification, each tweet has its own retweeting graph and we treat it as a
binary graph classification. Our metrics include AUC, AP, F1, Precision, and
Recall. For personality detection, we treat it as a multi-label classification
on graph nodes and evaluate the performance by Accuracy, Macro/Micro F1,
Macro/Micro Precision, and Macro/Micro Recall.

TABLE II: Personality Detection ($30\%$ labeled)

| Accuracy | Macro F1 | Macro Precision | Macro Recall | Micro F1 | Micro Precision | Micro Recall
---|---|---|---|---|---|---|---
metapath2vec | 0.1856 | 0.7450 | 0.7155 | 0.8315 | 0.7645 | 0.7106 | 0.8272
Confluence | 0.2201 | 0.7567 | 0.7362 | 0.8003 | 0.7573 | 0.7435 | 0.8066
OCCFRP | 0.2366 | 0.7786 | 0.7603 | 0.8102 | 0.8025 | 0.7701 | 0.8238
NCF | 0.2515 | 0.7805 | 0.7764 | 0.8035 | 0.7897 | 0.7805 | 0.8105
DeepInf | 0.2937 | 0.7703 | 0.7903 | 0.7685 | 0.7854 | 0.8006 | 0.7764
SIDM | 0.2891 | 0.7819 | 0.7607 | 0.8127 | 0.7974 | 0.7706 | 0.8260
Our (pluggable) | 0.2593 | 0.8169 | 0.7141 | 0.9902 | 0.8244 | 0.7197 | 0.9920
Our (unpluggable) | 0.4555 | 0.8337 | 0.8064 | 0.8695 | 0.8464 | 0.8171 | 0.8837
(bottom line) | (0.0625) | (0.5751) | (0.6885) | (0.4999) | (0.5792) | (0.6885) | (0.4999)

  * •

value: top 2 results.

Data Preparing  For the Movielens dataset, we remove 20% of the total observed
rating edges as a testing set, split 10% edges as validation, and the rest as
training edges. Each edge contains a rated score from $0$ to $5$ which will be
scaled within $[0,1]$ for training and evaluation. For the Facebook dataset,
we split training, validation, and testing edges with the same proportion in
Movielens. In the meanwhile, we also sample $100$ negative edges for each
positive edge so that we can evaluate the performance on link prediction via
AUC, AP, MRR, and Hits@K. For the FakeNews dataset, each independent graph
denotes a retweeting network and we treat the task as graph classification. We
take the default split by the dataset ($62:31:221$ for training, validating,
and testing); For the PersonalityCafe dataset, there are 42, 360 users in
total but only 22, 817 of them have MBTI personality labels. We split these
22, 817 users as training, validation, and testing nodes with the proportion
of $10\text{K}:2.8\text{K}:10\text{K}$. Users with observed personality labels
in the training process only take up 30% of the total.

Parameter Settings For most tasks, we train our model with 200 epochs, Adam
optimizer, 0.01 learning rate, and 100 dimension initial entity features
obtained from metapath2vec. We take two layers for the classical graph and
hypergraph neural networks in the encoding process and one layer of graph
neural networks in the decoding process. We design this asymmetric structure
so that we can learn stronger representations and reduce training complexity.
More detailed settings are presented in our open
project666https://tinyurl.com/tkde2023.

###  4.4 Behavior Analysis based on our Model

####  4.4.1 Effectiveness on Personality Detection

In the field of sociological psychology, users’ personality traits are the
most crucial factors to analyze their behaviors. One of the most acceptable
psychology models to denote one’s personality traits is the Myers-Briggs Type
Indicator, which evaluates users over four categories:
introversion/extraversion, sensing/intuition, thinking/feeling,
judging/perceiving. Each person has one preferred quality from each category,
making the target problem equal to multi-label classification.

We repeat our experiment $10$ times and report the average results of our
model and the other baselines in Table II, from which we can observe that our
method achieves the best results in most metrics. Here the “pluggable” method
means we only train our model in a self-supervised way and then load the
learned representations to train a downstream multi-label classifier. The
“unpluggable” method means we join the representation learning and multi-label
classifier together and then directly output users’ personality traits.
Compared with the unpluggable method, our pluggable method also outperforms
the rest baselines in most metrics, which can further indicate the
effectiveness of our learning framework. Note that SIDM, DeepInf, OCCFRP, and
Confluence all try to integrate some social influence observations into their
model but SIDM and DeepInf avoid manually defined measurements on these
subjective concepts and achieve better performance than the rest. However,
social influence is only a narrow aspect of sociology research and there are
more profound sociological routines that can further support personality
detection in our framework, which are also observable in the following
experiments.

(a) Initial Feature

(b) AutoEncoder

(c) GCN

(d) Our Model

Figure 5: Representation Distributions w.r.t Learning Models

####  4.4.2 Effectiveness on Rating Estimation

Rating items is one of the most common cases in many data mining applications
where users can score the item to present their preferences. Here we treat
this task as score value regression and report the average results in Table
III. Here the “pluggable” method means we only train our framework by our
proposed dual contrastive loss and then directly output the predicted score
via dot operation on user and item representations. “unpluggable” method means
we train our framework by the joint of dual contrastive loss and a task-
specific loss such as the “L1Loss” (a.k.a mean absolute error).

TABLE III: Rating Prediction

| MAE$\downarrow$ | MSE$\downarrow$ | R2$\uparrow$ | RMSE$\downarrow$ | Max Error$\downarrow$
---|---|---|---|---|---
metapath2vec | 0.2476 | 0.0920 | 0.0889 | 0.3033 | 0.9986
Confluence | 0.3079 | 0.1306 | 0.0065 | 0.3614 | 0.9988
OCCFRP | 0.2866 | 0.1049 | 0.0926 | 0.3239 | 0.9255
NCF | 0.2376 | 0.0901 | 0.1106 | 0.3002 | 0.8972
DeepInf | 0.2235 | 0.0860 | 0.1303 | 0.2933 | 0.8865
SIDM | 0.2011 | 0.0794 | 0.1326 | 0.2818 | 0.8990
Our (pluggable) | 0.1457 | 0.0371 | 0.2392 | 0.1925 | 0.8681
Our (unpluggable) | 0.1397 | 0.0347 | 0.2870 | 0.1863 | 0.8920

  * •

$\uparrow$: higher better; $\downarrow$: lower better; value: top 2 results.

As shown in Table III, our model still keeps better performance on all the
metrics. In particular, our unpluggable method achieves the best results in
most cases. Although it has larger Max Error than DeepInf, it is still very
competitive. Besides our methods, SIDM and DeepInf also achieve generally
better performance than the rest baselines, which may further suggest the
advantages of integrating the data mining model with profound sociological
observations.

####  4.4.3 Effectiveness on Following Relation Awareness

For the task of following relation awareness, we first sample $100$ negative
following edges for each positive edge and then use our learned node embedding
to discriminate the positive ones from the blended testing edge set. We treat
this task as unbalanced binary classification and report the average results
in Table IV where the “pluggable” way of our method means we train our model
in a self-supervised way and then load the learned representations to train a
binary edge classifier with the cross entropy loss. Since the positive and
negative edges are very unbalanced, we take the gradient harmonizing mechanism
(GHM) [34] to regular the task loss. The “unpluggalbe” way of our method means
we bland our model embedding with a binary classifier and then jointly train
the total pipeline with our proposed dual contrastive loss and the
classification loss.

TABLE IV: Following Relation Awarenesss

| AUC | AP | MRR | Hits@1 | Hits@5
---|---|---|---|---|---
metapath2vec | 0.9322 | 0.2122 | 0.4869 | 0.2385 | 0.7588
Confluence | 0.8902 | 0.2164 | 0.4991 | 0.2897 | 0.7536
OCCFRP | 0.8864 | 0.2201 | 0.5115 | 0.3006 | 0.7644
NCF | 0.9037 | 0.2334 | 0.5564 | 0.3537 | 0.7967
DeepInf | 0.9106 | 0.2606 | 0.5632 | 0.3162 | 0.8036
SIDM | 0.9237 | 0.2564 | 0.5537 | 0.3384 | 0.8364
Our (pluggable) | 0.9707 | 0.2678 | 0.5576 | 0.3508 | 0.8589
Our (unpluggable) | 0.9297 | 0.4158 | 0.6198 | 0.4896 | 0.7805

  * •

value: top 2 results.

Similar to previous tasks, our model achieved very competitive performance.
Specifically, the “pluggable” way of our method comes to the top in AUC and
Hits@5. the “unpluggable” way of our method performs the best in AP, MRR, and
Hits@1. We also observe that in some metrics such as AUC, MRR, and Hits@1, the
rest baselines can also achieve very competitive performance (usually second
the best), but we still hold a strong position and keep very close to them.
This task further demonstrates that our model is very reliable even in
unbalanced data distribution.

####  4.4.4 Effectiveness Analysis on Fake News Identification

In online social networks, fake news is usually spread via users’ retweeting
action, making the identification more like a graph classification task where
each graph is a retweeting network and nodes in the graph are users. Here we
take the default setting of the FakeNews dataset where each node in the graph
has a 310-dimensional feature as input. The initial feature contains 10
Twitter user profile attributes and a 300-dimensional representation of the
user’s historical tweets encoded by the word2vec model.

TABLE V: Fake News Identification ($30\%$ labeled)

| AUC | AP | F1 | Precision | Recall
---|---|---|---|---|---
metapath2vec | 0.8873 | 0.8387 | 0.7904 | 0.8582 | 0.7325
Confluence | 0.8525 | 0.8237 | 0.8149 | 0.8365 | 0.7944
OCCFRP | 0.8936 | 0.8364 | 0.8857 | 0.8972 | 0.8744
NCF | 0.9003 | 0.8537 | 0.9053 | 0.9106 | 0.9001
DeepInf | 0.9106 | 0.8865 | 0.8751 | 0.8573 | 0.8936
SIDM | 0.9264 | 0.9036 | 0.9045 | 0.8864 | 0.9233
Our (pluggable) | 0.9170 | 0.8911 | 0.8770 | 0.8688 | 0.8854
Our (unpluggable) | 0.9507 | 0.9525 | 0.9125 | 0.8957 | 0.9299

  * •

value: top 2 results.

As shown in Table V, our models keep the best performance in AUC, AP, F1, and
Recall. In particular, the “pluggable” way of our method means we first obtain
learned node-level embeddings and then get graph-level representation by graph
pooling. Afterward, we feed these graph representations to a graph classifier
to predict whether it belongs to fake news or not. The “unpluggable” way of
our method means we concatenate the graph classifier directly to our node-
level learning and jointly train the total workflow via the combination of
both self-supervised loss and cross-entropy loss. Although the “pluggable” way
lost the top 2 positions, it still keeps a very close performance to the
others. Since it can learn node embeddings independent from the specific task,
we would directly leverage them for other similar tasks without training from
scratch.

###  4.5 Visualization of Learned Embeddings

In this section, we select five groups in the PersonalityCafe dataset and
compare the discrimination capability of our model with other classic
embedding routines. Note that for fair competition, these five groups are not
found by our model but the original nature groups in the forum. We draw the
representation distributions by t-SNE and present the results in Figure 5. The
initial features (Figure 5a) are encoded by the metapath2vec model on the
pair-wise links with meta path “user-quote-user”, and then we feed them to the
rest routines such as AutoEncoder (Figure 5b), GCN (Figure 5c), and our model
(Figure 5d). From the comparison, we can find that our model learns higher-
quality representations for the users and the resolution is better than the
rest routines.

(a) cluster

(b) louvain community

(c) one-hop neighbour

(d) Ours

Figure 6: Hyperedge Structure

###  4.6 Effectiveness on Social Environment Awareness

To evaluate the hyperedge quality of our model, we present the hyperedge
structure on the KaraClub dataset using our model, and the other three mostly
used hyperedge construction methods including attribute clustering, community
finding, and one-hop neighbor. Results are shown in Figure 6, from which we
can find that the clustering-based method is usually not flexible to find
diversified hyperedges. As presented in Figure 6a, many hyperedges via
clustering are heavily overlapped and this method also ignore the connectivity
of pair-wise relations. Compared with the clustering-based method, the
community-based method (Figure 6b) can construct more hierarchical hyperedges
because we can recursively call back the community finding model within a
relatively large community to generate more sub-communities. However, this
method often leaves many isolated nodes or generates some very small-sized
groups, which may limit the hypergraph learning performance. Besides,
hyperedge overlapping of this method is usually insufficient to support
information diffusion on the hypergraph. According to Figure 6b, neighbor-
based hyperedges usually have much hyperedge overlapping and also keep
relatively hierarchical structures. However, this method usually generates
superfluous hyperedges, which may decrease efficiency. Compared with the above
methods, our model integrates node attributes and their connectivity and the
hyperedges are more concise than the neighbor-based method but still keep the
better overlapping quality and hyperedge sizes than cluster-based and
community-based methods.

We further analyze the hyperedge size distribution on a larger facebook
dataset and compare different hypergraph constructions in Figure 7, from which
we can find that our method keeps a balanced distribution between one-hop
neighbors and attribute clustering, and in the meanwhile present more
significant long-tailed quality than Louvain community finding.

Figure 7: Hyperdge Distributions

###  4.7 Ablation Study

####  4.7.1 Convergence Analysis on Negative Contrastive loss

We change negative ratios in our contrastive dual loss as one positive sample
with 10, 50, and 100 negatives. Then we repeat each type training $10$ times
and draw the loss reducing curve as shown in Figure 8 where each dark colored
line denotes the average loss value as training iteration increases and the
shallow color band is the 65% confidential band. From the figure, we can find
that our model can efficiently come to convergence within 200 iterations with
10 negatives. When the negative ratio increased, the convergence state might
come up a little late but can be still reached within 400 epochs.

Figure 8: Training Loss w.r.t Negative Ratio

####  4.7.2 Effectiveness Analysis on Main Components

To further demonstrate the effectiveness of our model, we remove several main
components and compare the following model variants: Our full model with
contrastive dual loss (Variant1) and normal reconstruction loss (Variant4);
GCN backend alone with contrastive dual loss (Variant2) and normal
reconstruction loss (Variant5); Hypergraph Convolution Network (HCN) backend
alone with contrastive dual loss (Variant3) and normal reconstruction loss
(Variant6). As shown in Figure 9, different backends with contrastive dual
loss outperform the performance with normal reconstruction loss, which
suggests the contribution of our proposed contrastive dual loss. We also find
that our full model performs better than either the GCN backend or the HCN
backend, which further demonstrates the effectiveness of our graph neural
network model.

Figure 9: Effectiveness on Main Components

###  4.8 Analysis on Social Equivalence and Conformity

In Figure 10a, we visualize user environments in Facebook datasets where red
nodes with closer distance indicate they are closely connected in testing data
and black nodes with far distance are also far away in the real world. For a
clear illustration, we did not show the other nodes. From this figure, we can
find that those closely connected pairs are usually located in similar
environments. On the contrary, users far away from each other usually have
different environments. This observation indicates that our model can reflect
the social equivalence and successfully integrate this sociological criterion
into model learning.

(a) Social Equavelence.

(b) Social Conformity.

Figure 10: Social Equivalence and Conformity by our Model

In Figure 10b, we present five groups (hyperedges) and their members on the
PersonalityCafe dataset. Here yellow color denotes the majority of personality
labels in a hyperedge, the green color is similar to personality labels, and
other dark colors denote the distinct labels. From this figure we can observe
that most members in a found hyperedge intend to be similar w.r.t their
personality traits, suggesting the natural social conformity during the model
learning.

###  4.9 Simulated Social Environment Evolving

We select 3 groups in the Facebook dataset and present the simulated group
evolving patterns in Figure 11 where the x-axis denotes the evolving stage,
which is simulated by different learning epochs; the y-axis denotes the ratio
of significant node number out of the original nodes in the hyperedge. Here
the “significant node” means we first use user representations at different
stages to multiply the hyperedge representation (such as the average of the
total members); then we get a probability of this user belonging to this
group; at last we select the significant node when the probability is larger
than $0.5$. The size of each bubble denotes the absolute number of significant
nodes at a specific step. The orange hyperedge has very few connections to
other hyperedges (less overlapping), while the green and the blue hyperedges
have many connections to other hyperedges (more overlapping).

Figure 11: Simulated Analysis on Group Evolving.

For clear illustration, we only present the selected hyperedges, from which we
can observe that the orange group increasingly attracts its members to be more
loyal because it is more irreplaceable than the other groups. The rest groups
have more overlapping with the other groups, making their positions more
alternative. Therefore the ratio of “loyal” members keeps stable after several
evolving stages.

###  4.10 Analysis on Social Polarization

Here we randomly selected five groups on the PersonalityCafe and then we draw
the member uncertainty distribution within each group as shown in Figure 12.
The overall uncertainty in a group can be measured by the group entropy shown
in Table VI. The higher group entropy means this group members are mostly not
that confident about their belief in the group. On the contrary, the lower
group entropy denotes the significance of social polarization.

(a) Initial Stage

(b) Updated Stage

Figure 12: Social Polarization: An Entropy Perspective. TABLE VI: Comparison
of Group Entropy

| G2 | G15 | G23 | G29 | G43
---|---|---|---|---|---
Initial | 963 | 2820 | 896 | 222 | 218
Updated | 443 | 1166 | 404 | 110 | 113

##  5 related works

Online behavior analysis plays a crucial role in many web-based applications
such as recommendation [35], network alignment [36], anomaly detection [37,
38], etc. Since most online users interact with each other via various social
relations, there are many works trying to organize them as graphs, leading to
a huge number of graph-based models applied. In recent years, the mainstream
models are graph neural networks such as GCN [27], GAT [39], GRN [40], and
their extensions for heterogeneous [41] and dynamic cases [42]. Usually, graph
neural network aims at learning low-dimensional representations for graph
nodes, which can support various downstream tasks. In particular, for some
graph-level tasks [30], we usually take graph pooling operation [43] on nodes
and get the graph-level representation; for edge-level tasks, we can design a
transition [44] on node pairs to get the edge embedding; for node-level tasks,
the learned node representation can be directly sent to downstream component
[41]. These task types cover the most common situations in online behavior
analysis such as fake news detection [30], recommendation [15], link
prediction [44], node classification [39] et al., making the graph structural
data widely used in this field. Beyond normal pair-wise relations, online
social networks also have rich higher level relations which might be not
directly observed. Recently, hypergraphs [1, 23] have attracted more attention
because the hyperedge can connect multiple nodes, which is perfect to learn
more latent higher-order interactions. Unfortunately, most social networks do
not directly present their hypergraph structure, leaving the hyperedge
awareness an open problem [26, 24]. Even with effective hypergraph models, we
can only learn the interaction between users and hyperedges [25] alone, which
is insufficient to learn many complicated patterns in sociological criteria.
Besides data mining techniques, there are lots of advanced studies in
sociological areas [45], which can be very promising to further enhance the
data mining research. However, sociological studies are very broad and it is
very crucial to apply clearly defined theories and quantitative methods to
understand the social world because online data analysis might be severely
restricted if they are based on a flawed understanding of sociological theory.
Some recent sociological works such as Markovsky and Webster [46, 47] further
systematically study the proper sociological theory construction.

##  6 Discussion and Conclusion

In this paper, we propose an effective social environment awareness method
that can go beyond classic pair-wise relations. The social environments
combine pair-wise relations, hyperedges, and the line graph to the hypergraphs
with a novel social influence flowing model that can preserve rich
sociological criteria. We further propose a package solution to evaluate
widely concerned sociological criteria. Extensive experiments from both the
data mining field and the sociological field strongly support the
effectiveness of our framework. It is worth noting that sociological research
on user behavior is a wide research field and there are more thorough theories
proposed recently. Data analysis might be limited if it is only based on some
partial understanding of sociological theory. Through our effort in this
paper, we wish to inspire more advanced work from other researchers that can
fuse more thorough sociological theory with data analysis techniques.

## Acknowledgments

The work was supported by grants from the Research Grant Council of the Hong
Kong Special Administrative Region, China (Project No. CUHK 14217622), NSFC
Grant No. U1936205 and CUHK Direct Grant No. 4055159. National Key Research
and Development Program of China (2022YFB4500300). NSFC Grant No. 61972087.
Australian Research Council Future Fellowship (Grant No. FT210100624) and
Discovery Project (Grant No. DP190101985). The first author, Dr. Xiangguo Sun,
in particular, wants to thank his parents for their kind support during his
tough period.

## References

  * Li et al. [2021] Y. Li, H. Chen, X. Sun _et al._ , “Hyperbolic hypergraphs for sequential recommendation,” in _CIKM_ , 2021, pp. 988–997.
  * Wei et al. [2017] H. Wei, F. Zhang, N. J. Yuan _et al._ , “Beyond the words: Predicting user personality from heterogeneous information,” in _WSDM_ , 2017, pp. 305–314.
  * Liu et al. [2020] B. Liu, X. Sun, Z. Ni _et al._ , “Co-Detection of crowdturfing microblogs and spammers in online social networks,” _World Wide Web_ , vol. 23, pp. 573–607, 2020.
  * Myers and Twenge [2022] D. G. Myers and J. M. Twenge, _Social psychology_. New York: McGraw Hill Education, 2022.
  * Patrick Doreian [1988] Patrick Doreian, “Equivalence in a social network,” _The Journal of Mathematical Sociology_ , vol. 13, no. 3, pp. 243–281, 1988.
  * Hanneman and Riddle [2005] R. Hanneman and M. Riddle, _Introduction to social network methods_. University of California, 2005.
  * Newman [2010] M. Newman, _Networks: An Introduction_. Oxford University Press, Mar. 2010.
  * Cialdini and Goldstein [2004] R. B. Cialdini and N. J. Goldstein, “Social influence: Compliance and conformity,” _Annual review of psychology_ , vol. 55, no. 1, pp. 591–621, 2004.
  * Cherng et al. [2022] F.-Y. Cherng, J. Fang, Y. Jiang _et al._ , “Understanding Social Influence in Collective Product Ratings Using Behavioral and Cognitive Metrics,” in _CHI_ , 2022, pp. 464:1–464:16.
  * Deutsch and Gerard [1955] M. Deutsch and H. B. Gerard, “A study of normative and informational social influences upon individual judgment.” _The journal of abnormal and social psychology_ , vol. 51, no. 3, p. 629, 1955.
  * Xu et al. [2021] D. Xu, C. Ruan, E. Korpeoglu _et al._ , “Rethinking Neural vs. Matrix-Factorization Collaborative Filtering: the Theoretical Perspectives,” in _ICML_ , 2021, pp. 11 514–11 524.
  * Yu et al. [2022] J. Yu, H. Yin, X. Xia _et al._ , “Self-Supervised Learning for Recommender Systems: A Survey,” _TKDE_ , vol. early access, 2022\.
  * Yang et al. [2022] M. Yang, M. Zhou, J. Liu _et al._ , “HRCF: Enhancing Collaborative Filtering via Hyperbolic Geometric Regularization,” in _The Web Conference_ , 2022, pp. 2462–2471.
  * Guo et al. [2017] H. Guo, R. Tang, Y. Ye _et al._ , “DeepFM: A Factorization-Machine based Neural Network for CTR Prediction,” in _IJCAI_ , 2017, pp. 1725–1731.
  * Xie et al. [2021] Q. Xie, Y. Zhu, J. Huang _et al._ , “Graph neural collaborative topic model for citation recommendation,” _TOIS_ , vol. 40, no. 3, pp. 1–30, 2021\.
  * Chen et al. [2021] H. Chen, S. Shi, Y. Li _et al._ , “Neural Collaborative Reasoning,” in _The Web Conference_ , Apr. 2021, pp. 1516–1527.
  * Tang et al. [2013] J. Tang, S. Wu, and J. Sun, “Confluence: conformity influence in large social networks,” in _SIGKDD_ , 2013, pp. 347–355.
  * Jiang et al. [2016] B. Jiang, J. Liang, Y. Sha _et al._ , “Retweeting Behavior Prediction Based on One-Class Collaborative Filtering in Social Networks,” in _SIGIR_ , 2016, pp. 977–980.
  * Qiu et al. [2018] J. Qiu, J. Tang, H. Ma _et al._ , “DeepInf: Social Influence Prediction with Deep Learning,” in _SIGKDD_ , Jul. 2018, pp. 2110–2119.
  * Feng et al. [2018] S. Feng, G. Cong, A. Khan _et al._ , “Inf2vec: Latent Representation Model for Social Influence Embedding,” in _ICDE_ , Apr. 2018, pp. 941–952.
  * Yin et al. [2019] H. Yin, Q. Wang, K. Zheng _et al._ , “Social Influence-Based Group Representation Learning for Group Recommendation,” in _ICDE_ , 2019, pp. 566–577.
  * Wang et al. [2020] H. Wang, Q. Meng, J. Fan _et al._ , “Social influence does matter: User action prediction for in-feed advertising,” in _AAAI_ , vol. 34, 2020, pp. 246–253.
  * Zhang et al. [2020] Y. Zhang, N. Wang, Y. Chen _et al._ , “Hypergraph label propagation network,” in _AAAI_ , vol. 34, no. 04, 2020, pp. 6885–6892.
  * An et al. [2021] G. An, Y. Huo, and S.-E. Yoon, “Hypergraph propagation and community selection for objects retrieval,” in _NeurIPS_ , vol. 34, 2021, pp. 3596–3608.
  * Feng et al. [2019] Y. Feng, H. You, Z. Zhang _et al._ , “Hypergraph neural networks,” in _AAAI_ , vol. 33, 2019, pp. 3558–3565.
  * Sun et al. [2021] X. Sun, H. Yin, B. Liu _et al._ , “Multi-level hyperedge distillation for social linking prediction on sparsely observed networks,” in _the Web Conference_ , 2021, pp. 2934–2945.
  * Kipf and Welling [2017] T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” _ICLR_ , 2017.
  * He et al. [2016] D. He, Y. Xia, T. Qin _et al._ , “Dual learning for machine translation,” in _NeurIPS_ , vol. 29, 2016.
  * Yang et al. [2020] R. Yang, J. Shi, X. Xiao _et al._ , “Scaling attributed network embedding to massive graphs,” _VLDB_ , vol. 14, no. 1, pp. 37–49, 2020.
  * Dou et al. [2021] Y. Dou, K. Shu, C. Xia _et al._ , “User preference-aware fake news detection,” in _SIGIR_ , 2021, pp. 2051–2055.
  * Zachary [1977] W. W. Zachary, “An information flow model for conflict and fission in small groups,” _Journal of anthropological research_ , vol. 33, no. 4, pp. 452–473, 1977.
  * Dong et al. [2017] Y. Dong, N. V. Chawla, and A. Swami, “metapath2vec: Scalable representation learning for heterogeneous networks,” in _SIGKDD_ , 2017, pp. 135–144.
  * He et al. [2017] X. He, L. Liao, H. Zhang _et al._ , “Neural Collaborative Filtering,” in _The Web Conference_ , 2017, pp. 173–182.
  * Li et al. [2019a] B. Li, Y. Liu, and X. Wang, “Gradient harmonized single-stage detector,” in _AAAI_ , vol. 33, no. 01, 2019, pp. 8577–8584.
  * Xia et al. [2021] X. Xia, H. Yin, J. Yu, Q. Wang, L. Cui, and X. Zhang, “Self-supervised hypergraph convolutional networks for session-based recommendation,” in _AAAI_ , vol. 35, no. 5, 2021, pp. 4503–4511.
  * Trung et al. [2020] H. T. Trung, T. Van Vinh, N. T. Tam, H. Yin, M. Weidlich, and N. Q. V. Hung, “Adaptive network alignment with unsupervised and multi-order convolutional networks,” in _ICDE_. IEEE, 2020, pp. 85–96.
  * Tam et al. [2019] N. T. Tam, M. Weidlich, B. Zheng, H. Yin, N. Q. V. Hung, and B. Stantic, “From anomaly detection to rumour detection using data streams of social platforms,” _VLDB_ , vol. 12, no. 9, pp. 1016–1029, 2019.
  * Tang et al. [2022] J. Tang, J. Li, Z. Gao, and J. Li, “Rethinking graph neural networks for anomaly detection,” in _International Conference on Machine Learning_ , 2022\.
  * Veličković et al. [2018] P. Veličković, G. Cucurull, A. Casanova _et al._ , “Graph attention networks,” in _ICLR_ , 2018.
  * Dai et al. [2018] H. Dai, Z. Kozareva, B. Dai _et al._ , “Learning steady-states of iterative algorithms over graphs,” in _International conference on machine learning_. PMLR, 2018, pp. 1106–1114.
  * Wang et al. [2019] X. Wang, H. Ji, C. Shi _et al._ , “Heterogeneous graph attention network,” in _The Web Conference_ , 2019, pp. 2022–2032.
  * Li et al. [2019b] J. Li, Z. Han, H. Cheng, J. Su, P. Wang, J. Zhang, and L. Pan, “Predicting path failure in time-evolving graphs,” in _SIGKDD_ , 2019, pp. 1279–1289.
  * Mesquita et al. [2020] D. Mesquita, A. Souza, and S. Kaski, “Rethinking pooling in graph neural networks,” in _NeurIPS_ , 2020, pp. 2220–2231.
  * Qu et al. [2020] L. Qu, H. Zhu, Q. Duan _et al._ , “Continuous-time link prediction via temporal dependent graph neural network,” in _The Web Conference_ , 2020, pp. 3026–3032.
  * Freese and Peterson [2017] J. Freese and D. Peterson, “Replication in social science,” _Annual Review of Sociology_ , vol. 43, no. 1, pp. 147–165, 2017.
  * Markovsky and Webster JR. [2015] B. Markovsky and M. Webster JR., _Theory Construction_. John Wiley & Sons, Ltd, 2015.
  * Webster Jr and Walker [2022] M. Webster Jr and L. S. Walker, _Unequals: The Power of Status and Expectations in Our Social Lives_. Oxford University Press, 2022.

|  Xiangguo Sun works as a postdoctoral research fellow at the Chinese
University of Hong Kong and works with Prof. Hong Cheng. He is also a visiting
researcher at the Research Institute of Artificial Intelligence in Zhejiang
Lab hosted by Dr. Hongyang Chen. He received his Ph.D. from Southeast
University in Jan 2022 under the supervision of Prof. Bo Liu. During his Ph.D.
study, he worked as a research intern in Microsoft Research Asia with Dr. Hang
Dong and Bo Qiao, and visited the University of Queensland as a research
scholar hosted by Prof. Hongzhi Yin. His research interests include social
media analytics and hypergraph learning. His work has been published in some
of the most prestigious venues such as SIGKDD, VLDB, TKDE, TNNLS, The Web
Conference (WWW), TOIS, WSDM, CIKM etc.
---|---
|  Hong Cheng is a Professor in the Department of Systems Engineering and
Engineering Management, Chinese University of Hong Kong. She received the
Ph.D. degree from the University of Illinois at Urbana-Champaign in 2008. Her
research interests include data mining, database systems, and machine
learning. She received research paper awards at ICDE’07, SIGKDD’06, and
SIGKDD’05, and the certificate of recognition for the 2009 SIGKDD Doctoral
Dissertation Award. She received the 2010 Vice-Chancellor’s Exemplary Teaching
Award at the Chinese University of Hong Kong.
---|---
|  Bo Liu is a full professor at the School of Computer Science and
Engineering, Southeast University, China. She received her doctoral degree
from Southeast University under the supervision of Chief Scientist Prof.
Junzhou Luo. She is the leader of three NSF projects of China and has
published more than 60 papers in reputed journals and conferences including
WWW, TKDE, WWWJ, TOIS, ToN et al. Her research interests include anomaly
detection, social community, social influence, and recommendation.
---|---
|  Jia Li is an assistant professor in HKUST (Guangzhou). He received Ph.D.
degree at The Chinese University of Hong Kong in 2021. Before that, he worked
as a full-time data mining engineer at Tencent from 2014 to 2017, and research
intern at Google AI (Mountain View) in 2020. His research interests include
graph learning and data mining. Some of his work has been published in TPAMI,
ICML, NeurIPS, WWW, KDD and et al.
---|---
|  Hongyang Chen is the deputy director and senior research expert for the
research center of graph computing, Zhejiang Lab, China. He was the
Distinguished Lecturer of the IEEE Communication Society from 2021 to 2022.
Prof. Chen is an adjunct professor at Hangzhou Institute for Advanced Study,
The University of Chinese Academy of Sciences, and Zhejiang University, China.
He received his Ph.D. degree from the University of Tokyo, Japan in 2011. He
has published more than 120 top-tier papers related to data-driven intelligent
systems, graph machine learning, big data mining, and intelligent computing.
Most of them are published in SIGMOD, AAAI, TKDE, JSAC, CIKM, etc. He has more
than 30 international patents, many of which have been adopted as
international standards.
---|---
|  Guandong Xu is a full professor and leader in the Data Science and Machine
Intelligence Lab at the University Technology of Sydney. He received Ph.D.
degree from Victoria University, Australia. He was the winner of the 2020
Global Efma-Accenture Insurance Innovation Award in Workforce Transformation
and the Digital Disruptors Winner for Skills Transformation of Small Work
Teams, Australian Computer Society (2019). He has more than 220 papers
published in top journals and conferences such as TOIS, TNNLS, KDD, AAAI,
ICDM, IJCAI and et al.
---|---
|  Hongzhi Yin works as ARC Future Fellow and associate professor with The
University of Queensland, Australia. He was recognized as Field Leader of Data
Mining & Analysis in The Australian’s Research 2020 magazine, the recipient of
the 2022 AI 2000 Most Influential Scholar Honorable Mention in Data Mining (10
Years Impact), and featured among the World’s Top 2% Scientists List published
by Stanford University (Single Year Impact) and AD Scientific Index 2022 (5
Years Impact). His current research interests include recommender systems,
graph embedding, and mining, chatbots, social media analytics, and mining,
edge machine learning, trustworthy machine learning, decentralized and
federated learning, and smart healthcare. He has published 220+ papers with
H-index 52, including 22 most highly cited publications in Top 1% (CNCI), 120
CCF A and 70+ CCF B, 120 CORE A* and 70+ CORE A, such as KDD, SIGIR, WWW,
WSDM, SIGMOD, VLDB, ICDE, AAAI, IJCAI, ACM Multimedia, ECCV, IEEE TKDE and
TNNL, VLDB Journal and ACM TOIS. He has won 6 Best Paper Awards, such as Best
Paper Award at ICDE 2019, Best Student Paper Award at DASFAA 2020, ACM
Computing Reviews’ 21 Annual Best of Computing Notable Books and Articles, and
Best Paper Award Nomination at ICDM 2018 (being invited to be published in
KAIS special issue of best papers).
---|---