# Knowledge Consensus in complex networks: the role of learning

Zhong-Yan Fan  Department of Electronic Engineering, City University of Hong
Kong, Hong Kong<EMAIL_ADDRESS>Ying-Cheng Lai  School of
Electrical, Computer and Energy Engineering, Arizona State University, Tempe,
AZ 85287, USA  Wallace K. S. Tang  Department of Electronic Engineering, City
University of Hong Kong, Hong Kong

###### Abstract

To reach consensus among interacting agents is a problem of interest for
social, economical, and political systems. A computational and mathematical
framework to investigate consensus dynamics on complex networks is naming
games. In general, naming is not an independent process but relies on
perception and categorization. Existing works focus on consensus process of
vocabulary evolution in a population of agents. However, in order to name an
object, agents must first be able to distinguish objects according to their
features. We articulate a likelihood category game model (LCGM) to integrate
feature learning and the naming process. In the LCGM, self-organized agents
can define category based on acquired knowledge through learning and use
likelihood estimation to distinguish objects. The information communicated
among the agents is no longer simply in some form of absolute answer, but
involves one’s perception. Extensive simulations with LCGM reveal that a more
complex knowledge makes it harder to reach consensus. We also find that agents
with larger degree contribute more to the knowledge formation and are more
likely to be intelligent. The proposed LCGM and the findings provide new
insights into the emergence and evolution of consensus in complex systems in
general.

##  1 Introduction

To reach consensus among a population of agents is a problem with significant
applications in social, economical, and political systems. A mathematical and
computational paradigm to describe, characterize, and understand consensus
dynamics is naming game (NG) and its variants [1, 2, 3, 4, 5, 6, 7, 8, 9, 10,
11, 12, 13, 14, 15, 16, 17, 18]. In an NG, agents attempt to reach consensus
about a certain object or event via interactions. Most existing works
emphasize on the consensus of vocabulary among the agents [1, 2, 3, 4, 5, 6,
7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]. While vocabulary is often related to
objects and plays an important role in the development of natural language,
for intelligent agents, another determining factor in NG is the contents and
features of the objects. In fact, a pivotal cognitive ability of human being
is categorization that recognizes, characterizes, and eventually names objects
according to their features. In our daily activities, majority of the naming
actions are based on a category rather than a specific term or vocabulary. For
example, the word “cat” refers to a category in which objects have same
features rather than a specific cat “Tom” or “Kitty.” For NG to better
describe consensus dynamics in the real world, it is necessary to incorporate
catagorization through learning and knowledge growth into the model. While
there were previous works on categorization games that deal with feature
recognition and analysis [19, 20, 21, 22, 23], the process of categorization
itself remains to be an outstanding topic of research, due to the complexity
of the underlying process. It is thus a challenging task to incorporate
perception and categorization into NGs. The aim of this paper is to introduce
a new kind of category game model to address this problem.

In previous works on NGs, consensus is achieved through some learning
processes solely governed by interactions between the agents [8]. Therefore,
topological features of the agents’ interaction network, such as degree,
clustering coefficient, and path distance, have significant influences, and it
is concluded that higher degrees, lower clustering and shorter network
distance tend to promote consensus [3, 1, 2, 4, 5]. The knowledge transfer in
NGs is not limited between two agents, for example, Li et al. [9] suggested a
model with multiple hearers while Gao et al [11] considered negotiation to
take place between multiple indirectly connected agents. An agent can also
play the roles of a speaker and a hearer simultaneously [11]. The more general
setting was also studied in which the agents possess different propensities
such as commitment [14, 12] and stubbornness [6, 10], with the finding that
the learning behaviors of these agents can affect significantly the consensus
dynamics over the whole population. In addition, other issues for more
realistic agents, such as learning errors during interactions [13], memory
loss [15] and multi-word/language [16, 17], have been investigated. These
works provide great insights into consensus dynamics. However, since only
vocabulary is focused, significant feature-based cognitive abilities such as
categorization were not taken into account which, as natural intuition would
suggest, may play a more significant role in the emergence and evolution of
consensus.

There were a few proposals on categorization games. For example,
discrimination and guessing games were designed to accomplish the task of
categorization [19]. In a discrimination game, a speaker is trained with
ground truth so that it can relate an object to the actual category using a
classifier. Guessing game is similar to a typical NG, however, the hearer not
only acquires the name but also updates his/her classifier for the category as
instructed by the speaker. Based on these two types of games, the problem of
color categorization was studied [19], and a category game model (CGM) was
developed [20]. In a game defined by CGM, a pair of objects are presented to a
speaker and a hearer, and the target topic (the object to be learned) is
selected by the speaker. If both objects are distinguishable to the speaker,
one is randomly chosen as the target topic. Otherwise, the speaker must
discriminate the two objects by creating a new boundary between them before
selecting one as the target topic. The interaction between the speaker and the
hearer then makes it feasible to learn and name the target topic. In the CGM
framework, factors such as language aging [21], persistence [22], and
individual biases [23] can be studied. A feature common to both discrimination
game and CGM is that certain pre-requisites are needed. Specifically, in a
discrimination game, it is necessary to relate the object to an actual
category, while in CGM, the two presented objects are assumed to belong to two
different categories. In the real world, it often occurs that a population can
reach an agreement without being given any ground truth, and this has
consequences. For example, different language systems can give different color
categorization, and the consensus of opinions among practitioners in a
financial market can trigger a herd behavior that leads to the fat-tail
distributions in prices and returns [24].

To understand and exploit consensus dynamics as realistically as possible
requires the development of a more comprehensive type of game models centered
about features. The base of our model is the recently proposed domain learning
naming game (DLNG) [18] to solve the color categorization problem through
elimination of pre-requisites in the learning process. The color perception
process in DLNG follows a variant algorithm based on the majority rule. Agents
in DLNG have numerous sensors uniformly distributed in the domain, and one
sensor can be dominated by at most one category. Then, an object is deemed as
in a category if the majority of sensors near the object belong to that
category. However, the original DLNG is a kind of coarse-grained model where a
determined category may contain distinct objects. For example, intelligent
agents such as humans are generally capable of assessing that crimson is much
redder than magenta, although both colors belong to the same category of red.

To overcome this difficulty, here we propose a likelihood category game model
(LCGM), in which self-organized agents define category based on acquired
knowledge and use likelihood estimation to distinguish the objects in the same
category. The information communicated among the agents is no longer simply
some absolute answer but involves agents’ perception. That is, the agents are
able to classify objects in terms of distinct likelihoods as determined by
their knowledge and update knowledge through learning new objects. For the
special case where the learning domain is highly localized, our model reduces
to a variant of the minimum NG. The primary contribution of this paper is the
proposed LCGM, with a new concept of likelihood to allow agents assess an
object via perception, which better reflect our real experiences. Moreover,
conclusive remarks related to consensus characteristics are provided. Firstly,
it is noticed that short distance and heterogeneity can facilitate the
consensus. Secondly, LCGM clearly reflects a more complex knowledge makes it
harder to reach consensus. Lastly, agents with larger degree contributes more
to the knowledge formation and consensus and accordingly to be more
“intelligent”. Our work in LCGM provides novel insights into the process of
consensus, with potential applications to predicting and controlling consensus
– problems with implications to social, economical, and even political systems
where achieving consensus is often a desired objective.

##  2 Model

###  2.1 Agent model

An object $o_{i}$ in our LCGM is represented by a point in an $N$-dimensional
domain [20], that is
$x_{i}\equiv\left(x_{i,1},x_{i,2},\cdots,x_{i,N}\right)\in D\subseteq\Re^{N}$.
Each agent $j$ maintains a set of categories $\mathcal{C}_{j}$ in its memory.
Each category, $c_{k}\in\mathcal{C}_{j}$, is described by a weight
$\omega_{k}$ and $N$’s normal distributions
$\mathcal{N}(\mu_{k,n},\sigma_{k,n}^{2})$ for the corresponding $n$ dimension
with $n=1,2,\cdots,N$. An agent, $\mbox{Agent}_{j}$, classifies the object
$o_{i}$ based on a likelihood score that characterizes how likely $o_{i}$
belongs to category $c_{k}$. The score can be calculated through

$LS_{j}(o_{i},c_{k})=g(\omega_{k})\prod_{n=1}^{N}f\left(\frac{x_{i,n}-\mu_{k,n}}{\sigma_{k,n}}\right)/f(0)$
(1)

where $f(\cdot)$ is the probability density function (PDF) of the standardized
normal distribution and $g(\cdot)$ is a scale function defined as

$g(\omega)=\tanh(\alpha\omega),$ (2)

where $\alpha$ is a constant. To be concrete, we set $\alpha=2.5$, so
$g(1)=0.987\approx 1.0$. Equation (1) indicates that the likelihood score
contains two components: prior and likelihood. The prior component,
represented by $g(\omega_{k})$, indicates the probability that category
$c_{k}$ is selected for an arbitrary object. The likelihood component is
specified by the product term in Eq. (1). The normalized value
$f\left(\frac{x_{i,n}-\mu_{k,n}}{\sigma_{k,n}}\right)/f(0)$ measures the
likelihood of $o_{i}$ belonging to category $c_{k}$ along the $n$th dimension.
In the absence of any correlation among different dimensions, the product term
gives the joint probability value.

In our model, each agent can execute four possible activities: object
identification, category updating, category creation, and category deletion,
which are described, as follows.

Objective identification activity (OIA). Given an object $o_{i}$,
$\mbox{Agent}_{j}$ returns $LS_{j}(o_{i},c_{a})$ and the corresponding
$\mbox{name}_{a}$, where $c_{a}=\arg\underset{c_{x}\in\mathcal{C}_{j}}{\max}\
LS_{j}(o_{i},c_{x})$ is the category having the highest score. For
$\mathcal{C}_{j}=\emptyset$, $\mbox{Agent}_{j}$ will return a null name
($\mbox{name}_{a}=\mbox{null}$) and a zero likelihood score
[$LS_{j}(o_{i},c_{a})=0$]. In addition, if $LS_{j}(o_{i},c_{a})$ is below a
specified threshold, denoted as $LST$, then the agent will also return a null
name and a zero likelihood score. An illustrative example of OIA for the one-
dimensional case is given in Fig. 1.

Figure 1:  An illustrative example of object identification in one dimension.
Suppose the agent has two categories: “Blue” and “Red”. With regard to objects
“a”, “b”, “c” and “d”, there are four cases ($a$, $b$, $c$ and $d$,
respectively). In case $a$, since $LS(a,\mbox{Blue})>LS(a,\mbox{Red})$ and
$LS(a,\mbox{Blue})>LST$, the agent returns the name of category “Blue” and
$LS(a,\mbox{Blue})$. In case $b$, $LS(b,\mbox{Red})>LS(b,\mbox{Blue})$ and
$LS(b,\mbox{Red})>LST$, the name of category “Red” and $LS(b,\mbox{Red})$ are
returned. In case $c$, $LS(c,\mbox{Blue})$ is higher but still lower than LST,
so the agent returns $\mbox{name}=\mbox{null}$ and $LS=0$. In case $d$,
$LS(d,\mbox{Red})=LS(d,\mbox{Blue})>LST$, resulting in category “Red” or
“Blue” being randomly picked. Case $d$ is rare because it occurs only at the
intersection point of the two curves.

Category updating activity (CUA). If $o_{i}$ is assigned to category $c_{a}$,
the features ($\mu_{a,1}$, $\sigma_{a,1},\cdots,\mu_{a,N}$, $\sigma_{a,N}$ and
$\omega_{a}$) of $c_{a}$ will be updated by merging the information of
$o_{i}$. Since $o_{i}$ is solely represented by the point $x_{i}$ in the
domain $D$, it is reasonable to let the weight of $x_{i}$ be one. As a result,
$\omega_{a}$ is increased by one, forming a new weight

$\omega_{a}^{*}=\omega_{a}+1$ (3)

For each $n$th dimension, the updated mean, $\mu^{*}_{a,n}$, is the weighted
average of the category $c_{a}$ and the point $x_{i}$, which can be obtained
through

$\mu^{*}_{a,n}=\frac{\omega_{a}\times\mu_{a,n}+x_{i,n}}{\omega_{a}^{*}}$ (4)

Finally, the updated $\sigma^{*}_{a}$ can be obtained from the following
equation:

$\sigma_{a,n}^{*}=\max\left(\frac{-|x_{i,n}-\mu_{a,n}^{*}|}{F^{-1}\left(\frac{1}{2\omega_{a}^{*}}\right)},\frac{-|\mu_{a,n}-\mu_{a,n}^{*}|}{F^{-1}\left(\frac{\omega_{a}}{2\omega_{a}^{*}}\right)},\sigma_{a,n}\right),$
(5)

where $F(.)$ is the CDF of $f(\cdot)$. An illustrative example is given in
Fig. 2. Note that $\sigma$ of the category distribution is non-decreasing
because only exceptions will result in its changes while normal observations
will have no effect on $\sigma$.

Figure 2:  An illustrative example of $\sigma$ updating. For each $n$th
dimension, the category $c_{a}$ (left) and object $x_{i}$ (middle) are merged
into a new category $c_{a}^{*}$ (right). Let the parameters be $\omega_{a}=7$
and $\sigma_{a}=1$ for $c_{a}$, and $\omega_{i}=1$ for the object. Equation
(3) thus gives $\omega^{*}_{a}=\omega_{a}+\omega_{i}=8$. For the $n$th
dimension, the shaded areas of $c_{a}$ and $x_{i}$ are $\omega_{a}/2=3.5$ and
$\omega_{i}/2=0.5$, respectively. Consider $S_{1}=\omega^{*}_{a}\times
Pr(x_{j,n}\leq\mu_{a,n})$ and $S_{2}=\omega^{*}_{a}\times
Pr(x_{j,n}\geq\mu_{i,n})$ in the updated category, where $Pr(\cdot)$ stands
for the probability. There are two cases with Eq. (5). First, if $\mu_{a,n}=0$
and $\mu_{i,n}=0.8$, then Eq. (4) gives $\mu_{a,n}^{*}=0.1$ and
$\sigma_{a,n}^{*}$ is kept at the same value as $\sigma_{a,n}$
[$\sigma_{a,n}^{*}=\max(0.456,0.636,1)=1$]. Correspondingly, one has
$S_{1}=3.68>\omega_{a}/2$ and $S_{2}=1.94>1/2$. Second, if $\mu_{a,n}=0$ and
$\mu_{i,n}=2$, then one has $\mu_{a,n}^{*}=0.25$ and
$\sigma_{a,n}^{*}=\max(1.141,1.589,1)=1.589$. As a consequence, one has
$S_{1}=3.5=\omega_{a}/2$ and $S_{2}=1.083>1/2$. Equation (5) stipulates that
the inequalities $S_{1}\geq\omega_{a}/2$ and $S_{2}\geq 1/2$ hold. Note that
object $x_{i}$ (middle) should be a Dirac delta function ($\sigma_{i,n}\to 0$)
at $\mu_{i,n}$, and it is plotted as a bell-shape only for visual clarity.

Category creation activity (CCA). If an object $o_{i}$ with $\mbox{name}_{i}$
concluded in a game cannot be related to any existing category associated with
the agent, a new category $c_{a}$ is created. The parameters $\sigma_{a}$ and
$\omega_{a}$ are set at some default values, while $\mu_{a}=x_{i}$ and
$\mbox{name}_{a}=\mbox{name}_{i}$.

Category deletion activity (CDA). In reality, if knowledge is not recalled, it
will be forgotten gradually. Similarly, in LCGM, category will be removed from
one’s memory if there is no updating, i.e., it has not been learned for a long
time. The removal of a category is based on its weight. For any time $t$, the
weight of category $c_{k}$, $\omega_{k}$ is updated as

$\omega_{k,t}=\omega_{k,t-1}\times e^{-\phi}$ (6)

where $\phi$ is the “forgetting” factor that describes the “forgetting speed”
of a category in the agent’s memory, $\omega_{k,t}$ and $\omega_{k,t-1}$ are
the value of $\omega_{k}$ at time $t$ and $t-1$, respectively.

As Eq. (1) indicates, a reduction in $\omega_{k}$ will also affect the chance
of category $c_{k}$ being assigned in OIA. If $\omega_{k,t}$ further reduces
and becomes smaller than a pre-defined threshold (denoted as $\omega_{Th}$),
the category $c_{k}$ will be removed from the memory of the agent. However,
once an object falls into category $c_{k}$, $\omega_{k}$ becomes larger than
one [Eq. (3)] via CUA.

###  2.2 Game rules.

Game participants are all agents in a population on a network. Initially,
every agent has empty memory: $\mathcal{C}_{j}=\emptyset,\ \forall j$. A pair
of connected agents, $\mbox{Agent}_{A}$ and $\mbox{Agent}_{B}$, are randomly
selected in a game that proceeds as follows.

(1) An object $o_{i}$ is presented to both agents. Via OIA, $\mbox{Agent}_{A}$
returns $\mbox{name}_{a}$ and $LS(x_{i},c_{a})$, while $\mbox{Agent}_{B}$
returns $\mbox{name}_{b}$ and $LS(x_{i},c_{b})$.

(2) $\mbox{Agent}_{A}$ and $\mbox{Agent}_{B}$ conclude the name of $o_{i}$,
referred to as $\mbox{name}_{i}$, based on the following rules: if
$LS(o_{i},c_{a})>LS(o_{i},c_{b})$, $\mbox{name}_{i}=\mbox{name}_{a}$ and
$\mbox{Agent}_{A}$ wins the game; if $LS(o_{i},c_{a})<LS(o_{i},c_{b})$,
$\mbox{name}_{i}=\mbox{name}_{b}$ and $\mbox{Agent}_{B}$ wins the game; if
$LS(o_{i},c_{a})=LS(o_{i},c_{b})=0$, $\mbox{name}_{i}$ is randomly generated
and the game is a draw; if $LS(o_{i},c_{a})=LS(o_{i},c_{b})\neq 0$, a winner
is randomly selected. For example, if $\mbox{Agent}_{A}$ is the winner, one
has $\mbox{name}_{i}=\mbox{name}_{a}$. In addition, if
$\mbox{name}_{a}=\mbox{name}_{b}\neq\mbox{null}$, the game is successful and
$\mbox{name}_{i}=\mbox{name}_{a}$. Otherwise, it has failed.

(3) $\mbox{Agent}_{A}$ and $\mbox{Agent}_{B}$ update/create their categories
according to a set of rules. The rules for $\mbox{Agent}_{A}$ can be described
as follows (similar for $\mbox{Agent}_{B}$). If $\mbox{name}_{i}$ equals the
name of a category, say $c_{k}\in\mathcal{C}_{A}$, CUA is operated on category
$c_{k}$. However, if $\mbox{name}_{i}$ is not contained in any category in
$\mathcal{C}_{A}$, CCA is carried out to create a new category with
$\mbox{name}_{i}$ and $\mu_{i}=x_{i}$.

(4) All agents in the population delete expired categories via CDA.

Two illustrative examples of the game process are given in Fig. 3.

Figure 3:  Two illustrative examples of game process in terms of color
objects. In Game I, via OIA, $\mbox{Agent}_{A}$ considers that the object is
“Blue” with 50% certainty, while $\mbox{Agent}_{B}$ regards it as “Green” with
70% certainty. As $\mbox{Agent}_{B}$ is more confident, the object is
concluded as “Green.” The two agents then update their memories based on the
new knowledge that the object is named “Green.” In addition, both agents
forget categories that have not been recalled for a long period of time. In
Game II, neither agent can identify the object. The agents return
$\mbox{name}=\mbox{null}$ and $LS=0$ via OIA. As a result, a new name, say
“Green,” is assigned. The remaining steps [Steps (3) and (4)] are the same as
those in Game I.

##  3 Methods

We focus on color categorization to demonstrate and analyze LCGM in the
present study. Color objects are randomly generated in $RGB$ form with sample
size $256\times 256\times 256$, and are then mapped into $CIELab$ color space
for agents. Specifically, the color objects and categories are represented in
the $CIELab$ color space which is three dimensional: (L*, a*, b*), where L*
stands for the lightness, a* and b* are the two abstract opponent dimensions
[25]. The $CIELab$ color space is assumed to be homogeneous, i.e., the
difference between two color objects is defined as the Euclidean distance
between them, as specified by the year 1976 version of $CIELab$ color space.
(Remark: To better fit with human vision, $\Delta E$ was re-defined in the
year 2000 version [26] where the color space is no longer homogeneous.
However, this does not affect our main results.)

LCGM is applied to a population of agents connected through a pre-defined
network, either from the real world or from a specific model. Initially, each
agent has an empty memory. Games are conducted as described in Game Rules, and
repeated until a predefined number of iterations is reached. For all the
simulations in this paper, the iteration number is $10^{7}$.

Model parameters are set as follows.

  1. 1.

Scaling factor in Eq. (2): $\alpha=2.5$

  2. 2.

Default value of $\sigma$ in CCA: $\sigma_{default}=5.0$. Empirically, two
colors with the value of $\Delta E$ between 1 and 10 appear similar in human
vision, so we set $\sigma_{default}=5.0$.

  3. 3.

Default value of $\omega$ in CCA: $\omega_{default}=1.0$.

  4. 4.

The threshold parameter $\omega_{Th}$: a category will be deleted during CDA
if its weight $\omega$ is smaller than $\omega_{Th}$. We set
$\omega_{Th}=0.01$.

  5. 5.

Likelihood score threshold (LST): If the LS computed from an object-category
pair is smaller than LST, the object should not belong to that category. We
choose different values for LST to investigate its impact on the learning
process.

  6. 6.

Forgetting factor $\phi$: it characterizes the speed at which a category is
forgotten. We choose different values of $\phi$ to investigate its impact on
the learning process.

We conduct simulations by applying LCGM to two social networks (a subgraph of
Facebook network and a subgraph of E-mail network) and various heterogeneous
networks. The results from the Facebook network and the E-mail network are
averaged by 10 runs. To investigate the effect of network topology, artificial
networks are constructed. For each setting, ten network realizations are used
to calculate the various statistical averages.

##  4 Results

To demonstrate LCGM, we test it on learning color categorization by a
population of agents. We define the following performance metrics.

Accuracy (ACC) indicates the success rate of the games, which is computed for
every $K$ games and defined as:

$ACC=\frac{k_{success}}{K}$ (7)

where $k_{success}$ is the number of successful games in every $K$ iterations.

Number of categories (NC) specifies the number of categories maintained by
each agent, which reflects the resolving power of agents after learning. The
average number of categories ($NC_{Avg}$) is primarily concerned in this
paper.

Total number of distinct names (TDN) records how many distinct names remained
in the whole population.

Consensus score (CS) characterizes the consensus of the whole population. A
set of sampled objects ($\mathcal{O}$) is firstly selected. In this work,
$\mathcal{O}$ consists of 512 objects uniformly sampled from the domain $D$.
These objects are presented to every pair of agents in turns (including agents
who are not neighbors) and games are performed. For $\mbox{Agent}_{i}$ and
$\mbox{Agent}_{j}$ with $i\neq j$, $CS_{i,j}$ is defined as

$CS_{i,j}=\frac{agm_{i,j}}{|\mathcal{O}|}$ (8)

where $agm_{i,j}$ is the total number of successful games that
$\mbox{Agent}_{i}$ and $\mbox{Agent}_{j}$ made for all $o_{i}\in\mathcal{O}$,
and $|\cdot|$ is the cardinality of a set. The $CS$ of the entire population
$\mathcal{V}$ is the average over all possible agent pairs, which can be
computed as

$CS=\frac{\sum_{i}{\sum_{j\neq
i}{CS_{i,j}}}}{|\mathcal{V}|\times(|\mathcal{V}|-1)}$ (9)

###  4.1 Application: consensus in social network

We apply our LCGM algorithm to two social networks, a subgraph of Facebook
social network [27] and a subgraph of an E-mail network [28], collected by the
SNAP laboratory of Stanford university. The Facebook subgraph has 1,034 nodes
and 26,749 edges. Its clustering coefficient, average path length and degree-
based Gini coefficient are $0.5$, $2.9$ and $0.48$, respectively. The E-mail
network has 986 nodes and 25,552 edges. Its clustering coefficient, average
path length and degree-based Gini coefficient are $0.31$, $2.6$ and $0.56$,
respectively. The relatively large value of the clustering coefficient and
short average path length are typical of small-world networks, while the
relatively high values of the Gini coefficient suggest that the networks are
inhomogeneous.

Typical simulations with $LST=0.1$ and $\phi=0.00002$ were conducted. On
Facebook social network, ACC reaches 88.0%, the average NC and TDN are about
4.7 and 6.8, respectively. And on E-mail network, ACC reaches 88.6%, the
average NC and TDN are 4.6 and 5.8, respectively.

However, the high accuracy does not guarantee full consistencies of agents
under the framework of category game. Since accuracy only considers gaming
actions occur between two mutual neighbors, consensus between agents who are
not neighbors within the population is ignored. The consensus score (CS) is
then used to characterize the consistency of the whole population. The average
CS for Facebook social network and E-mail network are about 0.71 and 0.85,
respectively.

The performance of CS was further investigated. The histograms of consensus
score are plotted in Fig. 4 (a) and (b). The majority of agent pairs have CS
higher than 0.8 for both networks. However, some agent pairs have relatively
low CS, particular for the Facebook social network. Then, we analyzed the
influence of distance of agent pairs and observed a negative correlation
between CS and distance of agent pairs (See Fig. 4 (c) and (d)). It is because
agents involved in the transmission path would incorporate their
understandings to the information. As a result, a longer distance between two
agents generally imposes a higher variation in the knowledge between them.

Figure 4:  Consensus score (CS) of agent pairs. Shown are (a) the histogram of
the consensus score of Facebook social network, (b) the histogram of the
consensus score of E-mail network, (c) boxplot of the correlation between
distance and CS of Facebook social network and (d) boxplot of the correlation
between distance and CS of E-mail network.

The adoption of LCGM on the two social networks revealed that agents basically
reach the consensus through the mutual learning, with consensus score 0.71
(the Facebook network) and 0.85 (the E-mail network) respectively. While the
CS of the E-mail network is much higher than that of the Facebook network even
though they have similar ACC. The extended analysis on CS and distance implies
that the shorter average path length of the E-mail networks may facilitate the
consensus. Other topology properties, such as clustering and heterogeneity,
will be discussed later in this section.

Based on LCGM, an interesting question is, among the agents, who more
frequently takes the lead in the game by providing the answer? The leader is
the winner of the game as specified by the game rules. To identify the leader,
we define the following game score for each agent:

$GS(A)=\sum_{i}GS_{i}(A)$ (10)

where $GS_{i}(A)$ is the game score obtained by $\mbox{Agent}_{A}$ in $i$th
game:

$GS_{i}(A)=\begin{cases}1,&\text{if $\mbox{Agent}_{A}$ participated in the
$i$th game and won}\\\ 0.5,&\text{if $\mbox{Agent}_{A}$ participated in the
$i$th game and the result was a draw}\\\ 0,&\text{otherwise.}\end{cases}$ (11)

Equation (11) indicates that our proposed learning game is effectively a
positive-sum game. In each game, the increment of the total score is one. When
an agent wins the game, it dominates the game solely and takes the whole
score. When the game is a draw, both players score 0.5 as they contribute to
knowledge formation equally. In this case, the loser receives no penalty for
the reason that learning should not be discouraged. To eliminate the effect of
opportunity earning (agents involved in more games likely will have higher
scores), we further define the following game scoring rate:

$GSR(A)=\frac{GS(A)}{\\#\mbox{Games}(A)}$ (12)

where $\\#Games(A)$ is the total number of games involving $\mbox{Agent}_{A}$.

As it is similar for both studied social networks, we only discuss the results
of the Facebook social network here. Figure 5 (a) shows the GSR distribution,
giving a bell-shape in which the majority (93.5%) of agents have GSR values
between 0.4 and 0.6. To check whether the distribution is normal, we use the
Quantile-Quantile plot (QQ plot). Specifically, a dot $(x,y)$ in the QQ plot
means $F_{X}(x)=F_{Y}(y)$, where $F_{X}(x)$ is the cumulative density function
of random variable $X$. If two distributions are identical, all dots would be
located along the diagonal. As shown in Fig. 5 (b), dots within $x\in(-2,2)$
are quite close to the diagonal, indicating that the GSR possesses the
characteristic of a normal distribution in this range. Such a bell-shape
distribution reflects most agents have moderate intelligence after sufficient
learning and communications. Furthermore, the emergence of a light right tail
and a heavy left tail observed in Fig. 5 (b) indicates that “geniuses are
minority” and “dummies are more than you expected.”

Figure 5:  Statistical features of game scoring rate for the Facebook social
network. Shown are (a) the histogram of the game scoring rate (GSR) and (b)
the corresponding Quantile-Quantile plot (QQ plot) for $\mu_{GSR}=0.49$ and
$\sigma_{GSR}$ = 0.06.
Figure 6:  Correlation between GS/GSR and degree centrality for the Facebook
social network. The Pearson correlation coefficients are
$\rho_{log(d),GS}=0.60$ and $\rho_{log(d),GSR}=0.40$.

The correlation between GS/GSR and degree is shown in Fig. 6. As defined in
Eqs. (10) and (11), GS reflects the frequency of transmitting self-knowledge
to others. Figure 6(a) shows that agents with large degrees are likely to have
a high GS, implying that agents with larger degree contributed more to the
consensus process. Different from GS, GSR measures the capability of scoring
in a game [Eq. (12)]. Figure 6(b) reveals a positive correlation between GSR
and degree, indicating that agents with larger degrees are more “intelligent”
with respect to eventual knowledge formation through more active learning and
communication.

###  4.2 Effects of Likelihood score threshold (LST) and forgetting factors
(FF)

Then, the effects of likelihood score threshold (LST) and forgetting factors
(FF) of categories were investigated. Again, the studies were based on the
Facebook social network.

Likelihood score threshold of agents. As shown in Fig. 1, LST imposes the
minimum requirement for classifying an object into a category. Accordingly, a
high LST value encourages the generation of a new category. To investigate the
effect of LST, we simulate the learning process for different values of LST
with fixed forgetting factor $\phi=0.00002$ in Eq. (6).

As shown in Table 1 (left), agents with high LST (the “rigorous” agents)
spontaneously possess more categories. And the more complex knowledge makes it
harder to reach consensus, resulting a lower accuracy. This is further
confirmed by the plot of consensus scores with different LST as shown in Fig.
7 (a), which clearly shows that the increment of LST would depreciate CS
significantly.

Table 1:  Summary of LGCM results (after $10^{7}$ iterations) for different values of LST/FF Each data point is the result of averaging over ten game realizations. LST ($\phi=0.00002$) |  FF ($LST=0.2$)
---|---
LST |  ACC |  Avg. NC |  TDN |  $\phi$ |  ACC |  Avg. NC |  TDN
0.1 |  88.0% |  4.7 |  6.8 |  0.00001 |  81.3% |  10.2 |  15.4
0.2 |  85.5% |  5.5 |  7.7 |  0.00002 |  85.5% |  5.5 |  6.7
0.3 |  84.1% |  7.5 |  10.0 |  0.00004 |  89.3% |  3.6 |  4.3
0.4 |  82.3% |  9.3 |  42.3 |  0.00008 |  93.8% |  2.2 |  2.2
0.5 |  79.4% |  13.6 |  412.9 |  0.00016 |  98.0% |  1.3 |  1.3
0.6 |  71.9% |  27.6 |  6472.6 |  0.00032 |  99.9% |  1.0 |  1.0
0.7 |  38.6% |  154.6 |  67670.0 |  0.00064 |  79.8% |  3.55 |  1399.6
0.8 |  4.2% |  394.3 |  174366.5 |  0.00128 |  0% |  6.92 |  3549.2

Figure 7:  CS of simulations with different LST/FF.

Forgetting factor of categories. We next study the mechanism responsible for
categories to be gradually forgotten associated with category deletion
activities. In general, the threshold for removal of a category,
$\omega_{Th}$, should be small because, if $\omega_{Th}$ is large, even useful
categories would be deleted, possibly resulting in a dramatic effect on the
learning process. However, if the value of $\omega_{Th}$ is too small, the
category with small $\omega$ results small likelihood score, leading to a
small probability of any update (learning). These empirical considerations
lead to our choice of $\omega_{Th}=0.01$.

To be concrete, we fix $\mbox{LST}=0.2$ and focus on the impact of varying the
forgetting factor $\phi$. The results are shown in Table 1 (right) and Fig. 7
(b). When $\phi$ increases, it becomes hard for agents to remember
information, eventually leading to a reduction in the number of categories.
Apparently, when there are fewer categories, it is easier to reach consensus,
which is verified through the ACC and CS results. However, the correlation
between the FF and CS is non-monotonous. For $\phi>0.00032$, the CS decreases
dramatically and even equals to 0 for $\phi=0.00128$ through the process. That
is, quite naturally, consensus can never be reached if agents forget things
too fast.

It is remarked that, the average result of CS is given. For the case with
$\phi=0.00064$, some experiments reached consensus with only one category
(similar to the case with $\phi=0.00032$) while others had $CS=0$ (similar to
the case with $\phi=0.00128$).

###  4.3 Effect of network topology.

Does the network topology have a significant effect on LCGM? To address this
question, we considered four topological characteristics, namely the average
degree, network size, heterogeneity, and clustering.

Based on original BA scale-free networks [29], the impacts of average degree
(with fixed network size) and network size (with fixed average degree) onto
the performance of LCGM were investigated. The simulation results show no
observable influence on LCGM by varying the average degree (see Supplementary
Table S1). While the network size affects the consensus of LCGM slightly (see
Supplementary Table S2), network with smaller size can reach a better
consensus, which meets most people’s intuition that consensus are easily to be
reached among fewer agents.

To investigate the effects of heterogeneity and clustering, two types of
heterogeneous networks were adopted: scale-free network with aging [30] and
scale-free network with clustering attachment [31]. The first type of networks
incorporates the aging effect of nodal attraction, and the connection
probability is redefined as

$\mbox{Prob}(i)=\frac{\mbox{degree}_{i}\times
e^{-\lambda\times\mbox{age}_{i}}}{\sum_{j}\mbox{degree}_{j}\times
e^{-\lambda\times\mbox{age}_{j}}}$ (13)

where $\lambda$ is a tunable parameter reflecting the aging speed of nodal
attraction and $\mbox{age}_{i}$ denotes the age of node $i$. When node $i$ is
newly added, $\mbox{age}_{i}=0$. After every nodal addition cycle,
$\mbox{age}_{i}$ is incremented by one. Networks with distinct degrees of
homogeneities can be generated by adjusting the value of $\lambda$.

We perform LCGM on the aging scale-free networks of size 1000 with average
degree of 10 for different values of $\lambda$. As $\lambda$ is increased, the
Gini coefficient is significantly reduced and the average path length
increases, but the clustering coefficient is hardly affected. We find that an
increase in the value of $\lambda$ has little effect on the accuracy (ACC) and
the average number of categories ($\mbox{NC}_{avg}$) in LCGM. However, the
consensus score (CS) decreases significantly with $\lambda$ even though the
accuracy remains high (see Supplementary Table S3). As discussed before, there
is a negative correlation between the agent-pair distance and CS. Therefore,
We further investigated average consensus scores of agent pairs with respect
to distance and $\lambda$ (see Supplementary Table S4). It is observed that
$\lambda$ has no observable influence on CS between agents with short distance
($\leq 2$), while for agent pairs with longer distance, CS decreases generally
as the network becomes more homogeneous ($\lambda$ increases).

Then, the effect of clustering on LCGM is investigated by considering the
conventional clustering coefficient. Scale-free networks with different
clustering coefficients are obtained through the process of clustering
attachment [31] where an edge is added to connect a new node and one of its
two-hop neighbors, forming a triangle. The probabilities of clustering and
preferential attachment are $p$ and $1-p$, respectively. If no two-hop
neighbor is available for a new node, preferential attachment is adopted. We
perform LCGM on networks of size 1000 (with average degree of 10) for
different values of $p$ (See results in Supplementary Table S5). It can be
easily observed that the clustering coefficient increases with the value of
$p$. An increase in the value of $p$, however, has only limited effect on the
Gini Coefficient and the average path length. From the results, it can be
concluded that the clustering coefficient does not hurt the consensus. Since
categories have weights in LCGM, several strong categories will survive
through evolution. Different from traditional naming game models, LCGM allows
these categories to coexist, and consequently, they will dominate the whole
population together.

##  5 Discussion

To reach consensus among a population of diverse agents is a problem of great
complexity but one with broad relevance. A computational and mathematical
paradigm to investigate this problem is naming games. Most existing works in
this area were based on some learning processes that solely rely on direct
interactions among the agents without taking into account agents’ perception.
The basic idea underlying our proposed likelihood category game model (LCGM)
is that knowledge acquiring is essential to achieving consensus. In LCGM,
self-organized agents rely on acquired knowledge to define category and employ
statistical likelihood estimation to distinguish and “name” objects that
belong to the same category. Particularly, the agents equipped with knowledge
acquiring are capable of exploiting distinct likelihoods as determined by
their knowledge to classify objects. Importantly, knowledge is not static but
dynamic: agents update knowledge through learning. The agents in our LCGM are
thus “smart,” more closely mimic those in the real social, economical, and
political world.

The distinct features of our proposed LCGM are the following. Firstly, it is a
truly autonomous category game model, eliminating the need for ground truth
knowledge. Secondly, introducing the concept of likelihood in this context
makes the game model more realistic because, in each game, agents not only
return the names but also the likelihood scores that represent their
confidence corresponding to the answer. The information communicated among the
agents is “smart” in the sense that it is no longer restricted to some form of
absolute answer but is more content or feature based. Thirdly, in a pairwise
game, the more knowledgeable agent (with a higher likelihood score) becomes
the “teacher,” a feature that fits with the interaction in the real world.
Through the incorporation of learning, our model generalizes the existing
naming game models and is closer to describing reality. (Our model reduces to
a variant of the much studied minimum name game in the special case where the
learning domain is singular in the sense that its extent is effectively zero,
i.e., no learning.)

Our model provides novel insights into consensus dynamics. For example, there
is a trade-off between the amount of knowledge and consensus, providing a
quantitative explanation for the phenomenon that consensus is hard to be
reached among serious agents with a high LST (who know more categories).
Another finding is that hubs contributed more to knowledge formation, which
accordingly have a larger probability to become “smart” and to take lead in a
game. By studying the effects of network structural characteristics on the
consensus dynamics, we identify the impacts of distance and heterogeneity on
consensus. While the findings are preliminary, they are useful for
understanding the dynamical evolution of consensus and may even serve as the
base for formulating control strategies to harness consensus dynamics,
warranting further efforts in investigating learning and likelihood based
games.

## References

  * [1] Dall’Asta, L., Baronchelli, A., Barrat, A. & Loreto, V.  Nonequilibrium dynamics of language games on complex networks.  _Physical Review E_ 74, 036105 (2006).
  * [2] Dall’Asta, L., Baronchelli, A., Barrat, A. & Loreto, V.  Agreement dynamics on small-world networks.  _Europhysics Letters_ 73, 969–975 (2006).
  * [3] Barrat, A., Baronchelli, A., Dall’Asta, L. & Loreto, V.  Agreement dynamics on interaction networks with diverse topologies.  _Chaos: An Interdisciplinary Journal of Nonlinear Science_ 17, 026111 (2007).
  * [4] Lu, Q., Korniss, G. & Szymanski, B. K.  The naming game in social networks: community formation and consensus engineering.  _Journal of Economic Interaction and Coordination_ 4, 221–235 (2009).
  * [5] Lou, Y., Chen, G., Fan, Z. & Xiang, L.  Local communities obstruct global consensus: Naming game on multi-local-world networks.  _Physica A: Statistical Mechanics and its Applications_ 492, 1741–1752 (2018).
  * [6] Wu, C.  Can stubbornness or gullibility lead to faster consensus? a study of various strategies for reaching consensus in a model of the naming game.  In _IEEE International Symposium Circuits and Systems (ISCAS)_ (2011).
  * [7] Liu, R.-R. _et al._ Optimal convergence in naming game with geography-based negotiation on small-world networks.  _Phys. Lett. A_ 375, 363–367 (2011).
  * [8] Steels, L.  _Experiments in Cultural Language Evolution_ (John Benjamins Publishing, 2012).
  * [9] Li, B., Chen, G., & Chow, T. W. S.  Naming game with multiple hearers.  _Communications in Nonlinear Science and Numerical Simulation_ 18, 1214–1228 (2013).
  * [10] Treitman, Y., Lim, C., W., Z. & A., T.  Naming game with greater stubbornness and unilateral zealots.  In _IEEE $2^{nd}$ Network Science Workshop_ (2013).
  * [11] Gao, Y., Chen, G. & Chan, R. H. M.  Naming game on networks: Let everyone be both speaker and hearer.  _Sci. Rep._ 4, 6149 (2014).
  * [12] Mistry, D., Zhang, Q., Perra, N. & Baronchelli, A.  Committed activists and the reshaping of status-quo social consensus.  _Physical Review E_ 92, 042805 (2015).
  * [13] Lou, Y. & Chen, G.  Analysis of the “naming game” with learning errors in communications.  _Sci. Rep._ 5, 12191 (2015).
  * [14] Niu, X., Doyle, C., Korniss, G. & Szymanski, B. K.  The impact of variable commitment in the naming game on consensus formation.  _Sci. Rep._ 7, 41750 (2017).
  * [15] Fu, G., Cai, Y. & Zhang, W.  Analysis of naming game over networks in the presence of memory loss.  _Physica A: Statistical Mechanics and its Applications_ 479, 350–361 (2017).
  * [16] Lou, Y., Chen, G. & Hu, J.  Communicating with sentences: A multi-word naming game model.  _Physica A: Statistical Mechanics and its Applications_ 490, 857–868 (2018).
  * [17] Zhou, J., Lou, Y., Chen, G. & Tang, W. K.  Multi-language naming game.  _Physica A: Statistical Mechanics and its Applications_ (2018).
  * [18] Li, D., Fan, Z. & Tang, W. K. S.  Domain learning naming game for color categorization.  _PLoS One_ 12, e0188164 (2017).
  * [19] Steels, L. & Belpaeme, T.  Coordinating perceptually grounded categories through language: A case study for colour.  _Behavioral and Brain Sciences_ 28, 469–488 (2005).
  * [20] Puglisi, A., Baronchelli, A. & Loreto, V.  Cultural route to the emergence of linguistic categories.  _Proceedings of the National Academy of Sciences_ 105, 7936–7940 (2008).
  * [21] Mukherjee, A., Tria, F., Baronchelli, A., Puglisi, A. & Loreto, V.  Aging in language dynamics.  _PLoS One_ 6, 16677 (2011).
  * [22] Cui, P., Tang, M. & Wu, Z. X.  Message spreading in networks with stickiness and persistence: Large clustering does not always facilitate large-scale diffusion.  _Sci. Rep._ 4, 6303 (2014).
  * [23] Baronchelli, A., Loreto, V. & Puglisi, A.  Individual biases, cultural evolution, and the statistical nature of language universals: The case of colour naming systems.  _PLoS One_ 10, 0125019 (2015).
  * [24] Eguiluz, V. M. & Zimmermann, M. G.  Transmission of information and herd behavior: An application to financial markets.  _Physical Review Letters_ 85, 5659 (2000).
  * [25] Hunter, R. S.  Photoelectric color difference meter.  _J. Opt. Soc. Ame._ 48, 985–995 (1958).
  * [26] Luo, M. R., Cui, G. & Rigg, B.  The development of the cie 2000 colour-difference formula: Ciede2000.  _Color Res. Appl._ 26, 340–350 (2001).
  * [27] SNAP: Network dataset: facebook.  http://snap.stanford.edu/data/egonets-Facebook.html.  Accessed: 2017-08-30.
  * [28] SNAP: Network dataset: email-eu-core network.  https://snap.stanford.edu/data/email-Eu-core.html.  Accessed: 2018-06-19.
  * [29] Barabási, A.-L. & Albert, R.  Emergence of scaling in random networks.  _science_ 286, 509–512 (1999).
  * [30] Zhu, H., Wang, X. & Zhu, J. Y.  Effect of aging on network structure.  _Physical Review E_ 68, 056121 (2003).
  * [31] Holme, P. & Kim, B. J.  Growing scale-free networks with tunable clustering.  _Physical Review E_ 65, 026107 (2002).

## Acknowledgements

YCL would like to acknowledge support from the Vannevar Bush Faculty
Fellowship program sponsored by the Basic Research Office of the Assistant
Secretary of Defense for Research and Engineering and funded by the Office of
Naval Research through Grant No. N00014-16-1-2828. ZYF and WKST would like to
acknowledge support from City University of Hong Kong through Research Grant
No. 7004835.

## Author contributions

W.K.S.T. and Z.Y.F. conceived research. Z.Y.F. performed computations.
W.K.S.T. and Z.Y.F. analyzed the results. W.K.S.T., Z.Y.F., and Y.C.L. wrote
the paper.


# Action Quality Assessment Across Multiple Actions

Paritosh Parmar Brendan Tran Morris
University of Nevada, Las Vegas
<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Can learning to measure the quality of an action help in measuring the quality
of other actions? If so, can consolidated samples from multiple actions help
improve the performance of current approaches? In this paper, we carry out
experiments to see if knowledge transfer is possible in the action quality
assessment (AQA) setting. Experiments are carried out on our newly released
AQA dataset (http://rtis.oit.unlv.edu/datasets.html) consisting of 1106 action
samples from seven actions with quality as measured by expert human judges.
Our experimental results show that there is utility in learning a single model
across multiple actions.

Diving

(a)

Gymvault

(b)

Skiing

(c)

Snowboard

(d)

S.Dive 3m

(e)

S.Dive 10m

(f)

Trampoline

(g)  Figure 1: Preview of our dataset. To see the videos play, please download
the manuscript and view in an Adobe Reader.

[loop,autoplay,poster=17,height=0.1]3videos/d1/image_00010030
[loop,autoplay,poster=17,height=0.1]3videos/d2/image_00010030
[loop,autoplay,poster=17,height=0.1]3videos/d3/image_00010030
[loop,autoplay,poster=17,height=0.1]3videos/d4/image_00010030
[loop,autoplay,poster=17,height=0.1]3videos/d5/image_00010030
[loop,autoplay,poster=17,height=0.1]3videos/d6/image_00010030
[loop,autoplay,poster=17,height=0.1]3videos/d7/image_00010030
[loop,autoplay,poster=17,height=0.1]3videos/g1/image_00010030
[loop,autoplay,poster=17,height=0.1]3videos/g2/image_00010030
[loop,autoplay,poster=17,height=0.1]3videos/g3/image_00010030
[loop,autoplay,poster=17,height=0.1]3videos/g4/image_00010030
[loop,autoplay,poster=17,height=0.1]3videos/g5/image_00010030
[loop,autoplay,poster=17,height=0.1]3videos/g6/image_00010030
[loop,autoplay,poster=17,height=0.1]3videos/g7/image_00010030
[loop,autoplay,poster=17,height=0.1]3videos/sk1/image_00010030
[loop,autoplay,poster=17,height=0.1]3videos/sk2/image_00010030
[loop,autoplay,poster=17,height=0.1]3videos/sk3/image_00010030
[loop,autoplay,poster=17,height=0.1]3videos/sk4/image_00010030
[loop,autoplay,poster=17,height=0.1]3videos/sk5/image_00010030
[loop,autoplay,poster=17,height=0.1]3videos/sk6/image_00010030
[loop,autoplay,poster=17,height=0.1]3videos/sk7/image_00010030
[loop,autoplay,poster=17,height=0.1001]3videos/sn1/image_00010030
[loop,autoplay,poster=17,height=0.1001]3videos/sn2/image_00010030
[loop,autoplay,poster=17,height=0.1001]3videos/sn3/image_00010030
[loop,autoplay,poster=17,height=0.1001]3videos/sn4/image_00010030
[loop,autoplay,poster=17,height=0.1001]3videos/sn5/image_00010030
[loop,autoplay,poster=17,height=0.1001]3videos/sn6/image_00010030
[loop,autoplay,poster=17,height=0.1001]3videos/sn7/image_00010030
[loop,autoplay,poster=17,height=0.1]3videos/ssd1/image_00010030
[loop,autoplay,poster=17,height=0.1]3videos/ssd2/image_00010030
[loop,autoplay,poster=17,height=0.1]3videos/ssd3/image_00010030
[loop,autoplay,poster=17,height=0.1]3videos/ssd4/image_00010030
[loop,autoplay,poster=17,height=0.1]3videos/ssd5/image_00010030
[loop,autoplay,poster=17,height=0.1]3videos/ssd6/image_00010030
[loop,autoplay,poster=17,height=0.1]3videos/ssd7/image_00010030
[loop,autoplay,poster=17,height=0.1]3videos/sp1/image_00010030
[loop,autoplay,poster=17,height=0.1]3videos/sp2/image_00010030
[loop,autoplay,poster=17,height=0.1]3videos/sp3/image_00010030
[loop,autoplay,poster=17,height=0.1]3videos/sp4/image_00010030
[loop,autoplay,poster=17,height=0.1]3videos/sp5/image_00010030
[loop,autoplay,poster=17,height=0.1]3videos/sp6/image_00010030
[loop,autoplay,poster=17,height=0.1]3videos/sp7/image_00010030
[loop,autoplay,poster=17,height=0.1]3videos/t1/image_00010030
[loop,autoplay,poster=17,height=0.1]3videos/t2/image_00010030
[loop,autoplay,poster=17,height=0.1]3videos/t3/image_00010030
[loop,autoplay,poster=17,height=0.1]3videos/t4/image_00010030
[loop,autoplay,poster=17,height=0.1]3videos/t5/image_00010030
[loop,autoplay,poster=17,height=0.1]3videos/t6/image_00010030
[loop,autoplay,poster=17,height=0.1]3videos/t7/image_00010030

##  1 Introduction

Action quality assessment (AQA) is the process of quantifying _how well_ an
action was performed or computing a score representing the execution quality
of an action. Automatic AQA, in particular, can be an important component of
many practical applications. For _e.g_., AQA can be employed in low-cost at-
home physiotherapy to manage diseases like cerebral palsy [15, 27]. High
quality AQA can be incorporated into medical training to assess the surgical
skills of a student [28, 5]. AQA systems can be used to mimic Olympic judges
during sports training [17, 16, 23, 28]; or provide a second opinion in light
of recent judging scandals [3, 24]. Despite having numerous potential
applications, automatic AQA and skill assessment have received little
attention in literature. Consequently, there is a dearth of available
datasets.

Current AQA systems/frameworks [17, 16, 23, 5] are concerned with measuring
the quality of a single action. As such, the models are trained on examples of
that particular action. However, existing AQA datasets are typically very
small, consisting of only a few samples since the collection of training data
requires the use of a domain expert (significant effort by comparison with
action recognition). Due to this data limit, current AQA approaches may not be
reaching their full performance potential.

Pre-training helps with almost every computer vision task. For _e.g_., object
detection [7] is aided by pre-training on ImageNet [4], performance of the C3D
network [22] was boosted by pre-training on I380K [22] and then finetuning
Sports-1M [11], UCF-101 [19] action classification performance was enhanced
when the network is pre-trained on Sports-1M [11], etc. Transfer learning is
particularly helpful when target datasets are small. Inspired by several
instances where pre-training has helped, we pose the following questions: are
there common action quality elements (see Sec. 3.2) among different actions?
If so, would it be helpful to train/pre-train a model across various actions
(instead of following current approach of training on one particular action)?
Can a model trained on various actions measure the quality of an unseen
action?

The remaining of the paper is as follows. First, related work in AQA is
reviewed in Sec. 2. Then, a new AQA dataset (the largest to date) is
introduced, and notion of common action quality elements is discussed in Sec.
3. Following which, we answer previously stated questions and introduce a
different approach of using all-action model, a model that learns to measure
the quality of multiple actions. In Sec. 5, we discuss the experiments and
results, which provide us the evidence that there is a utility in learning an
all-action model over single-action model.

##  2 Related work

The first major work in the area of AQA was by Pirsiavash _et al_. [17], which
used pose (of divers and skaters) as features and learned a SVR to map those
features to a quality score. By focusing on Olympic sports, they were able to
use professionals to provide an objective measure of quality through the
judged score. Venkataraman _et al_. [23] extended this work by modifying the
feature representation using concatenated approximate entropy of poses rather
than DCT of pose. This better encoding resulted in a 4% improvement in
performance. Inspired by the success of deep-learning on the task of action
recognition, Parmar and Morris [16] hypothesized and verified that
spatiotemporal features from C3D[22] capture the quality aspect of the actions
well. Furthermore, they compared different video feature aggregation schemes,
such as averaging clip-level features, and LSTM, to get sample-level features;
and regression schemes, SVR, and fully connected layer.

A unique concept from AQA is feedback for performance improvement. Pirsiavash
_et al_. [17] were able to suggest minor pose tweaks would could have been
adopted in order to improve scores. The LSTM-based approaches by Parmar and
Morris [16] did not offer such detailed feedback, but could identify action
segments (video clips) where an error might have been made. These types of
error detection mechanisms are important for coaching and faster analysis of
athlete performance.

Steering away from sports, Zia _et al_. [28] proposed an approach to measure
the surgical skills in an automated fashion. In this work, a classifier is
trained on frequency-domain transformed spatiotemporal interest points.
Doughty _et al_. [5] also address the problem of surgical skill assessment as
well as the use of chopsticks, dough-rolling, and drawing. Outside of surgery
task, quality was assessed not with a score but based on pairwise comparison
of skill in a “which is better” framework. While other quality or skills
assessment works do exist, they tend to measure specific characteristics of
the task which are not easily generalizable. An example is the basketball
skills assessment work by Bertasius _et al_. [2], wherein they present an
approach to measure a person’s basketball skills from analyzing the video
captured by the camera worn by the person. Firstly, atomic basketball events
are detected, which are then passed through a Gaussian mixtures to obtain a
features which would be representative of player’s skills. A skill assessment
model is finally learnt on these features to assess basketball skills. Also,
it should be noted that skills assessed in [2] are subjective to evaluator’s
preference, since annotation was based on a single basketball coach’s
assessment. In contrast, our approach would not be biased towards any one
judge, since our annotated scores are averages of scores from individual
judges (usually 5-7 judges).

All of these works use one model each for every action class. Single-acion
models or action-specific models don’t exploit the fact that there are some
common, shared action quality concepts/elements among actions (refer to Sec.
3.2). Transfer learning is not new and has been used effectively in the
literature[18, 21, 13, 14, 20, 25, 26]. In a recent work by Abramovich and
Pensky [1], it was seen that having many classes may be a blessing and not a
curse for a classification purposes suggesting that similar result could be
true for AQA. In this work, we evaluate to see if there are shared action
quality concepts, and if we can exploit them by learning a single model on
datapoints from all actions.

##  3 AQA-7 Dataset

Sport | Avg. Seq. Len. | # Samples | Score Range | # Participants | View Variation
---|---|---|---|---|---
Single Diving 10m platform | 97 | 370 | 21.60 - 102.60 | 1 | negligible
Gymnastic vault | 87 | 176 | 12.30 - 16.87 | 1 | large
Big Air Skiing | 132 | 175 | 8 - 50 | 1 | large
Big Air Snowboarding | 122 | 206 | 8 - 50 | 1 | large
Sync. Diving 3m springboard | 156 | 88 | 46.20 - 104.88 | 2 | negligible
Sync. Diving 10m platform | 105 | 91 | 49.80 - 99.36 | 2 | negligible
Trampoline | 634 | 83 | 6.72 - 62.99 | 1 | small
Table 1: Characteristics of AQA-7 dataset.

To the best of our knowledge, there are only two publicly available AQA
datasets [17, 16] and they have limited number of samples for each individual
activity. This is partly due to the extra effort required to collect samples
as compared to an action recognition dataset. In case of action recognition
dataset compilation, an annotator might go to video hosting website such as
YouTube and run a search query on names of actions. While in compiling an AQA
dataset, annotator has to mark starting and ending frames of a sample, and
note down execution score, difficulty level, final score, etc. Additionally,
field experts are required to evaluate and score AQA data samples which can be
quite difficult in many domains. These factors limit dataset size.

Furthermore, for action recognition, many more meaningful short samples can be
produced from one original longer video. By meaningful sample, we mean that,
even if you clip an action recognition video, you would still have the action
class concept captured – it has been shown that action classification task
maybe performed using as little as a single frame [11]. Unlike action
recognition, in case of AQA, a full action sequence must be examined because
an error in execution can be made at any time during the action sequence.
Leaving out frames where the error occurred would result in predicted scores
that are not indicative of the true quality. This limits data augmentation
tricks for more effective sample utilization.

To fill this data void, we introduce Action Quality Assessment 7 (AQA-7)
dataset (Fig. 1a), comprising samples from seven actions: {singles diving-10m
platform, gymnastic vault, big air skiing, big air snowboarding, synchronous
diving-3m springboard, synchronous diving-10m platform, and trampoline}
captured during Summer and Winter Olympics. Scoring rules change from time to
time; however, we made sure that rules were same for the samples that we
collected. A summary of the AQA-7 dataset characteristics are given in Table
1.

###  3.1 Description of Actions

Diving:  Individual diving has been a Summer Olympic sport since the early
1900s and synchronized diving was introduced in 2000. Divers perform acrobatic
stunts such as somersaults and twists either from a platform or a springboard.
The score is based on the difficulty of and the quality of execution of the
dive. AQA-7 contains three different events – the 10m platform (Diving in
first row of Fig. 1a), synchronized 10m platform (Sync Dive 10m in sixth row
of Fig. 1a), and 3m springboard (Sync Dive 3m in fifth row of Fig. 1a). The
three diving events have negligible view variation as they are all shot from a
consistent angle for all samples. The synchronized dives have two athletes
participating and their synchronization, which is given more importance than
execution, is also a measure of quality of the action. The singles diving-10m
platform class is an extension of the 159 samples originally from Pirsiavash
et al. [17] to 370 dives.
Gymnastic vault:  Gymvault is a gymnastic event from the Summer Olympics where
the athlete runs down a runway and uses a springboard to launch themselves
over the vault and perform aerial tricks. The gymvault dataset was collected
from various international events in addition to Olympics. Like diving, the
quality of gymvault is assessed based on the difficulty and the quality of
execution. Examples tend to have large view variation both over the course of
a single vault attempt (running, planting on vault, spinning in air, and
landing) as well as due to differences broadcast camera placement at different
events (see second row of Fig. 1a).
Big Air:  The Ski (BigSki) and Snowboard (BigSnow) Big Air events were new
events in the 2018 Winter Olympics. Samples were obtained from previous
X-Games since it is the premiere venue for the event. Big Air has significant
view variation between events and within an example due to camera location
(rows 3 and 4 of Fig. 1a). Scoring is based on four components – difficulty,
execution, amplitude, and progression and landing – a more complicated formula
than the other actions especially since there is a qualitative component of
pushing Big Air forward by doing tricks that nobody else is doing. Note that
for Snowboard Big Air the pixel coverage size of the person is similar to that
of the snowboard.
Trampoline:  The Trampoline event was added to the Olympics in 2000. It is
judged based on difficulty, execution, and time of (in air) flight. Like for
diving, the camera is in a consistent side-view position but also moves up and
down to follow the athlete for minimal view variation. Unlike all the other
sports which consist of a single “action” performed in less than five seconds,
a trampoline sample will contain multiple (10) action phases, where the
athlete twists and turns, over its 20 seconds.

###  3.2 Common Action Quality Elements

The AQA-7 sports have similar action elements which include flipping and
twisting (Fig. 1a). As such, the quality of actions are assessed in similar
fashion. For _e.g_. in Diving and Gymvault, judges expect athletes to have
legs perfectly straight in pike position (execution quality aspect) and degree
of difficulty is directly proportional to the number of twists and
somersaults. Similarly, in ski and snowboard Big Air events, the difficulty is
related to the number of vertical and horizontal spins. In all sports there is
a expectation of a perfect landing with high impact on final score (an entry
with minimal splash or ‘ripping’ is the diving equivalent).

The reason behind these similarities is that having to complete more number of
somersaults, twists, or spins (difficulty aspect) while keeping legs straight,
and having the body in tight tuck or pike position (execution quality aspect)
in the limited time from take-off to landing makes it harder to achieve and,
therefore, worthy of more points from judges (equivalently, higher quality).
The final score is a function of execution quality aspect and the degree of
difficulty. In some cases, like Diving, it will be the product function, in
another case, like Gymault, it will be the summation, while other actions
(BigAir events) may use a more holistic combination approach. Given the
similarities in actions, it is believed that knowledge of what aspects to
value in one action can help in measuring the quality in another.

##  4 Our Approach

Figure 2: C3D-LSTM network

Although, in general, action quality may be highly dependent on the action, we
hypothesize that actions do have commonalities (Sec. 3.2) that can be
exploited despite individual differences such as judging criteria. However, it
may be that each action is unique and do not have common quality elements and
our hypothesis is wrong, and hence it would not be possible to share knowledge
or learn a consistent model that can measure quality across multiple actions.

In order to test if our hypothesis is correct, and see if we can transfer
action quality knowledge across actions, we propose to use the C3D-LSTM
framework [16], illustrated in Fig. 2. This network consists of a smaller
version of C3D network [22] followed by a single 256-dimensional LSTM [8]
layer and a fully-connected (fc) layer, which outputs the final AQA score. A
video is processed in clips of 16 frames to generate C3D features at the fc6
layer which is connected to the LSTM layer for temporal feature aggregation.
The C3D network is kept frozen during training so only the LSTM and final fc
layer parameters are tuned. Euclidean distance between the predicted scores
and true scores is used as the loss function to be minimized. The main
difference in this work is rather than building an individual model for each
action (we refer to these as action-specific or single-action models), a
single model is learned by training with samples from all/multiple actions (we
refer to our model as all-action or multi-action models).

To answer our fundamental questions regarding quality elements shared across
actions, three different experiments are designed: 1) to check if it is
possible to learn an all-action, and if so, compare the performance of our
proposed all-action model against action-specific model (Sec. 5.1), 2) to
evaluate how well can an all-action model quantify the quality of unseen
action classes (Sec. 5.2), and 3) to evaluate the generalization of all-action
model to novel action classes (Sec. 5.3).

##  5 Experiments

| Diving | Gymvault | Skiing |  | Snowb-
---
oarding
| Sync.
---
Dive 3m
| Sync.
---
Dive 10m
| Avg.
---
Corr.
Pose+DCT [17] | 0.5300 | - | - | - | - | - | -
| Single-action
---
C3D-SVR[16]
0.7902 | 0.6824 | 0.5209 | 0.4006 | 0.5937 | 0.9120 | 0.6937
| Single-action
---
C3D-LSTM[16]
0.6047 | 0.5636 | 0.4593 | 0.5029 | 0.7912 | 0.6927 | 0.6165
| Ours All-action
---
C3D-LSTM
0.6177 | 0.6746 | 0.4955 | 0.3648 | 0.8410 | 0.7343 | 0.6478
Table 2: All-Action vs. Single-Action models. Performance evaluation of single-action and all-action models in terms of action-wise and average Spearman’s rank correlation (higher is better). First two frameworks simply average features to aggregate them and use SVR as the regression module. The bottom two frameworks use LSTM to aggregate features and use a fully-connected layer as the regression module. Our approach can be directly compared with single-action C3D-LSTM [16], since both have the same architecture.  Training action class Unseen action class | Diving |  | Gym-
---
vault
Skiing |  | Snow-
---
board
| Sync-
---
Dive 3m
| Sync-
---
Dive 10m
| Avg.
---
Corr.
| Random Wts./Ini.
---
0.0590 | 0.0280 | -0.0602 | -0.0703 | -0.0146 | -0.0729 | -0.0218
Diving | 0.6997 | -0.0162 | 0.0425 | 0.0172 | 0.2337 | 0.0221 | 0.0599
Gymvault | 0.0906 | 0.8472 | 0.0517 | 0.0418 | -0.1642 | -0.3200 | -0.0600
Skiing | 0.2653 | -0.1856 | 0.6711 | 0.1807 | 0.1195 | 0.2858 | 0.1331
Snowboard | 0.2115 | -0.2154 | 0.3314 | 0.6294 | 0.0945 | 0.1818 | 0.1208
Sync. Dive 3m | 0.1500 | -0.0066 | -0.0494 | -0.1102 | 0.8084 | 0.0428 | 0.0053
Sync. Dive 10m | 0.0767 | -0.1842 | 0.0679 | 0.0360 | 0.4374 | 0.7397 | 0.0868
Multi-action | 0.2258 | 0.0538 | 0.0139 | 0.2259 | 0.3517 | 0.3512 | 0.2037
Table 3: Zero-shot AQA. Performance comparison of randomly-initialized model,
single-action models (for _e.g_., first row shows the results of training on
diving action measuring the quality of the remaining (unseen) action classes),
and multi-action model (all-action model trained on five action classes) on
unseen action classes. In multi-action class, the model is trained on five
action classes and tested on the remaining action class (column-wise). In
single-action model rows, diagonal entries show results of training and
testing on the same action. Avg. Corr. shows the result of average (using
Fisher’s z-score) correlation across all columns.

(a) Diving - 25

(b) Diving - 75

(c) Diving - 125

(d) Gymv - 25

(e) Gymv - 75

(f) Gymv - 125

(g) Ski - 25

(h) Ski - 75

(i) Ski - 125

(j) Snowb - 25

(k) Snowb - 75

(l) Snowb - 125

(m) S.D 3m - 15

(n) S.D 3m - 25

(o) S.D 3m - 35

(p) S.D 10m - 15

(q) S.D 10m - 25

(r) S.D 10m - 35

Figure 3: Finetuning from scratch vs. finetuning from pre-trained multi-action
model. Plot of Spearman’s rank correlation against every hundred iterations
for different number of training samples. Blue and red curves represent multi-
action and randomly initialized models, respectively. The gap in the initial
iterations suggest that good initialization of LSTM weights was achieved by
training on multiple actions. In most of the cases, multi-action model has
better performance than randomly initialized model on test samples throughout
all the iterations.

Test action | Diving | Gymvault | Skiing
---|---|---|---
# samples | 25 | 75 | 125 | 25 | 75 | 125 | 25 | 75 | 125
RI | 0.5633 | 0.5952 | 0.6935 | 0.3197 | 0.4231 | 0.5278 | 0.0955 | 0.5050 | 0.5862
AA | 0.5937 | 0.6742 | 0.7443 | 0.4509 | 0.5350 | 0.5894 | 0.1279 | 0.5778 | 0.5991

Snowboard Sync. Dive 3m Sync. Dive 10m 25 75 125 15 25 35 15 25 35 0.1813
0.4507 0.4751 0.2659 0.3382 0.4268 0.3511 0.4913 0.6305 0.1978 0.3347 0.4437
0.5235 0.5980 0.7429 0.4500 0.7900 0.8123

Table 4: Finetuning from scratch vs. finetuning from pre-trained multi-action
model. Experimental results (Spearman’s rank correlation) of finetuning a
randomly-initialized (RI) model and an all-action (AA) model pre-trained on
five action classes. The numbers represent the best results from all the
iterations.

Three different experiments are performed to test whether it is advisable to
learn a joint action quality model rather than individual model. To be
consistent with existing literature, Spearman’s rank correlation (higher is
better) is used as the performance metric. When presenting aggregated results,
the average Spearman’s rank correlation is computed from individual per action
correlations using Fisher’s z-value as described in [6].

Data Preparation: Since different actions have different ranges for scores, as
shown in Table 1, we divide the raw scores of all the actions by the training
standard deviation of the corresponding action. At test time, we multiply the
predicted scores by the standard deviation of the appropriate action to get
the final judged value. Experiments were only conducted on six of the action
classes since they are of similar short length. All videos are normalized to a
fixed size of 103 frames by zero-padding the first frames where needed.
Trampoline is excluded from the experiments since the average length of 650
frames is much longer and composed of multiple “tricks.” The model is trained
using 803 videos, and tested on the remaining 303 videos. During training,
temporal data augmentation is used to have six different copies of the same
video sample with one frame start difference (effectively 4818 training
samples).

Implementation details: Caffe[10] was used to implement our model on a Titan-X
GPU. The C3D network was pretrained on UCF-101 [19] for 100k iterations with
an initial learning rate of 0.001 and annealed by a factor of 10 after every
40k iterations, momentum of 0.9, and a weight decay of 0.005. After pre-
training, C3D is frozen and used as a feature extractor.

For the LSTM layer, the ADAM solver [12] is used with an initial learning rate
of 0.001 and annealed by a factor of 2 after every 3K iterations. Optimization
was carried out for 20K iterations after the LSTM layer was initialized with
Gauassian noise with standard deviation of 0.1. Using a stride of 16 frames,
the C3D network “sees” a whole action sequence in 6 LSTM steps. We use a batch
size of 90 clips (15 full video samples).

###  5.1 All-Action vs. Single-Action Models

In the first experiment, rather than learning a model using only samples from
a single action, our All-action model is learned using diverse and larger
dataset with samples from across six actions. A comparison of the all-action
model with other state-of-the-art approaches is given in Table 2.

Pirsiavash _et al_. [17] optimized their pose estimator for each action
(diving, figure-skating) and therefore cannot be compared with actions other
than diving fairly, so we provide their results only on diving. The two other
comparisons, C3D-SVR and C3D-LSTM, are the state-of-the-art single-action
(action-specific) results from Parmar and Morris [16]. Pose+DCT [17] and
C3D-SVR [16] use SVR as regression module. The baseline for the All-action
model is the single-action C3D-LSTM since both use same feature aggregation
method (LSTM) and regression module (fc layer).

It is clear that the use of spatio-temporal features (C3D net, fc6 layer
activations) improves over Pose+DCT. The proposed all-action model outperforms
the single-action model for five of the six actions – all but for Snowboarding
which was less by 0.14. On average across the actions, the all-action model
improves Spearman’s rank correlation performance by 0.03 without changing the
network and instead leveraging data samples from all actions.

The 0.6478 correlation value from all-action model is competitive even with
action-specific C3D-SVR’s 0.6937 (six different C3D-SVR’s are used, one for
each action). This is noteworthy since C3D-SVR was the best performing AQA
system in [16] and, as they mentioned, LSTM aggregation is preferred for its
ability to temporally localize action quality drops (error identification).

Note that our strategy to train a single model using datapoints from all the
actions is complementary to the existing approaches and employing it may help
improve their performance.

###  5.2 Zero-Shot AQA

One open question for AQA is whether the learned concept of quality can
generalize. If action quality concepts are shared among actions, then the
knowledge of how to measure the quality of a group of actions, is likely to
help in measuring the quality of other, unseen action/s.

####  5.2.1 Random-initialization vs. Multi-action pre-training

While the previous experiment showed that all-action C3D-LSTM model performed
better than the single-action counterpart, it did not necessarily indicate
that learning one action helped to learn another action. To address this
issue, a zero-shot AQA experiment is designed. In the zero-shot setting, a
model is trained on five actions and tested on the remaining unseen sixth
action. The baseline for comparison is the same network with random LSTM
weights (C3D weights remain same for all models). Comparing multi-action model
with randomly-initialized weights may not seem very fair, but interestingly, a
hierarchy of random, untrained convolution filters have been shown to perform
almost as well as learned filter weights in a work by Jarrett _et al_. [9]. It
is assumed that if innate quality concepts were not learned, then the
resulting multi-action model (an all-action model trained using five action
classes) would have similar performance to random initialization. If instead
the multi-action model is much better than random initialization on the zero-
shot AQA task then it indicates that there are common quality elements across
actions and that there is utility in pre-training a model across multiple
actions.

A summary of AQA performance on an unseen action is detailed in Table 3. With
random initialization the AQA system is not able to perform with any
reliability, as indicated by Spearman’s rank correlation close to zero value.
In contrast, the all-action version shows some positive correlation. We
believe that the reason for better performance of the all-action model is that
the use of multiple actions provides a good initialization since there are
common/shared action quality elements. In addition, with all-action model
there is an advantage of having access to more training videos from which to
learn. Although, our all-action works better than random-initialization across
four actions, there are two actions - Gymvault, and Skiing - that seem to be
providing very weak indication. So, we explore further (refer Sec. 5.3) in
order to be sure that our proposed all-action actually attains a good
initialization, in general.

####  5.2.2 Single-Action vs. Multi-Action Transfer

To further examine knowledge transfer, a sub-experiment is conducted to assess
the performance of a single-action model on the unseen action classes. The
results are compiled in middle six rows of Table 3. Note: single-action pre-
training is actually a special case of our proposed multi-action pre-training.

When examining the single-action results, note they resemble a confusion
matrix with high diagonal elements (matched training and testing data) but
with low off-diagonals (mismatched test classes to training class). The
single-action models tend to perform well on very similar actions e.g. Diving
on Sync. Dive 3m/10m, or vice-versa, or Snowboarding on Skiing, and vice-
versa. This result is intuitive since similar actions have more shared
concepts. However, non-intuitive relationships are apparent as well. An
example is the Skiing model tested on Diving, which have quite different
judging criteria yet still manages significant rank correlation. While no one
single-action model is strong across all classes it does support the idea of
shared quality concepts across actions.

On average, the All-action model greatly outperforms any single-action model.
With more action classes included in training, more quality concepts/elements
are shared and there are more datapoints from which to learn each of the
elements of quality resulting in better learning of the concepts. Also, the
ratio of the overall number of quality concepts to the total number of
datapoints is reduced due to more sharing of elements. In other words, as we
increase our action bank, by including more actions, chances of sharing
quality elements with an unseen action increase.

###  5.3 Fine-tuning to a Novel Action Class

In the final experiment, fine-tuning is used to adjust a pre-trained model for
a new unseen action. The experiment considers the situation where there are
limited number of samples in the dataset of a newly considered action to learn
with (_i.e_. it is difficult to obtain many labeled samples as might be the
case for an obscure action).

In particular, the all-action model is pre-trained on five actions and fine-
tuned on the remaining unseen action with minimal datapoints. For the more
data rich actions (Diving, Gymvault, Skiing, and Snowboarding) {25, 75, 125}
training samples are used, while for data poor (Sync. Dive 3m/10m) only {15,
25, 35} training samples are used for fine-tuning. Testing is performed on 50
of the remaining samples for all actions. The settings are the same for this
experiment except the hyperparameters are directly proportional to the
training set size – a very simple rule. The learning rate step size is
adjusted as {100, 300, 500} and training iterations of {500, 1200, 2000} for
the training set size of {25, 75, 125} respectively. Similar scaling was
performed for the {15, 35} training sizes as well.

Figure 3 shows Spearman’s rank correlation performance at every hundred
training iterations on the test set for each of the different training set
sizes for all the actions. Finetuning from multi-action pretrained model is
shown by blue-colored curves while red curves represent the result of
finetuning from scratch (_i.e_. random initialization). Overall, the blue
curve is above the red in most situations. Further, the all-action blue curve
quickly reaches a high value after only a few tens of iterations. The random-
initialization red curve takes longer to gradually update weights and climb
from a low starting value. Table 4 provides the the best performance during
training where we see that all-action is better in 16 of the 18 cases.

Note that even in the case of Gymvault and Skiing which seemed to have poor
all-action initialization (from the zero-shot AQA experiment in Table 3) the
performance quickly climbs during fine-tuning. Deep networks don’t typically
have smooth error surfaces. So, it could be the case that there are valleys
near good solution in cases of Gymvault and Skiing; and during finetuning, a
good solution was reached because relatively higher initial learning rate of
0.001 helped in escaping local minima. Individually setting hyperparameters
for each individual action may further enhance performance but is outside the
scope of this work. Instead the emphasis here is on the ability to learn
generalizable quality concepts by pre-training on multiple actions and
providing a better initialization for fine-tuning.

###  5.4 Discussion

The key outcome of this experimental evaluation was that learning AQA models
by learning from multiple actions is advisable. The experiments showed that
there are quality concepts shared among actions that can be learned. In
addition, another advantage of using the multi-action training procedure is
enhanced scalability. We also followed a simple rule to set the
hyperparameters. Moreover, pre-training with datapoints from all actions
provides a better initialization point for faster convergence with the need
for fewer training samples when fine-tuning for a new action. One experiment
that was not performed is using the all-action model for pre-training and
fine-tuning for a single action. However, this is not advisable since the data
has already been incorporated into the all-action model and may lead to over-
fitting.

Actions that we have considered in this work, although from different sports,
have common action elements which is a condition necessary for the transfer to
work. We have not yet explored transferring or sharing knowledge among actions
from different domains. For _e.g_., our approach is not intended to share
knowledge between a surgery task and a completely different task like
gymnastic vault. To be able to apply our approach, tasks should be from same
domains. In the surgery skills domain, one should consider sharing knowledge
between tasks like Knot-tying, Needle-passing and Suturing. We are looking
forward to exploring cross-domain knowledge sharing in future works.

##  6 Conclusion

This work demonstrates that like many other computer vision tasks, such as
object classification or action recognition, action quality assessment (AQA)
can benefit from knowledge transfer/sharing by training a shared model across
samples from multiple actions. We experimentally demonstrated this on our
newly introduced dataset, AQA-7. The experiments showed: 1) that by
considering multiple actions, limited per-action data is better leveraged for
improved per action performance (leveraging data more efficiently), 2) multi-
action pre-training provides better initialization for novel actions, hinting
at an underlying consistency in the notion of quality in actions
(demonstrating that all-action model is generalizable than single-action
models). The results hint at the potential to extend AQA beyond sports
scenarios into more general actions.

##### Acknowledgments:

We would like to thank Nicolas Gomez, Ryan Callier, and Cameron Carmendy for
helping us with dataset collection.

## References

  * [1] F. Abramovich and M. Pensky.  Classification with many classes: challenges and pluses.
  * [2] G. Bertasius, H. S. Park, X. Y. Stella, and J. Shi.  Am i a baller? basketball performance assessment from first-person videos.  In Computer Vision (ICCV), 2017 IEEE International Conference on, pages 2196–2204. IEEE, 2017.
  * [3] M.-J. Borzilleri.  Olympic figure skating controversy: Judging system is most to blame for uproar, 2014.
  * [4] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, pages 248–255. IEEE, 2009.
  * [5] H. Doughty, D. Damen, and W. Mayol-Cuevas.  Who’s better, who’s best: Skill determination in video using deep ranking.  arXiv preprint arXiv:1703.09913, 2017.
  * [6] E. Garcia.  A tutorial on correlation coefficients.
  * [7] R. Girshick, J. Donahue, T. Darrell, and J. Malik.  Region-based convolutional networks for accurate object detection and segmentation.  IEEE transactions on pattern analysis and machine intelligence, 38(1):142–158, 2016.
  * [8] S. Hochreiter and J. Schmidhuber.  Long short-term memory.  Neural computation, 9(8):1735–1780, 1997.
  * [9] K. Jarrett, K. Kavukcuoglu, Y. LeCun, et al.  What is the best multi-stage architecture for object recognition?  In Computer Vision, 2009 IEEE 12th International Conference on, pages 2146–2153. IEEE, 2009.
  * [10] Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, and T. Darrell.  Caffe: Convolutional architecture for fast feature embedding.  arXiv preprint arXiv:1408.5093, 2014.
  * [11] A. Karpathy, G. Toderici, S. Shetty, T. Leung, R. Sukthankar, and L. Fei-Fei.  Large-scale video classification with convolutional neural networks.  In CVPR, 2014.
  * [12] D. P. Kingma and J. Ba.  Adam: A method for stochastic optimization.  arXiv preprint arXiv:1412.6980, 2014.
  * [13] Z. Luo, Y. Zou, J. Hoffman, and L. F. Fei-Fei.  Label efficient learning of transferable representations acrosss domains and tasks.  In Advances in Neural Information Processing Systems, pages 164–176, 2017.
  * [14] I. Misra, A. Shrivastava, A. Gupta, and M. Hebert.  Cross-stitch networks for multi-task learning.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3994–4003, 2016.
  * [15] P. Parmar and B. T. Morris.  Measuring the quality of exercises.  In Engineering in Medicine and Biology Society (EMBC), 2016 IEEE 38th Annual International Conference of the, pages 2241–2244. IEEE, 2016.
  * [16] P. Parmar and B. T. Morris.  Learning to score olympic events.  In Computer Vision and Pattern Recognition Workshops (CVPRW), 2017 IEEE Conference on, pages 76–84. IEEE, 2017.
  * [17] H. Pirsiavash, C. Vondrick, and A. Torralba.  Assessing the quality of actions.  In European Conference on Computer Vision, pages 556–571. Springer, 2014.
  * [18] L. Y. Pratt.  Discriminability-based transfer between neural networks.  In Advances in neural information processing systems, pages 204–211, 1993.
  * [19] K. Soomro, A. R. Zamir, and M. Shah.  Ucf101: A dataset of 101 human actions classes from videos in the wild.  arXiv preprint arXiv:1212.0402, 2012.
  * [20] A. Torralba, K. P. Murphy, and W. T. Freeman.  Sharing visual features for multiclass and multiview object detection.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(5):854–869, 2007.
  * [21] L. Torrey and J. Shavlik.  Transfer learning.  In Handbook of research on machine learning applications and trends: algorithms, methods, and techniques, pages 242–264. IGI Global, 2010\.
  * [22] D. Tran, L. Bourdev, R. Fergus, L. Torresani, and M. Paluri.  Learning spatiotemporal features with 3d convolutional networks.  In Computer Vision (ICCV), 2015 IEEE International Conference on, pages 4489–4497. IEEE, 2015.
  * [23] V. Venkataraman, I. Vlachos, and P. Turaga.  Dynamical regularity for action analysis.
  * [24] Wikipedia contributors.  List of olympic games scandals and controversies — Wikipedia, the free encyclopedia, 2018.  [Online; accessed 29-June-2018].
  * [25] J. Yosinski, J. Clune, Y. Bengio, and H. Lipson.  How transferable are features in deep neural networks?  In Advances in neural information processing systems, pages 3320–3328, 2014.
  * [26] A. R. Zamir, A. Sax, W. Shen, L. J. Guibas, J. Malik, and S. Savarese.  Taskonomy: Disentangling task transfer learning.  In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018.
  * [27] J. Zhang.  Multivariate analysis and machine learning in cerebral palsy research.  Frontiers in neurology, 8, 2017.
  * [28] A. Zia, Y. Sharma, V. Bettadapura, E. L. Sarin, T. Ploetz, M. A. Clements, and I. Essa.  Automated video-based assessment of surgical skills for training and evaluation in medical schools.  International journal of computer assisted radiology and surgery, 11(9):1623–1636, 2016.