# Extracting Fast and Slow: User-Action Embedding with Inter-temporal
Information

Akira Matsui,1, Emilio Ferrara2,3,4 Corresponding author: matsui-akira-
<EMAIL_ADDRESS>
###### Abstract

With the recent development of technology, data on detailed human temporal
behaviors has become available. Many methods have been proposed to mine those
human dynamic behavior data and revealed valuable insights for research and
businesses. However, most methods analyze only sequence of actions and do not
study the inter-temporal information such as the time intervals between
actions in a holistic manner. While actions and action time intervals are
interdependent, it is challenging to integrate them because they have
different natures: time and action. To overcome this challenge, we propose a
unified method that analyzes user actions with intertemporal information (time
interval). We simultaneously embed the user’s action sequence and its time
intervals to obtain a low-dimensional representation of the action along with
intertemporal information. The paper demonstrates that the proposed method
enables us to characterize user actions in terms of temporal context, using
three real-world data sets. This paper demonstrates that explicit modeling of
action sequences and inter-temporal user behavior information enable
successful interpretable analysis.

## Introduction

Human temporal behaviors contain a wide range of valuable information. Mining
human physical, social, or economic dynamic behavior has contributed greatly
to our understanding of human mobility, social phenomena, or consumption
behavior. However, most human dynamic behaviors models focus only on the
sequence of users’ actions. For example, statistical time-series models study
the variation of values in the data over the course of time. While this
analysis is very natural in many contexts, those analysis does not explicitly
model or integrate inter-time information such as time intervals between
actions. Despite the importance of the information about time intervals for
human behavior analysis, the existing user models in computer science mainly
focus on sequences of actions and do not integrate time-interval information
into action.

Time intervals between actions provide crucial information about the context
of those actions. There is much literature that shows that time intervals
between actions can tell the human cognitive states (Stanovich and West 2000;
Graesser, Millis, and Zwaan 1997; Wickens and Carswell 2021; Kahneman 2011).
The most famous theory in the literature and popular science is “Dual-process
theory” introduced by (Graesser, Millis, and Zwaan 1997) and developed by
(Kahneman 2011). In addition to the cognitive process that generates action
time intervals, the statistical properties of the time interval distributions
have attracted attention from researchers, especially in network science
(Holme and Saramäki 2012; Masuda and Lambiotte 2016; Vázquez et al. 2006;
Barabasi 2005; Oliveira and Barabási 2005).

To take into account the context when mining and interpreting actions, a model
that incorporates the information of time intervals into action is needed. An
obstruction that prevents us from modeling user actions and their time
intervals in a holistic way is the fact that actions and time intervals are
incompatible. Action is categorical data, and time interval is numerical data.
One natural solution is to discretize time intervals using several time bins
(Wang et al. 2016). However, making time bins requires hyperparameters, and
validating those predetermined parameters is a cumbersome process. In
addition, even if we successfully discretize time intervals, it is not trivial
how to incorporate time interval information into actions.

To overcome the aforementioned challenges, this paper proposes a model that
embeds inter-temporal information to obtain the low-dimensional
representations of users actions. Firstly, to discretize time intervals, we
leverage the statistical properties of time intervals that have been well
exploited. Our model estimates as a mixture exponential distribution and use
those estimated parameters to group the time intervals. Leveraging that
statistical procedure can ensure the objectivity of the analysis avoiding the
hyper-parameters selection for time bins. Also, this procedure ensures that
discretized time intervals capture the nature of observed time intervals. We
then construct the action sequences where action and its time intervals show
up one after the other and make n-grams from them. Then our framework
calculates a low-dimensional representation of user actions with time interval
information using a word embedding technique. Lastly, we study the inter-
temporal context for each action using the obtained action embedding vectors.
To do so, we propose an interpretable measurement of action timing context
(ATC), which represents whether a given action belongs to a long-term (fast
pace) context or a short-term (slow pace) context. We illustrate the schematic
of our study in Figure 1 and summarize the terminologies in Table 1 to make
our paper easy to follow.

With the proposed model, we conduct the three empirical studies to reveal the
inter-temporal context of users’ actions using ATC. We firstly show the inter-
temporal context of actions corresponds to apps’ categories. Second, we
demonstrate that ATC can compare the behavior differences of different types
of students in terms of their temporal behavior. The second analysis
investigates the behavior differences between the drop-out and non drop-out
students in a Massive Open Online Courses (MOOC) platform. In the third study,
we study the dynamics of the context of actions. With a student behavior
dataset over an academic term, we demonstrate that the inter-temporal context
of actions changes according to the event on the campus, such as the mid-term
weeks. Those three empirical studies show that our framework and proposed
measurement (ATC) captures the inter-temporal context of actions in an
interpretable way.

Figure 1: Schematic of studying the inter-temporal context of user actions

Note: (a) We firstly construct the action sequence of each user and calculate
time intervals between two consecutive actions.(b) Then, we estimate the
mixture distribution of time intervals to discretize the time intervals. (c)
With the discretized time-intervals and action, we construct the sequences of
action $+$ time interval. Also, we construct trigrams from those sequences.
(d) Using the data constructed at (c), We learn the embedding vector of
actions and trigrams. (e) We study the inter-temporal context of action
extracting Action Timing Context (ATC) from obtained embedding vectors.

Table 1: Glossary of Terms Terminology |  Description
---|---
Action |  Single unit of the observed behavior (e.g., user smartphone application, physical contact, …)
Time interval |  A time gap between two consecutive actions
ATC |  Action Timing Context defined at Eq 7 in Section Extracting action timing context (ATC) using n-gram actions.

Finally, we summarize the contributions of this paper as:

  * •

Constructing a framework to extract the inter-temporal context of actions

  * •

Proposing an interpretable measurement that represents the inter-temporal
context of action

  * •

Conducting several empirical studies to demonstrate our framework reveal
inter-temporal human behaviors

## METHOD

This paper aims to study human behavior considering inter-temporal
information. For this aim, our method will try to obtain a low-dimensional
representation of the action sequence. We construct action sequences of each
user with the actions and their time interval and then utilize a word
embedding method on the constructed action sequences.

### Capturing inter-temporal information with time bins

This subsection will discuss how we estimate the time intervals and convert
them into the action sequence using time bins. While calculating time
intervals between consecutive actions is straightforward, it is not trivial to
discrete this continuous-time value to put them in the action sequence. A
natural solution for this discretization problem is to let some time bins be
as hyperparameters. For example, (Wang et al. 2016) use predefined bins such
as ($[0-1min],[1min-10min],[10min-60min],[60min<]$) and then classify
calculated time intervals into those time bins. However, since there are
numerous patterns of the time bins, it isn’t easy to make an appropriate
pattern of time bins without any prior information. Such a combination of time
bins without considering the features of the time intervals in the data, such
combination of the time bins might miss the inter-temporal information between
actions, which is the primary interest of this study.

Therefore, this study will estimate the statistical property of the time
intervals of data to construct the time bins based on the estimated parameters
rather than using hyperparameter time bins. The goal of this time bins
construction is to capture the behavior states of the users from their time
intervals and represent the behavior states by the time bins. For this aim, we
study the behavior states estimating the mixture distributions.

### Time interval bins estimation by a mixture of exponential distributions

This paper estimates the time interval distribution between actions and
constructs the time bins based on the estimated distribution. We assume that
the observed time intervals follow the mixture of exponential distributions.
To estimate the mixture of exponential distributions, we utilize the EM
algorithm and the model selection criterion.

Recent development in the literature reveals that the mixture of the
exponential distribution with a few components can well fit the time intervals
of various empirical data (Okada, Yamanishi, and Masuda 2020)111 (Okada,
Yamanishi, and Masuda 2020) uses a slightly different wordings than this
paper. They describe actions in this paper as “events”, and time intervals as
“inter-event times”. To keep our wordings consistent, we use our terminology
in Table 1.. Following (Okada, Yamanishi, and Masuda 2020), we estimate the
mixture of the exponential distribution using the EM algorithm and selecting
the number of components by the model selection criteria.

#### EM algorithm:

To study the mixture of exponential distribution from the observed time
interval between actions $x=\left\\{x_{1},\ldots,x_{n}\right\\}$ , we estimate
the parameters in the following equation,

$\textbf{f}\left(x\right)=\sum_{k=1}^{K}\pi_{k}f_{k}\left(x;\lambda_{k}\right)$
(1)

, where of the time $\pi_{k}$ is the mixture parameters (mixing weights),
$\pi_{k}\geq 0$ and $\sum_{j=0}^{K}\pi_{j}=1$; $f$ is the probability density
function (PDF) of an exponential distribution and $\lambda_{k}$ is the
parameter of the exponential distribution of $k$ component (rate parameter).

To estimate the parameters $\pi_{k}$ and $\lambda_{k}(k=1,\dots,K)$, the EM
algorithm takes E (Expectation) step and M (Maximization) step. At E step
during the iteration $t$, the algorithm calculates so called membership
probability (weight) $r^{t}_{ik}$ by,

$\gamma^{t}_{ik}=\frac{\pi_{k}^{t}f(x_{i};\lambda_{k}^{t})}{\sum_{l=1}^{K}\pi_{l}^{t}f(x_{i};\lambda_{l}^{t})}.$
(2)

Intuitively, the membership weight $r^{t}_{ik}$ describes the probability of a
given data (time interval, $x_{i}$) is generated from the component $k$ (i.e.,
the exponential distribution with the parameter $\lambda_{k}$).

With the membership weight calculated at the E step at the iteration $t$, the
algorithm calculates the parameters as the M step as

$\displaystyle\pi_{k}^{(t+1)}=\frac{1}{N}\sum_{i=1}^{N}\gamma_{ik}^{(t)},\quad\mu_{k}^{(t+1)}=\frac{\sum_{i=1}^{N}\gamma_{ik}^{(t)}x_{i}}{\sum_{i=1}^{N}\gamma_{ik}^{(t)}}.$
(3)

Then the algorithm iterates those two steps and obtains the final parameters
after they converge.

As the equations 2 and 3 show, the number of components $K$ is given in the
algorithm. For the criteria to select the number of components (model
selection), we use decomposed normalized maximum-likelihood (DNML) codelength
following (Okada, Yamanishi, and Masuda 2020).

#### Attributing time intervals to time bins:

We attribute the time intervals between actions to the time bins based on the
estimated mixture of exponential distribution. We construct as many time bins
as the components of the mixture distribution and the bin for zero time
interval $T_{0}$. In other words, we construct $K+1$ time bins
$\\{T_{0},\ldots T_{K}\\}$, when we have $K$ components. For a given time
interval $x_{i}>0$, we find the time bin $T^{*}$ fir $x_{i}$ using the
membership probability (Eq 2) as

$\displaystyle
T^{*}=\operatorname*{argmax}_{l}\frac{\pi_{l}f\left(x_{i};\lambda_{l}\right)}{\sum_{k=1}^{K}\pi_{k}f\left(x_{i};\lambda_{k}\right)}.$
(4)

Since $f$ is the PDF of exponential distribution, the time bin attribution
cannot be calculated for $x_{i}=0$, and this is the reason that we make the
special time bin $T_{0}$ for $x_{i}=0$.

### Action sequence and N-gram

To represent the context of the actions by users, we first construct the
sequence of actions and their time intervals where they are in chronological
order. Then we construct N-grams from those actions sequences.

#### Action sequence:

With the actions and time intervals between them, we construct action
sequences for each user. Let User $i$ have the actions
$\\{A_{i,j}\\}_{j=0}^{J}$. His/her j th action $A_{i,j}\in\\{A_{1}\ldots
A_{M}\\}$ where $M$ is the number of unique actions, and its time interval is
$x_{ij}$ that is the time interval between $A_{i,j}$ and $A_{i,j+1}$. By using
the time bin constructed in with the equation 4, we attribute time interval
$x_{ij}$ to a time bin $T_{ij}$, which is one of $\\{T_{0},\ldots T_{K}\\}$.
With User i’s actions $\\{A_{i,j}\\}_{j=0}^{J}$ and their time intervals
$\\{T_{i,j}\\}_{j=0}^{J}$, we construct User i’s action sequence as

$A_{i,1}T_{i,1}A_{i,2}T_{i,2}A_{i,3}\ldots T_{i,J-2}A_{i,J-1}T_{i,J-1}A_{i,J}$

We construct this action sequence for each user and we will make N-grams from
those actions sequences.

#### Action N-gram:

Our action n-gram captures the context of users’ actions, representing the
order of actions and their time interval. For an action sequence 222 We use
$A_{j}$ and $T_{j}$ for $A_{i,j}$ and $T_{i,j}$ respectively, abbreviating the
notation $i$ for user  $A_{1}T_{1}\ldots T_{J-1}A_{J}$, we make n-gram
separating action $A_{j}$ and $T_{j}$. In this study, we use trigram ($n=3$)
of the action sequence, and it will be

${\\{A_{1}T_{1}A_{2},\ T_{1}A_{2}T_{2},\ \ldots T_{J-2}A_{J-1}T_{J-1},\
A_{J-1}T_{J-1}A_{J}\\}}$

While the others may prefer to use the set of action and its time interval as
one unit to make n-gram such as $A_{J}T_{J}$, making n-gram with separating
those two will benefit from studying the time interval context of the action
(to be discussed in Section Extracting action timing context (ATC) using
n-gram actions).

### Embedding

We use the Skip-gram with negative sampling (SGNS) algorithm to obtain low-
dimensional representation of actions and N-grams (Mikolov et al. 2013a, b).
This section will discuss the prediction problem that SGNS solves in this
paper to understand the embedding vectors to be obtained from the action
sequence. The SGNS model the distribution, $p(d\mid w,c)$, where $d$ takes 1
when a pair of word $c$ and context $c$ is observed in the data otherwise 0.
The SGNS maximize the following conditional log-likelihood $\mathcal{L}$ (Dyer
2014),

$\mathcal{L}=\sum_{(w,c)\in\mathcal{D}}\\{\log p(d=1\mid c,w)\\\
+kE_{\bar{w}\sim q}\log p(d=0\mid c,\bar{w})\\},$

where $q$ is the noise distribution in negative sampling, and k is the sample
size from the noise333More precisely, this log-likelihood maximization with
the noise distribution is considered as Noise Contrastive Estimation (NCE)
(Gutmann and Hyvärinen 2010), and the SGNS is one of the variations of NCE.
The SGNS especially calculates the conditional probability $p(d=1\mid c,w)$ by
$\sigma(v_{c}\cdot v_{w})$, where $\sigma(x)$ is sigmoid function and
$v_{w},v_{c}\in R^{d}$. In other words, the SGNS seeks the parameter
$v_{w},v_{c}$ that maximizes the above conditional probability, and
$v_{w},v_{c}$ is the embedding vectors of interests.

#### “Word” and “Context” in this paper:

This paper treats actions and n-grams of action sequences as either contexts
or words. Note that we treat actions as both words and contexts, and we treat
n-grams of action sequences as well. We calculate the conditional probability
of actions/n-gram given actions/n-gram to obtain the low-dimensional
representation. For implementation, we use the ngram2vec (Zhao et al. 2017),
which is the modified version of word2vecf (Levy and Goldberg 2014).

### Extracting action timing context (ATC) using n-gram actions

In order to contextualize the actions with the inter-temporal information, we
use the embedding vectors of the n-grams. As discussed in Section Action
N-gram:, n-grams contain elements that have an action between time intervals,
such as $TAT$. We leverage the embedding vector of this type of n-gram as the
references that represent the inter-temporal context.

#### Constructing the reference vectors:

To do so, we firstly make the two types of reference n-grams: the long and
short time interval context. Let $T_{long}$ and $T_{short}$ be the longest and
the shortest time interval bin, respectively. Note that those bins are defined
in Section Time interval bins estimation by a mixture of exponential
distributions. Then, we calculate the reference vector for the long and the
short interval context as

$\displaystyle{v}_{long}=\frac{1}{|\mathcal{V}_{long}|}\sum_{v\in\mathcal{V}_{long}}v$
(5)

, where $V_{long}$ is the set of the embedding of the n-gram where actions are
between the longest time intervals. That is,

$\displaystyle\mathcal{V}_{long}=\\{T_{long}AT_{long}\\}_{A\ \in\mathcal{A}}$
(6)

, and $\mathcal{A}$ is the set of all actions. Similarly, we make
$\mathcal{V}_{short}$ using $T_{short}$.

#### Definition of the “long term context” and “short term context”:

The reference vectors, $\mathcal{V}_{long}$ and $\mathcal{V}_{short}$
represent an action taken between the long term intervals or the short term
intervals. For example, when an action $A$ is similar to ${V}_{long}$, it
means that the user takes a long time break (interval) before/after that
action. We can interpret such action as the action that the users spend a long
time execution time. On the other hand, when an action $A$ is similar to
${V}_{short}$, it means the users tend to execute with a short-term execution
time.

#### Aligning actions into long vs. short term context:

To study the time interval context of a given action $A$, we calculate the
relative distance $r(A)$ between the two reference vectors for each action of
interest,

$\displaystyle r(A)=cos({v}_{long},a)-cos({v}_{short},a)$ (7)

where $a$ is the embedding vector of action $A$, and $cos$ is the cosine
similarity. When $r(A)$ is large, action $A$ is in the long term context; the
other means $A$ is in the short term context. We provide a schematic
illustration of this relative distance in Figure 2.

Figure 2: Extracting action timing contexts with n-gram action

Note:  We calculate the relative distance from long-term and short-term
context n-gram. We firstly construct the vector of long-term ${v}_{long}$ and
short-term ${v}_{short}$ context n-gram (as Equation 5 and 6). Then, for each
action vector $a$ of Action, we calculate defences between the cosine
distance: $cos({v}_{long},a)$ and $cos({v}_{short},a)$ This relative distance
$r(A)$ defined at 7 represents Action belongs to which context in their action
timing (A large $r(A)$ suggests that action is used in the long term context).

## Data & Experiment Settings

In this dataset, we will discuss the dataset and experiment setting for the
embedding model. We employ the three datasets to investigate the wide range of
human behavior in terms of inter-temporal context using ATC discussed in
Section Extracting action timing context (ATC) using n-gram actions.

Table 2: Basic dataset statistics

Dataset Statistics |  App Usage |  MoocData |  StudentLife
---|---|---|---
Observation period |  1 week |  2 years 11 days |  11 weeks
Observation date |  Jun’16 |  Jun’15-Jun’17 |  Mar’13 - May’13
Observation field |  Smartphone |  MOOC platform |  University campus
Environment |  Digital device |  Digital platform |  Digital device and Real place
# unique actions |  1,696 |  22 |  800
# total users |  871 |  225,642 |  49
# total actions |  $4,171,950$ |  $42,110,402$ |  $219,360$
Avg. # actions per user |  $4789.83$ |  $186.62$ |  $4476.73$
Avg. # unique actions per user |  $1.94$ |  $9.74$ |  $16.326$
# components of the mixture of exp distributions |  $3$ |  $3$ |  $3$

Note: The basic statistics of the three datasets used for our analysis. This
table reports the basic statistics of the data sets after preprocessed
described in Sec Data & Experiment Settings. # components is determined by
DNML as discussed in Sec Time interval bins estimation by a mixture of
exponential distributions.

### Datasets

We apply our method to the three data sets. We firstly use the app usage
history dataset (Feng et al. 2019) to study the correspondences between the
inter-temporal context differences (ATC) and the category of the apps. Then,
we use ATC to study the behavior difference between different individuals to
demonstrate that our method captures human behavior in terms of inter-temporal
context. For this analysis, we use the clickstream data from the MOOC platform
(Yu et al. 2018). Lastly, we turn to the dynamics of ATC, studying the student
behavior sensing data (Wang et al. 2014).

App Usage Dataset (Feng et al. 2019) We use the smartphone app usage history
data published by (Feng et al. 2019). The dataset provides the user history of
smartphone app usage, including the timestamp and the category of the app. We
consider the usage of the app as actions and labeled with its categories.

MoocData (Yu et al. 2018) MoocData is the clickstream data on the users in the
XuetangX platform (the MOOC platform in China). The dataset contains the
users’ learning activities logs, including what functions of the platform they
use during their learning. While the dataset provides a massive amount of
logs, it also serves as the dataset for the dropout prediction. We utilize
this subset of the dataset to compare the behavior differences between two
distinct types of users in terms of academic performance: Dropout VS Non-
Dropout.

StudentLife (Wang et al. 2014) StudentLife dataset provides a wide range of
behavior data from automatic sensing using participants’ smartphone. (Wang et
al. 2014) recruits the students for the study and track their behavior on the
campus during one academic term. The dataset contains the smartphone usage
history and eating behavior (e.g., lunch), physical activities (e.g.,
walking), etc. In addition to the action data, this dataset provides the
survey data based on Ecological Momentary Assessment (EMA). The EMA data
contains the self-report of the physiological state of the students throughout
the data period. We use the three questionnaires to study the correlation
between the ATC of students’ activity and the students’ psychological state.

Table 3: EMA Questions

Question |  Option (scale)
---|---
how happy do you feel? |  1.little bit, 2.somewhat, 3.very much, 4.extremely
how sad do you feel? |  1\. little bit, 2.somewhat, 3.very much, 4.extremely
How are you right now? |  1.happy, 2.stressed, 3.tired
How many hours did you sleep last night? |  18 scales (0.5-hour grid from less than 3 hours to more than 12 hours)

Note: The EMA questions used in this paper. The participants who only answer
“yes” to “Do you feel AT ALL happy (sad) right now?” answer the first two
questions (“how happy(sad) do you feel? ”).

We utilize the above three datasets to demonstrate that ATC can capture human
behavior from the inter-temporal point of view.

### Experiment Settings

As discussed in Sec Time interval bins estimation by a mixture of exponential
distributions, we estimate the mixture of time intervals between actions to
construct the time bins. We use the implementation by (Okada, Yamanishi, and
Masuda 2020). For each dataset, we randomly sample 10k time intervals and
estimate the distributions.

For embedding, we use the ngram2vec implemented by (Zhao et al. 2017) based on
word2vecf (Levy and Goldberg 2014). We use SGNS to learn embedding vectors
(300 dim). We use a flexible window size: two windows for bi-gram and one
window for actions as illustrated in Fig 1 (d) to ensure that our embedding
model captures the dependency among actions and time intervals. We remove the
users who have less than 10 action

## Empirical analysis with action timing context (ATC)

This section reports the empirical analysis of extraction of action timing
contexts with real-world datasets. Our empirical analysis to demonstrate the
action timing context extracts the informative insights of human behavior from
the action sequence. To this aim, we utilize the three different datasets: the
student behavior data observed in a field study, the smartphone app usage
data, and the students’ behavior in a Massive Open Online Courses (MOOC).

Our empirical analysis compares the differences in action timing contexts
between different types of entities. First, we show that the ATC of app usage
depends on its category. Since each application has a different purpose, it is
evident that these differences can reflect on its ATC. Next, we utilize our
method to find behavior differences between successful students and those not
in the academic environment. We compare the differences in the ATC of the
dropout student and non-dropout student in the MOOC platform. Lastly, we study
the transition of the ATC of the student behavior in a real academic
environment. We use the datasets covering physical and digital behavior
(smartphone usage) over an academic term.

### ATC differences among smartphone apps usage

Smartphone apps are an essential tool for our modern daily life, providing a
wide range of functions such as games or health. Those apps are supposed to
have different ATCs depending on their purpose. To study this point, we
calculate the mean ATC for each app category. Figure 3 demonstrates that
different category apps have different ATC means.

In the figure, the Reference category app has the smallest ATC (-0.27),
meaning that the apps in the reference, such as the dictionary, are used in a
short span. On the other hand, Infant & Mom has the largest ATC (1.1). The
users of the apps for infant care and monitoring use those apps for the long-
term context. Our analysis also reveals that the apps that do not belong to a
definitive context. Finance category apps, for example, are around the median
ATC (0.27). Therefore, the finance category apps usage spans from the long
term to the short term context.

Figure 3: Smartphone Apps usage context

Note:  Comparisons of smartphone app usage by the action timing context
(standardized on all samples). The figure plots the mean value of each
category’s action timing context (with 95%CIs by bootstrap). While the apps
for “Infant & Mom” are used in the long action timing context, “Reference”
category apps are used in the short-term context. Also, the “Finance” category
apps are neutral in terms of the action timing, suggesting that the users use
those apps in both contexts.

### ATC differences between dropout and non-dropout students

Our method can reveal the behavior differences in an interpretable way. To
demonstrate this, we use the student clickstream data from the MOOC platform.
Along with the clickstream, the datasets provide labels of dropout or not for
each student. We split the dataset into the data of dropout students and non-
dropout and then learn embedding from each to calculate ATC. By doing this, we
can compare ATCs of the same action but different types of students
(dropout/non-dropout).

Figure 4 plots the differences in ATC of the same action between the dropout
and non-dropout students. The figure calculates the differences by subtracting
the ATC of dropout students from one of the non-dropout. Therefore, a positive
difference in ATC of action implies that the dropout students use that action
in the long term, but the non-dropout students do in the short term.

The most evident distinction in the figure is in “Pause Video” (0.45). This
positive difference suggests that the dropout students do not often pause the
course video, but the non-dropout students pause a video in a short span in
their learning. Contrastingly, we find the negative differences in commenting
(ATC of “Delete Comment”: $-0.61$; “Create Comment”: $-0.51$), revealing that
the non-drop out students take their time for their commenting compared to
dropout students in their learning. Although we find the differences in ATC
between the two distinct types of students, they are similar in their ATC of
“Close Courseware” (-0.01). Combining these three differences will shed light
on the behavior differences between dropout and non-drop students. Compared to
non-dropout students, dropout students are less likely to “pause” the video
when it plays and less likely to spend a lot of time commenting on it.

Figure 4: Action timing context differences: Drop-out students VS Non-dropout
students

Note:  Action timing context difference between the different types of users
(standardized on each student type). We calculate the action timing context of
the students in the MOOC platform. We calculate the action timing context for
the students who dropped out of their course and the student. A positive
difference of action represents the dropout users take that action in the long
term context, but the non-dropout students use that action in the short term
context. For example, the dropout students use “Pause Video” in the long-term
context, but the non-dropout students use it in the short-term (difference
0.45). This difference implies that the dropout students do not often pause
the course video, but the non-dropout students do so.

(a)

(b)

Figure 5: Transition of the action timing context of the students over the
academic term

Note: Calculating the action timing context of the student behavior over the
course of an academic term (11 weeks, standardized on each week). We calculate
the action timing context of the actions in each week: the physical behavior
(e.g., walking), the smartphone app usage (e.g., YouTube), and the eating
activity (e.g., lunch). Figure 5(a) shows the ATCs changes over the academic
terms. The area between the dot lines represents the mid-term weeks (Week4-6).
While the context of physical behavior is stable, eating activity transitions
from the long-term context to the short-term context. This transit implies
that the important academic event off the students stride in their eating
habit. Figure 5(b) plots the box plot of the action timing context of the
actions, and shows that while the physical activities are stable, the eating
activities are volatile (having longer bars).

Figure 6: Correlations between ATCs of the eating actions and Physiological
state

Note The figure report the correlation between ATCs of the eating
actions(Breakfast, Lunch, Supper, Snack) and the EMA (weekly average)
described in Table 3. We use all available EMA answers during the period (11
weeks) for each question. On each subfigure, we report the Pearson correlation
coefficient ($r$) and p-value for testing non-correlation, and the blue line
represents the fitted simple linear regression.

Figure 7: Correlations between ATCs of the physical actions and Physiological
state

Note: The figure reports the correlation between ATCs of the physical actions
(Walking, Running, Other Activity) and the EMA (weekly average) described in
Table 3. We use all available EMA answers during the period (11 weeks). On
each subfigure, we report the Pearson correlation coefficient ($r$) and
p-value for testing non-correlation, and the blue line represents the fitted
simple linear regression.

### Capturing behavior dynamics by ATC

Lastly, we use our method to study a dynamic behavior from the action
sequence. We utilize the StudentLife dataset to analyze how ATC transit over
an academic term (11 weeks). StudentLife dataset provides a wide range of
students’ behavior trace from their smartphones such as physical activity,
eating behavior, or apps usage. We first construct action sequences of the
students and split them into weeks (i.e., 11 weeks). Then we obtain the
embedding vectors from each week and calculate the ATC of each action.

Figure 5(a) plots the transition of ATCs over the course of the academic term.
The figure shows that the ATCs of physical behavior settles over the term but
the ATCs of eating behavior swings around Week 45 where the students had the
midterm. This suggests that the students compress the time interval between
their eating actions during the mid-term to save their time for their study.
After the mid-term, their eating behavior gets stable in terms of ATC. The ATC
of the eating behavior remains a plateau after Week 6. For reverence, we
picked up the popular smartphone apps among the participants (Gmail and
Youtube), which are between physical and eating behaviors.

The result above shows that the ATC can reveal the action that can be affected
by external circumstances. Figure 5(a) tells that eating behaviors can change
according to an educational event (the mid-term), but the physical behaviors
do not. The box plot in Figure 5(b) demonstrates this difference in the
vulnerability of ATC, where the ATCs of the physical behaviors have shorter
bars and ones of the eating behavior have more extended bars.

Finally, we study the correlation between ATCs and students’ physiological
states. Figure 6 plots the correlation between the answer of EMA by students
and the ATCs of the eating actions (Breakfast, Lunch, Supper, Snack). Figure 6
shows that the eating ATCs are correlate with the mental state (happy or sad).
ATC of eating action is positively correlated with the happy mood, and it is
negatively correlated with the sad mood. On the other hand, they are not
correlate with the question about sleep or question about moods in general
(“How are you right now?”) . Those correlations suggest that eating in the
short term (low ATC) is associated with a sad mood and vice versa. We also
study the correlation between the EMA and the ATCs of the physical actions
(Walking, Running, Other Activity), and we do not find any clear correlation.
We acknowledge that they are just correlations found by a simple linear
regression model. However, this result suggests a promising result that our
model with the inter-temporal context between actions may capture the user’s
psychological state.

## Related Research

This section discusses the related work to our study. We firstly discuss the
research on user modelings with embedding methods and human temporal behavior
using embedding techniques. Then, we study the research on the inter-action
times of human behavior.

### User modeling with embedding

Word embedding models have been mainly used to obtain the low dimensional
representation of word semantics (Mikolov et al. 2013a, b). It is also known
that word embedding can capture the biases hidden in the text data (Gonen and
Goldberg 2019; Bolukbasi et al. 2016; Garg et al. 2018; Caliskan, Bryson, and
Narayanan 2017) or semantic dynamics (Hamilton, Leskovec, and Jurafsky 2016;
Di Carlo, Bianchi, and Palmonari 2019).

The applications of such embedding techniques are not only in natural language
processing but also in user modeling. The most popular application is the
models that take embedding vectors of user consumption as feature vectors for
recommendations (Grbovic et al. 2015; Chen et al. 2018) or online
advertisement (Ren et al. 2018; Djuric et al. 2014; Attenberg, Pandey, and
Suel 2009; Zhang, Chen, and Wang 2016). In addition, there is a line of
literature that employs the temporal points process to model the dynamics of
user behaviors (Du et al. 2016; Sharma et al. 2021)

For modeling user dynamic behavior, (Han et al. 2020) propose an embedding
model that obtains the vector-reorientation of user dynamic from the user
actions sequence. Also, it (Liu et al. 2020) predicts the students’ academic
performances by embedding the students’ daily behavior sequences.

Recently, it has been found that the embedding model can construct the axis to
study the hidden structure of the platform. (Waller and Anderson 2021) Propose
the method and application to study the social phenomena in the online
community (Reedit) in a data-driven way. They characterize the online
community (sub-reedit) by the users’ posts and study the political
polarization dynamics.

The major difference between our study and the studies discussed above is that
our study explicitly embeds temporal structure into actions. Existing studies
use action sequences to represent the dynamics of user behavior and abstract
the time intervals between actions, even though, as mentioned in the
introduction, temporal information is necessary to understand human behavior.
Certainly, using point processes and other methods can improve prediction
accuracy, but such models complicate understanding human behavior. In this
study, we explicitly model the temporal structure in the embedding of behavior
by estimating the time interval. This makes the model structure very simple
and enables a unified analysis of user behavior using indicators such as ATC
as proposed in this study.

### Inter-action times of human behavior

Studying interactions between human actions and their time intervals reveals a
deep mechanism of human behaviors such as the human cognitive states
(Stanovich and West 2000; Graesser, Millis, and Zwaan 1997; Wickens and
Carswell 2021; Kahneman 2011). Many researchers have devoted themselves to
model human cognition based on this temporal information. The most popular
theory in this literature is “Dual-process theory” introduced by (Graesser,
Millis, and Zwaan 1997) and (Kahneman 2011) developed. We propose our
framework, ATC, to study the temporal structure of users’ actions based on
this simple yet solid foundation.

In addition to the cognitive process that generates action time intervals, the
statistical properties of the time interval distributions have attracted the
attention of researchers, especially in network science (Holme and Saramäki
2012; Masuda and Lambiotte 2016; Vázquez et al. 2006; Barabasi 2005; Oliveira
and Barabási 2005). Especially, (Okada, Yamanishi, and Masuda 2020) summarize
the literature that the two classes of models generate the distribution of the
time intervals as priority queue models (Barabasi 2005) and modulated Markov
processe (Holme and Saramäki 2012; Masuda and Lambiotte 2016)

## Conclusion & Discussion

In this study, we proposed a method to analyze users’ actions while
considering the inter-temporal context. Our framework is based on the idea
that the time interval between actions corresponds to users’ cognitive state.
To examine the cognitive state from the distribution of time intervals, we
discretized the time intervals based on mixture distribution estimation. Then,
we learned embedding vectors of actions based on the discretized time
intervals and action sequences and proposed ATC, an inter-temporal context
index for each action, using the obtained low-dimensional representation of
actions.

Our proposed framework conducted empirical studies on user behavior with the
three different datasets. As a result of the analysis, we found that ATC
captures actions and differences in behavior among users and how the inter-
temporal context changes depending on the situation. We also showed that ATC
is a unified and interpretable measure of inter-temporal context.

This paper has shown that incorporating the vital information, “time-
interval”, into the human behavior analysis can lead to a unified
understanding of human dynamic behavior. However, our new framework opens up
several exciting directions for future work. First, we need to analyze the
extent to which our framework captures human cognitive changes. While a simple
analysis with EMA data showed a promising result, we need to conduct more
rigorous analysis on this point. It needs to build a dataset that combines
behavioral data with subjects’ diachronic psychological changes for such an
analysis. Such an interdisciplinary study may open a new academic research
topic. For example, the dual-process theory assumes binary human cognitive
behavior. An analysis with large-scale and high-dimensional data, however,
might make it possible to analyze these cognitive states in detail, such as
how people switch cognitive states capturing the transition between states not
as a binary transition but the continuous change of cognitive state.

Lastly, there is a need to analyze the user’s behavior with the embedding
vectors of actions. This paper focused on the inter-temporal context of
actions. It is natural to calculate the user-behavior vector that represents
the user behavior from the obtained embedding vector of actions. Just as
existing research obtains the vector of text by averaging the word embeddings
of each word in the text, we can calculate the feature vector of the user by
averaging the vectors of the actions in the user’s action sequence. However,
the dependency between the time points of the user’s actions is lost in this
case. Therefore we need a method that preserves the order of the vectors to
calculate the user’s feature vector.

## References

  * Attenberg, Pandey, and Suel (2009) Attenberg, J.; Pandey, S.; and Suel, T. 2009.  Modeling and predicting user behavior in sponsored search.  In _KDD_.
  * Barabasi (2005) Barabasi, A.-L. 2005.  The origin of bursts and heavy tails in human dynamics.  _Nature_ , 435(7039): 207–211.
  * Bolukbasi et al. (2016) Bolukbasi, T.; Chang, K.-W.; Zou, J. Y.; Saligrama, V.; and Kalai, A. T. 2016.  Man is to computer programmer as woman is to homemaker? debiasing word embeddings.  In _Advances in neural information processing systems_ , 4349–4357.
  * Caliskan, Bryson, and Narayanan (2017) Caliskan, A.; Bryson, J. J.; and Narayanan, A. 2017.  Semantics derived automatically from language corpora contain human-like biases.  _Science_ , 356(6334): 183–186.
  * Chen et al. (2018) Chen, C.; Kim, S.; Bui, H.; Rossi, R.; Koh, E.; Kveton, B.; and Bunescu, R. 2018\.  Predictive analysis by leveraging temporal user behavior and user embeddings.  In _CIKM_.
  * Di Carlo, Bianchi, and Palmonari (2019) Di Carlo, V.; Bianchi, F.; and Palmonari, M. 2019.  Training temporal word embeddings with a compass.  In _AAAI_.
  * Djuric et al. (2014) Djuric, N.; Radosavljevic, V.; Grbovic, M.; and Bhamidipati, N. 2014.  Hidden conditional random fields with deep user embeddings for ad targeting.  In _ICDM_. IEEE.
  * Du et al. (2016) Du, N.; Dai, H.; Trivedi, R.; Upadhyay, U.; Gomez-Rodriguez, M.; and Song, L. 2016\.  Recurrent marked temporal point processes: Embedding event history to vector.  In _KDD_.
  * Dyer (2014) Dyer, C. 2014.  Notes on noise contrastive estimation and negative sampling.  _arXiv preprint arXiv:1410.8251_.
  * Feng et al. (2019) Feng, W.; Tang, J.; Liu, T. X.; Zhang, S.; and Guan, J. 2019.  Understanding Dropouts in MOOCs.  In _AAAI_.
  * Garg et al. (2018) Garg, N.; Schiebinger, L.; Jurafsky, D.; and Zou, J. 2018.  Word embeddings quantify 100 years of gender and ethnic stereotypes.  _Proceedings of the National Academy of Sciences_ , 115(16): E3635–E3644.
  * Gonen and Goldberg (2019) Gonen, H.; and Goldberg, Y. 2019.  Lipstick on a pig: Debiasing methods cover up systematic gender biases in word embeddings but do not remove them.  _arXiv preprint arXiv:1903.03862_.
  * Graesser, Millis, and Zwaan (1997) Graesser, A. C.; Millis, K. K.; and Zwaan, R. A. 1997.  Discourse comprehension.  _Annual review of psychology_ , 48(1): 163–189.
  * Grbovic et al. (2015) Grbovic, M.; Radosavljevic, V.; Djuric, N.; Bhamidipati, N.; Savla, J.; Bhagwan, V.; and Sharp, D. 2015.  E-commerce in your inbox: Product recommendations at scale.  In _KDD_.
  * Gutmann and Hyvärinen (2010) Gutmann, M.; and Hyvärinen, A. 2010.  Noise-contrastive estimation: A new estimation principle for unnormalized statistical models.  In _AISTAT_.
  * Hamilton, Leskovec, and Jurafsky (2016) Hamilton, W. L.; Leskovec, J.; and Jurafsky, D. 2016.  Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change.  In _ACL_.
  * Han et al. (2020) Han, L.; Checco, A.; Difallah, D.; Demartini, G.; and Sadiq, S. 2020.  Modelling user behavior dynamics with embeddings.  In _CIKM_.
  * Holme and Saramäki (2012) Holme, P.; and Saramäki, J. 2012.  Temporal networks.  _Physics Reports_ , 519(3): 97–125.  Temporal Networks.
  * Kahneman (2011) Kahneman, D. 2011.  _Thinking, fast and slow_.  Macmillan.
  * Levy and Goldberg (2014) Levy, O.; and Goldberg, Y. 2014.  Dependency-Based Word Embeddings.  In _ACL_.
  * Liu et al. (2020) Liu, H.; Zhu, Y.; Zang, T.; Yu, J.; and Cai, H. 2020.  _Jointly Modeling Individual Student Behaviors and Social Influence for Prediction Tasks_.
  * Masuda and Lambiotte (2016) Masuda, N.; and Lambiotte, R. 2016.  _A guide to temporal networks_.  World Scientific.
  * Mikolov et al. (2013a) Mikolov, T.; Chen, K.; Corrado, G.; and Dean, J. 2013a.  Efficient Estimation of Word Representations in Vector Space.  In _ICLR_.
  * Mikolov et al. (2013b) Mikolov, T.; Sutskever, I.; Chen, K.; Corrado, G.; and Dean, J. 2013b.  Distributed Representations of Words and Phrases and Their Compositionality.  In _NIPS_.
  * Okada, Yamanishi, and Masuda (2020) Okada, M.; Yamanishi, K.; and Masuda, N. 2020.  Long-tailed distributions of inter-event times as mixtures of exponential distributions.  _Royal Society open science_ , 7(2): 191643.
  * Oliveira and Barabási (2005) Oliveira, J. G.; and Barabási, A.-L. 2005.  Darwin and Einstein correspondence patterns.  _Nature_ , 437(7063): 1251–1251.
  * Ren et al. (2018) Ren, K.; Fang, Y.; Zhang, W.; Liu, S.; Li, J.; Zhang, Y.; Yu, Y.; and Wang, J. 2018\.  Learning multi-touch conversion attribution with dual-attention mechanisms for online advertising.  In _CIKM_.
  * Sharma et al. (2021) Sharma, K.; Zhang, Y.; Ferrara, E.; and Liu, Y. 2021.  Identifying Coordinated Accounts on Social Media through Hidden Influence and Group Behaviours.  In _KDD_.
  * Stanovich and West (2000) Stanovich, K. E.; and West, R. F. 2000.  Individual differences in reasoning: Implications for the rationality debate?  _Behavioral and Brain Sciences_ , 23(5): 645–665.
  * Vázquez et al. (2006) Vázquez, A.; Oliveira, J. G.; Dezsö, Z.; Goh, K.-I.; Kondor, I.; and Barabási, A.-L. 2006.  Modeling bursts and heavy tails in human dynamics.  _Physical Review E_ , 73(3): 036127.
  * Waller and Anderson (2021) Waller, I.; and Anderson, A. 2021.  Quantifying social organization and political polarization in online platforms.  arXiv:2010.00590.
  * Wang et al. (2016) Wang, G.; Zhang, X.; Tang, S.; Zheng, H.; and Zhao, B. Y. 2016.  Unsupervised clickstream clustering for user behavior analysis.  In _CHI_ , 225–236.
  * Wang et al. (2014) Wang, R.; Chen, F.; Chen, Z.; Li, T.; Harari, G.; Tignor, S.; Zhou, X.; Ben-Zeev, D.; and Campbell, A. T. 2014.  StudentLife: assessing mental health, academic performance and behavioral trends of college students using smartphones.  In _UbiComp_.
  * Wickens and Carswell (2021) Wickens, C. D.; and Carswell, C. M. 2021.  Information processing.  _Handbook of Human Factors and Ergonomics_ , 114–158.
  * Yu et al. (2018) Yu, D.; Li, Y.; Xu, F.; Zhang, P.; and Kostakos, V. 2018.  Smartphone app usage prediction using points of interest.  _IMWUT_.
  * Zhang, Chen, and Wang (2016) Zhang, W.; Chen, L.; and Wang, J. 2016.  Implicit look-alike modelling in display ads.  In _ECIR_.
  * Zhao et al. (2017) Zhao, Z.; Liu, T.; Li, S.; Li, B.; and Du, X. 2017.  Ngram2vec: Learning Improved Word Representations from Ngram Co-occurrence Statistics.  In _EMNLP_.