# Improve Temporal Awareness of LLMs for Sequential Recommendation

Zhendong Chu1, Zichao Wang2, Ruiyi Zhang2, Yangfeng Ji1, Hongning Wang1, Tong
Sun2
1University of Virginia 2Adobe Research
{zc9uy, yangfeng<EMAIL_ADDRESS><EMAIL_ADDRESS>
Work done during an internship at Adobe Research.

###### Abstract

Large language models (LLMs) have demonstrated impressive zero-shot abilities
in solving a wide range of general-purpose tasks. However, it is empirically
found that LLMs fall short in recognizing and utilizing temporal information,
rendering poor performance in tasks that require an understanding of
sequential data, such as _sequential recommendation_. In this paper, we aim to
improve temporal awareness of LLMs by designing a principled prompting
framework inspired by human cognitive processes. Specifically, we propose
three prompting strategies to exploit temporal information within historical
interactions for LLM-based sequential recommendation. Besides, we emulate
_divergent thinking_ by aggregating LLM ranking results derived from these
strategies. Evaluations on MovieLens-1M and Amazon Review datasets indicate
that our proposed method significantly enhances the zero-shot capabilities of
LLMs in sequential recommendation tasks.

Improve Temporal Awareness of LLMs for Sequential Recommendation

Zhendong Chu1††thanks:  Work done during an internship at Adobe Research.,
Zichao Wang2, Ruiyi Zhang2, Yangfeng Ji1, Hongning Wang1, Tong Sun2
1University of Virginia 2Adobe Research {zc9uy, yangfeng<EMAIL_ADDRESS><EMAIL_ADDRESS>

##  1 Introduction

Large language models (LLMs) such as ones with commercially available APIs
including ChatGPT Achiam et al. (2023) and
Claude111https://www.anthropic.com/index/claude-2 have emerged as one of the
primary, if not the de facto, choices in a wide range of applications thanks
to their remarkable capabilities in dealing with natural language and
generalizing to various domains without further fine-tuning. In deed, an
emerging trend is to use natural language as a uniform interface and leverage
the LLMs to complete a task.

Figure 1: LLM-based sequential recommendation baselines show comparable
performance even when historical interactions (Sequential) order is randomized
(Random). Tempura significantly boosts performance by utilizing historical
orders, i.e., temporal information.

Following this trend, recent research has been exploring the use of LLMs for
processing sequential data, with applications such as sequential
recommendation (SRS) Hou et al. (2023b); Bao et al. (2023), which require LLMs
to comprehend temporal patterns within user historical interactions. In the
case of sequential movie recommendation, historical interactions such as
users’ movie watching records can be represented as natural language (i.e.,
movie titles and other meta data) for the LLMs to process and recommend the
next movie, instead of item identifiers which are typically used in
traditional recommender systems Kang and McAuley (2018); Sun et al. (2019).
The extensive generalization ability and vast world knowledge Wang et al.
(2020); Singhal et al. (2023) of LLMs endow them with the potential to serve
as a single model for many recommendation domains without fine-tuning, making
it a general, capable, and easy-to-use alternative to traditional recommender
systems that usually specialize in one selected domain and require extensive
training or fine-tuning.

However, recent research shows that LLMs exhibit a limited sensitivity to
temporal information in the input text, particularly in discerning changes in
user interests Hou et al. (2023b). In Figure 1, we compare the recommendation
performance of LLM-based methods using randomized (denoted as Random) versus
correctly ordered (denoted as Sequential) historical interactions on two
widely-used SRS datasets. Both methods show similar performance, suggesting
that LLMs are not effectively utilizing the temporal information present in
the input text. This limitation stems from a lack of specialized mechanisms
within LLMs to automatically recognize and utilize temporal information, which
is crucial for understanding the context and progression within the data.

In this paper, we focus on improving LLMs’ awareness and interpretation of
temporal information, particularly within the SRS scenario. Temporal
information is ubiquitous in real-world applications, such as recommender
systems McAuley (2022), intelligent document processing Fischer (2001) and
financial market analysis Tsay (2005). By effectively capturing and
integrating this temporal aspect, we have the opportunity to significantly
enhance the understanding of user preferences via LLMs, thus providing users
with better recommendations that suit their backgrounds, needs, and
preferences. This improvement is also important for boosting the effectiveness
of LLMs in downstream applications, where accurate user preference modeling is
crucial McAuley (2022). To this end, we design a principled prompting
framework inspired by human cognitive process, which is training-free and
domain agnostic. We name our approach as Tempura (phonetically similar to
Temporal Prompt). Our main contributions are:

  * •

We propose a principled method to construct in-context examples Min et al.
(2022) for sequential recommendation, by analyzing how Transformer-based SRS
models (e.g., Kang and McAuley (2018)) learn to utilize temporal information.

  * •

Inspired by the results in neuroscience Nobre and Van Ede (2018); Griffiths et
al. (1998), we add explicit structure analysis in input sequences as
additional prompts, particularly temporal cluster analysis, to enhance the
temporal understanding capabilities of LLMs.

  * •

We emulate the process of _divergent thinking_ Runco (1991) by aggregating
ranking results derived from various prompting strategies.

  * •

We evaluate our method on MovieLens-1M and Amazon Review datasets, the results
show that our proposed method significantly enhances the zero-shot
capabilities of LLMs in sequential recommendation tasks.

##  2 Related Works

Figure 2: An illustrative overview of Tempura. We learn sequential
recommendation via two kinds in-context demonstrations. Explicit cluster
structure analysis is conducted to improve the temporal understanding
capabilities of LLMs. Each prompting strategy independently generates a
respective ranking by LLMs (marked by different colors). Rankings from
different prompting strategies are aggregated to form the final ranking.

LLMs for recommendation. Recently, the use of LLMs in recommendation systems
has garnered significant research interest due to their capability to
comprehend and encapsulate a user’s preferences and past interactions through
natural language (Fan et al., 2023; He et al., 2023). Current LLM-based
recommender systems are primarily designed for rating prediction (Kang et al.,
2023; Bao et al., 2023) and sequential recommendation tasks (Wang and Lim,
2023; Hou et al., 2023b; Xu et al., 2024). In both tasks, a user’s previous
interactions with items, along with other optional data like the user profile
or item attributes, are concatenated to formulate a natural language prompt.
This is then fed into an LLM with options for no fine-tuning (Wang and Lim,
2023), full-model fine-tuning (Chen, 2023) or parameter-efficient fine-tuning
(Bao et al., 2023). Liu et al. (2023a) designs a series of prompts to evaluate
ChatGPT’s performance over five recommendation tasks. Wang et al. (2023)
develops a ChatGPT-based agent to improve recommendation ability by using
tools such as SQL and Web search. Contrary to existing works that focus on the
tentative evaluation of LLMs’ ability in recommendation, we focus on improving
the LLM’s inefficacy of utilizing temporal information by designing temporal-
aware prompting strategies.

Sequential recommendation. Sequential recommendation (SRS) Hidasi et al.
(2015); Kang and McAuley (2018) aims to predict the next interacted items
based on historical interaction sequences. Early works follow the Markov
assumption Rendle et al. (2010), by designing various neural network models to
capture user preference within interaction sequences, including Recurrent
Neural Network Hidasi et al. (2015); Li et al. (2017), Convolutional Neural
Network Tang and Wang (2018), Transformer Kang and McAuley (2018); Sun et al.
(2019), Graph Neural Network Chang et al. (2021); Wu et al. (2019). However,
most of these approaches are developed based on item IDs Kang and McAuley
(2018) or attributes Zhang et al. (2019) defined on specific domains, making
it difficult to be generalized to other domains. Recently, Hou et al. (2023a),
Hou et al. (2022) and Li et al. (2023) propose to learn unified item
representations for SRS based on pretrained language models. They follow the
paradigm that pretraining an unified text-based sequence encoder on source
domains and then fine-tune the encoder on the target domain. However, all
aforementioned methods need massive user interaction sequences on a specific
domains and can not be easily transfer to unseen domains. In contrast, we
propose utilizing LLMs to establish a domain-agnostic learning process for
sequential recommendation systems. Our approach is training-free and readily
generalizable to unseen domains using only prompts.

##  3 Methodology

In this section, we introduce Tempura in detail. As shown in Figure 2, Tempura
consists of three major components: 1) a in-context learning module that
learns sequential recommendation tasks from sequences of historical
interactions; 2) a temporal structure analysis module that enhances the
model’s understanding by explicitly integrating cluster structures within the
sequences; 3) a prompt ensemble module that aggregates recommendation results
from various prompting strategies. We begin with the definition of notations
to be used in our technical discussions.

###  3.1 Problem Definition

Given a user’s historical interactions $\mathcal{H}=\\{i_{j}\\}_{j=1}^{n}$,
ordered chronologically up to timestamp $n$, the task of sequential
recommendation involves ranking a set of _candidate_ items
$\mathcal{C}=\\{i_{j}\\}_{j=1}^{m}$ for the subsequent timestamp $n+1$. Items
of higher interest are expected to be ranked at more prominent positions. In
practice, candidate items are typically selected from the entire item set
$\mathcal{I}$, where $m\ll|\mathcal{I}|$, through candidate generation models
Covington et al. (2016). Further, we follow the approach of Hou et al. (2022)
by associating each item $i$ with a descriptive text $t_{i}$, which could be
the item’s name and its attributes or properties.

Different from training-based SRS models, we leverage general-purpose LLMs
(e.g., ChatGPT) to solve the recommendation task in an instruction-following
paradigm Wei et al. (2021). Specifically, for each user, we construct a
history prompt from the user’s historical interactions $\mathcal{H}$, and a
candidate item prompt from the candidate item set $\mathcal{C}$. The
aforementioned prompts are concatenated along with an instruction that
explicitly describes the recommendation task, forming the final prompt for
LLMs. LLMs are anticipated to generate rankings of $\mathcal{C}$, reflecting
user preferences, in accordance with the format specified by the instruction.
A post-hoc text parser is employed to convert the natural language rankings
generated by LLMs into structured ranked lists, which is used to calculate the
ranking metrics Hou et al. (2023b).

###  3.2 Sequential Recommendation via In-Context Learning

Given the vast scale of LLMs, fine-tuning domain-specific models becomes
impractical. Thus, we propose to learn sequential recommendation via in-
context learning, offering a training-free approach that can be easily adapted
across various domains by leveraging the world knowledge and comprehension
capabilities of LLMs Hou et al. (2023b); Harte et al. (2023). To this end, we
first analyze the learning process of training-based SRS models, and then
mapping it onto the principles of constructing effective in-context
demonstrations.

The key distinction between SRS and other recommender systems lies in the SRS
model’s requirement to not only identify a user’s preferences based on
historical user-item interactions but also to track the evolution of the
user’s interests over time. Training-based SRSs depend on learning from large-
scale user-item interaction data via GRUs Hidasi et al. (2015) or Transformers
Sun et al. (2019). We utilize In-Context Learning (ICL) Min et al. (2022) as a
training-free alternative to learn a SRS model. We follow Dai et al. (2022) to
analyze the learning process of training-based SRSs. Given the historical
interaction sequence of an user, a trained Transformer-based SRS, such as
SASRec Kang and McAuley (2018), can be represented as,

$\mathcal{F}_{\text{SASRec}}(\boldsymbol{x}_{n})=(W_{0}+\Delta
W)\boldsymbol{x}_{n}.$ (1)

where $W_{0}$ is the initialized parameter matrix, $\Delta W$ is the update
matrix and $\boldsymbol{x}_{n}$ is the representation of a candidate item. The
output of $\mathcal{F}_{\text{SASRec}}$ is the score of the examined candidate
item. In the back-propagation algorithm, $\Delta W$ is computed by
accumulating the outer products of historic item representations
$\boldsymbol{x}_{i}^{\prime T}$ and the error signals $\boldsymbol{e}_{i}$ of
their corresponding outputs:

$\Delta
W=\sum_{i=1}^{n-1}\boldsymbol{e}_{i}\otimes\boldsymbol{x}_{i}^{\prime},$ (2)

where error signals $\boldsymbol{e}_{i}$ is the prediction error on the
historic item $\boldsymbol{x}_{i}^{\prime}$. Thus, the trained SASRec can also
be rewritten into,

$\displaystyle\mathcal{F}_{\text{SASRec}}(\boldsymbol{x}_{n})$
$\displaystyle=(W_{0}+\Delta W)\boldsymbol{x}_{n}$
$\displaystyle=W_{0}\boldsymbol{x}_{n}+\sum_{i=1}^{n-1}(\boldsymbol{e}_{i}\otimes\boldsymbol{x}_{i}^{\prime})\boldsymbol{x}_{n}$
$\displaystyle=W_{0}\boldsymbol{x}_{n}+\text{LinAtt}(E,X^{\prime},\boldsymbol{x}_{n}),$

where LinAtt($V,K,\textbf{q}$) denotes the linear attention operation, in
which we regard error signals $E$ as values and interacted items $X^{\prime}$
as keys, and the current input $\boldsymbol{x}_{n}$ as the query. The learning
process of the SASRec model can be expained as the model predicting the next
item in a sequence based on preceding items and updating itself based on the
prediction error. The trained SASRec model is designed to update user
preferences as the sequence expands, effectively tracking the evolution of the
user’s interests.

Let $\textbf{q}=W_{Q}\boldsymbol{x}_{n}$ represent the attention query vector
for the input candidate item $\boldsymbol{x}_{n}$. An ICL-based SRS can be
represented as,

$\displaystyle\mathcal{F_{\text{ICL}}}(\textbf{q})=(W_{\text{ZSL}}+\Delta
W_{\text{ICL}})\textbf{q}$

where $W_{\text{ZSL}}=W_{V}X(W_{K}X)^{T}$ is the initialized parameters to be
updated and $W_{\text{ZSL}}\textbf{q}$ is the attention result in zero-shot
learning (ZSL) setting, where no demonstration are given. $X$ denotes the
input representations of query tokens before $\boldsymbol{x}_{n}$, such as the
task description of sequential recommendation. Based on the results of Dai et
al. (2022), the second term can be rewritten into,

$\displaystyle\Delta
W_{\text{ICL}}\textbf{q}=\text{LinAtt}(W_{V}X^{\prime},W_{K}X^{\prime},\textbf{q}),$

where $X^{\prime}$ denotes the input representations of demonstrations. Here
we observe a similar form between $\mathcal{F}_{\text{SASRec}}$ and
$\mathcal{F}_{\text{ICL}}$, where $W_{V}X^{\prime}$ can be explained as the
error signal from historic items. This analogy illustrates that by utilizing
historic items as in-context demonstrations, an LLM can learn to capture the
temporal information within the sequence of historical interactions. Hou et
al. (2023b) discussed using the last item in the history as an in-context
demonstration. Based on our analysis, this method is equivalent to training
the SASRec model solely with the last historical interaction, a practice
insufficient for capturing the dynamic nature of historical interactions.
Thus, we are motivated to use several historical interactions as
demonstrations to improve the temporal awareness of LLMs.

Proximal temporal demonstrations (PCL). Based on the above principle, we
design the following prompt to learn to capture temporal information via ICL,

Placeholders, highlighted in orange, structure the input for our model. The
first placeholder captures the initial $n-k$ historic items, serving as the
_context_ for inferring user preferences. The subsequent placeholder is
designated for the $n-k+1$ item, illustrating the next item to be recommended
based on the current context. Following this, we inform the LLM that the
$n-k+1$ item has been interacted with, indicating that the $n-k+2$ item is the
next recommendation target. This setup is repeated to create $k$-shot
demonstrations. We utilize the most recent $k$ items as demonstrations to
capture the proximal interest of the user. We denote this prompting strategy
as PCL.

Global interest demonstrations. In previous studies Kang and McAuley (2018);
Hou et al. (2023b), the number of historic items was constrained by the
limited input length of models. Thus the whole interaction history is
typically truncated and the most recent items are remained. Empirically, we
also observed that extending the context window has limited impact on
improving performance and may even detract from it. The reason could be: 1)
the prolong context distract LLMs Liu et al. (2023b); 2) too old history has
little impact on the current user interest in the SRS scenario. However,
simply omitting distant historic items risks overlooking users’ long-term
interests. Hence, we randomly sample a subset of historic items from the whole
history sequence to retain user’s global interest. Specifically, we use the
same template as PCL, but the context is filled with randomly sampled historic
items. Similarly, we incorporate the most recent items as in-context examples.
We denote this prompting strategy as GCL.

###  3.3 Temporal Structure Analysis

It has been recognized in the neuroscience area that the human brain is more
sensitive to temporal structures Nobre and Van Ede (2018); Griffiths et al.
(1998) \- “ _Embedded relationships among the attributes of events over
different timescales carry predictions that guide proactive sensory and motor
preparation in the brain_ ”. Only providing item sequences may make it
difficult for LLM to identify and utilize temporal patterns inside the
sequence. Thus, we are motivated to explicitly provide temporal structures to
LLM. Specifically, we conduct cluster analysis on the item sequence according
to two criteria: items that are (1) _temporally proximate_ and (2) _share
similar features_ should be clustered. In practice, we also use LLMs to
complete the cluster tasks and find it can provide reasonable cluster results.
The results are used as additional input to the LLM for ranking.

###  3.4 Prompt Ensemble

The most straightforward way to combine various prompting strategies is to
concatenate them and use the resulted long prompt. However, this approach
risks exceeding the context length limitations of LLMs. Moreover, it has been
observed that LLMs may lose important information within overly lengthy
prompts Liu et al. (2023b). To effectively utilize different prompting
strategies, we propose ensembling the respective ranking outcomes derived from
each. In this approach, we create several LLM sessions and obtain ranking
lists with different prompts. Following Hou et al. (2023b), we explicitly
define the output format for the ranking results produced by LLMs, and
subsequently extract the ranking list using a post-hoc text parser. These
ranking lists are aggregated to obtain the final ranking, as the process shown
in Figure 2. Existing research also highlights the benefits of collaboration
among multiple LLMs Wu et al. (2023). Specifically, we assign scores to each
rank in the ranking list. For instance, in a ranking list of 20 items, the
item in the 1st place receives 20 points, the 2nd place item gets 19 points,
and so on, decreasing by one point per rank. Finally, we sum the scores for
each item across all rankings.

Table 1: Performance comparison on ML-1M and Amazon Review datasets. We
highlight the best performance in bold. N$@K$ denotes NDCG@$K$.

Method | ML-1M | Games | Kindle |
---|---|---|---|---
N@1 | N@5 | N@10 | N@1 | N@5 | N@10 | N@1 | N@5 | N@10 |
BM25 | 4.00 | 13.14 | 20.53 | 16.50 | 30.09 | 37.19 | 6.50 | 18.07 | 24.96 |
UniSRec | 9.00 | 20.08 | 26.72 | 19.50 | 34.86 | 40.82 | 5.00 | 16.21 | 25.03 |
VQ-Rec | 9.50 | 19.52 | 27.11 | 5.50 | 16.76 | 25.27 | 4.30 | 14.22 | 23.58 |
Sequential | 21.43 | 42.57 | 48.59 | 24.12 | 47.26 | 53.03 | 10.20 | 27.96 | 33.72 |
RF | 26.56 | 45.99 | 51.27 | 25.63 | 50.02 | 53.72 | 11.11 | 28.77 | 35.71 |
ICL | 26.40 | 47.51 | 53.32 | 26.00 | 49.68 | 53.63 | 13.07 | 30.82 | 36.41 |
Cluster | 27.00 | 45.82 | 52.04 | 26.15 | 47.41 | 52.39 | 13.20 | 25.77 | 34.07 |
PCL | 29.16 | 48.44 | 54.21 | 29.00 | 51.56 | 55.11 | 11.55 | 29.45 | 36.46 |
GCL | 30.50 | 48.53 | 53.26 | 32.00 | 51.61 | 56.63 | 10.00 | 31.45 | 36.67 |
PCL + Cluster | 30.50 | 48.35 | 54.88 | 35.50 | 53.89 | 58.74 | 12.00 | 30.15 | 38.23 |
Tempura | 31.50 | 48.64 | 54.49 | 39.00 | 56.51 | 60.95 | 14.00 | 32.17 | 37.59 |

##  4 Experiments

In this section, to fully demonstrate the effectiveness of Tempura in
improving temporal awareness of LLMs, we conduct a set of extensive
experiments to study the following research questions: (1) Can Tempura improve
LLM’s performance on sequential recommendation compared to other methods? (2)
Can Tempura enhance the sensitivity of LLMs to temporal information in the
input data? (3) How do factors like history length, the number of in-context
examples or the choice of backbone LLMs influence the effectiveness of
Tempura?

###  4.1 Setup

Datasets. The experiments are conducted on three widely-used public sequential
recommendation datasets: (1) the movie rating dataset MovieLens-1M Harper and
Konstan (2015) (ML-1M) where user rated movies are regarded as interactions,
(2) one category from the Amazon Review dataset Ni et al. (2019) named Games
where reviews are regarded as interactions, and (3) another category from
Amazon Review dataset named Kindle. We sort the interactions of each user by
timestamp, with the oldest interactions first, to construct the corresponding
interaction sequences. The movie or product titles are used as the descriptive
text of an item.

Evaluation configurations. Following existing works Kang and McAuley (2018);
Sun et al. (2019); Hou et al. (2023b), we apply the leave-one-out strategy for
evaluation. For each interaction sequence, the last item is used as the
ground-truth item. We adopt the widely used metric NDCG@N to evaluate the
ranking performance over the given candidate set $\mathcal{C}$ where
$N\leq|\mathcal{C}|$. In the remainder of this paper, unless otherwise
specified, $|\mathcal{C}|$ is set to 20. The candidate set consists of one
ground-truth item and 19 randomly sampled negative items.

Baselines. We consider three prompt-based baselines discussed in Hou et al.
(2023b): Sequential prompting: Arrange the historical interactions in
chronological order. Recency-focused prompting (RF): In addition to the
sequential interaction records, a sentence is additionally added to emphasize
the most recent interaction. In-context learning (ICL): Similar to PCL, but
only use the most recent historic item as the in-context example. We also
consider three methods designed for domain generalization: BM25 Robertson et
al. (2009) ranks items according to the textual similarity between candidates
and historic items. UniSRec Hou et al. (2022) equips textual item
representations with an MoE-enhanced adaptor for domain fusion and adaptation.
VQ-Rec Hou et al. (2023a) learns vector-quantized item representations, which
can map item text into a vector of discrete indices (i.e., item codes) and use
them to retrieve item representations from a code embedding table in
recommendations. Additionally, we report the results with each single
prompting strategy, as well as the results from ensembling PCL and cluster
analysis.

Training-based methods such as Kang and McAuley (2018); Sun et al. (2019) are
not considered as baselines because: (1) They are designed based on item IDs,
which can not be generalized to new domains with new ID spaces. (2) Our
research focuses on improving the temporal awareness of LLMs, as evidenced by
improved performance in sequential recommendations. Thus, our goal is not
necessarily to develop a state-of-the-art sequential recommendation method.

Figure 3: Performance vs. history length $|\mathcal{H}|$ (ML-1M).

Implementation details. Considering economic and efficiency factors, we follow
Hou et al. (2023b); Xu et al. (2024) to randomly sample 200 users along with
their historical interactions for each dataset. Unless specified, we use the
Azure OpenAI API gpt-3.5-turbo222https://azure.microsoft.com/en-
us/pricing/details/cognitive-services/openai-service/. We set history length
$|\mathcal{H}|$ as 15 and use the most recent 5 interactions as demonstrations
in PCL. We found the length of the history significantly affects performance;
therefore, we also searched for the optimal $|\mathcal{H}|$ for baselines.
Empirically, $|\mathcal{H}|=10$ yielded the best results for baselines in
general. All the reported results are the average of three repeat runs to
reduce the effect of randomness.

Table 2: Performance of Tempura with randomized items, clusters and correctly
ordered inputs.

| Item-R | Cluster-R | Correct
---|---|---|---
ML-1M | 51.78 | 52.47 | 54.49
Games | 51.83 | 54.18 | 60.95
Kindle | 34.13 | 33.92 | 37.59

Main results. We present the results on three datasets in Table 1. We can
observe our prompting strategies in the third group improves upon existing
baselines across all metrics. It is interesting to observe that PCL
outperforms ICL significantly, where more demonstrations are used in PCL but
ICL only use the last interaction as demonstration. This observation align
with our analysis that more demonstrations are needed to learn to utilize
temporal information in historical interaction sequences. Although the Cluster
strategy exhibits limited performance on its own, it can significantly enhance
performance when combined with other strategies in an ensemble. Additionally,
we provide a case study of cluster analysis results in Section 4.4. By
comparing individual prompting strategies with two ensemble-based methods, we
find that ensembling consistently enhances performance by leveraging the
strengths of different strategies. This suggests that different strategies
emphasize various aspects, resulting in complementary results.

Figure 4: Impact of #in-context examples in PCL. Several more examples can
improve performance. Table 3: Case study of structure analysis in the
historical interaction sequence. Cluster 1: [Mad Max - PlayStation 4, Metal
Gear Solid V: The Phantom Pain - PlayStation 4].
---
Cluster summary: Action games on PlayStation 4.
Cluster 2: [Star Wars: Battlefront - Standard Edition - PlayStation 4, Fallout
4 - PlayStation 4,
Just Cause 3 - PlayStation 4, Far Cry Primal - PlayStation 4 Standard
Edition].
Cluster summary: Open-world action games on PlayStation 4.
Cluster 3: [Tom Clancyś The Division - PlayStation 4, Uncharted 4: A Thiefś
End - PlayStation 4,
Homefront: The Revolution - PlayStation 4, Deus Ex: Mankind Divided -
PlayStation 4].
Cluster summary: Action games with a focus on story and/or multiplayer on
PlayStation 4.
Cluster 4: [Rise of the Tomb Raider: 20 Year Celebration - PlayStation 4,
Dishonored 2 - PlayStation 4,
Resident Evil 7: Biohazard - PS4 Digital Code, Horizon Zero Dawn - PlayStation
4, Tom Clancy’s
Ghost Recon Wildlands - PlayStation 4].
Cluster summary: Single-player action shooting games with a focus on
exploration and/or stealth on PS4.
Target item: Prey - Pre-load - PS4 Digital Code First-person action-adventure
shooting game

###  4.2 Sensitivity of Temporal Information

In this paper, we aim to improve temporal awareness by designing temporal-
aware prompting strategies. To evaluate whether these proposed strategies
effectively capture and utilize temporal information within historical
interaction sequences, we compare the performance with randomized and
correctly ordered histories. We hypothesize that an approach adept at
utilizing temporal information should demonstrate superior performance with
correctly ordered history. Specifically, our manipulation occurs at two
levels: the item level and the cluster analysis level. At the item level, we
alter the order of individual items, while at the cluster analysis level, we
rearrange the order of clusters derived from cluster analysis. We present the
results in Table 2. After randomizing the history, performance on all the
datasets drop significantly. This phenomenon indicates that understanding and
effectively utilizing temporal information within historical interaction
sequences is crucial for capturing and predicting users’ future interests.

###  4.3 Ablation Study

Impact of history length.  It has been reported in Hou et al. (2023b) that
increasing the number of historical user behaviors does not improve the
ranking performance, but even negatively impacts the ranking performance. To
study the impact of history length on Tempura, we vary the history length
$|\mathcal{H}|$ used for constructing the prompt from 15 to 50. We compare
Tempura with the standard baseline Sequential and the best performing baseline
ICL. Here history length $|\mathcal{H}|$ is the maximum allowed history
length, the real history length could be shorter. We did not include the
results on Games and Kindle since the user interaction history on these two
datasets is short.

The results are reported in Figure 3. We observe that utilizing a longer
history does not improve performance; in fact, it results in decreased
performance on the ML-1M dataset. We hypothesize that the extensive history
distracts LLMs, making it difficult for baselines to understand the evolution
of user interests. By using temporal-aware prompts and the prompt ensemble
strategy, Tempura demonstrates robust performance even with long historical
interaction sequences.

Impact of the number of in-context examples. We utilize a user’s historic
items as in-context demonstrations to understand the temporal information in
his / her behavior sequence. It is important to understand how many examples
are needed. To this end, we study the performance with different number of
examples in PCL. We keep the total length of the user’s history as 15 and use
the latest $k$ items as examples, setting $k$ to values in the set
$[1,3,5,7]$. We report the results on the ML-1M and Games datasets in Figure
4. We can observe more examples can boost the performance significantly than
only one demonstration. As we analyzed in Section 3.2, LLMs learn to utilize
temporal information by learning to predict a series of historical items.
However, it is not always the case that more is better. It is observed that a
slight performance drop with more examples. We speculate that longer prompts
may cause distraction for LLMs.

Results on GPT-4. More advanced LLMs, like GPT-4 Achiam et al. (2023),
demonstrate enhanced capabilities in knowledge, understanding, and reasoning.
Therefore, we evaluate the sequential recommendation performance using GPT-4
to determine if Tempura can also augment GPT-4’s capabilities. We present the
results in Table 4. It has been observed that GPT-4 exhibits a robust capacity
for sequential recommendation, even when employing the most standard prompting
strategy, Sequential. Notably, the improvement is most significant on the
Kindle dataset, leading to the hypothesis that GPT-4 possesses extensive
knowledge about Kindle books. The performance improvement with GPT-4 shows its
strong ability in understanding and utilizing temporal information. By
applying Tempura, the performance can be further improved when the backbone
LLM is more powerful.

Table 4: Performance with GPT-4 (NDCG@10). Tempura can further improve the performance when the backbone LLM is more powerful.  Method | ML-1M | Games | Kindle
---|---|---|---
Sequential | 55.75 | 66.43 | 57.65
ICL | 54.82 | 67.84 | 54.72
Tempura | 58.39 | 68.13 | 58.59

###  4.4 Case Study

We present an example result from the cluster analysis conducted on the Games
dataset. We employ gpt-3.5-turbo to cluster historic items using the prompt
discussed in Section 3.2. The historic items was successfully clustered into 4
clusters, accompanied by a generated summary for each cluster. It can be
easily observed that the user’s most recent interest lies in action shooting
games. With this analysis, the target item can be easily identified since it
is a first-person action-adventure shooting game, aligning with the user’s
latest interest.

##  5 Conclusion

In this paper, we focus on improving the temporal awareness of LLMs through
the study of the sequential recommendation problem. Specifically, we introduce
two kinds of prompting strategies: one to learn sequential recommendations via
in-context learning and another to explicitly analyze the temporal structures
in historical interaction sequences. An ensemble strategy is adopted to
aggregate results from various prompting strategies. Our study demonstrates
that by incorporating specific prompting strategies, LLMs can significantly
improve in capturing and utilizing temporal information. This advancement not
only strengthens the capabilities of LLMs in sequential recommendation tasks
but also opens up new avenues for applying these models in time-sensitive
domains.

## Limitations

Firstly, although LLMs demonstrate notable capabilities in sequential
recommendation, their performance still does not match that of training-based
methods Xu et al. (2024); Hou et al. (2022); Kang and McAuley (2018). The
reason could be that dataset-specific biases, which can be captured by trained
models, are not inherently stored within LLMs. We leave combining prompt-based
methods and training-based methods as an important future work. Secondly,
considering the latency and costs in LLM inference, deploying LLM-based
recommender systems could be cost-inefficient. However, LLMs provide a
powerful and explainable protocol to understand complex human behaviors in
real-world recommender systems, making LLM-based systems a valuable complement
to existing methodologies.

## References

  * Achiam et al. (2023) Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023.  Gpt-4 technical report.  _arXiv preprint arXiv:2303.08774_.
  * Bao et al. (2023) Keqin Bao, Jizhi Zhang, Yang Zhang, Wenjie Wang, Fuli Feng, and Xiangnan He. 2023.  Tallrec: An effective and efficient tuning framework to align large language model with recommendation.  _arXiv preprint arXiv:2305.00447_.
  * Chang et al. (2021) Jianxin Chang, Chen Gao, Yu Zheng, Yiqun Hui, Yanan Niu, Yang Song, Depeng Jin, and Yong Li. 2021.  Sequential recommendation with graph neural networks.  In _Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval_ , pages 378–387.
  * Chen (2023) Zheng Chen. 2023.  Palr: Personalization aware llms for recommendation.  _arXiv preprint arXiv:2305.07622_.
  * Covington et al. (2016) Paul Covington, Jay Adams, and Emre Sargin. 2016.  Deep neural networks for youtube recommendations.  In _Proceedings of the 10th ACM conference on recommender systems_ , pages 191–198.
  * Dai et al. (2022) Damai Dai, Yutao Sun, Li Dong, Yaru Hao, Zhifang Sui, and Furu Wei. 2022.  Why can gpt learn in-context? language models secretly perform gradient descent as meta optimizers.  _arXiv preprint arXiv:2212.10559_.
  * Fan et al. (2023) Wenqi Fan, Zihuai Zhao, Jiatong Li, Yunqing Liu, Xiaowei Mei, Yiqi Wang, Jiliang Tang, and Qing Li. 2023.  Recommender systems in the era of large language models (llms).  _arXiv preprint arXiv:2307.02046_.
  * Fischer (2001) Gerhard Fischer. 2001.  User modeling in human–computer interaction.  _User modeling and user-adapted interaction_ , 11:65–86.
  * Griffiths et al. (1998) Timothy D Griffiths, Christian Büchel, Richard SJ Frackowiak, and Roy D Patterson. 1998.  Analysis of temporal structure in sound by the human brain.  _Nature neuroscience_ , 1(5):422–427.
  * Harper and Konstan (2015) F Maxwell Harper and Joseph A Konstan. 2015.  The movielens datasets: History and context.  _Acm transactions on interactive intelligent systems (tiis)_ , 5(4):1–19.
  * Harte et al. (2023) Jesse Harte, Wouter Zorgdrager, Panos Louridas, Asterios Katsifodimos, Dietmar Jannach, and Marios Fragkoulis. 2023.  Leveraging large language models for sequential recommendation.  In _Proceedings of the 17th ACM Conference on Recommender Systems_ , pages 1096–1102.
  * He et al. (2023) Zhankui He, Zhouhang Xie, Rahul Jha, Harald Steck, Dawen Liang, Yesu Feng, Bodhisattwa Prasad Majumder, Nathan Kallus, and Julian McAuley. 2023.  Large language models as zero-shot conversational recommenders.  _arXiv preprint arXiv:2308.10053_.
  * Hidasi et al. (2015) Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. 2015.  Session-based recommendations with recurrent neural networks.  _arXiv preprint arXiv:1511.06939_.
  * Hou et al. (2023a) Yupeng Hou, Zhankui He, Julian McAuley, and Wayne Xin Zhao. 2023a.  Learning vector-quantized item representation for transferable sequential recommenders.  In _Proceedings of the ACM Web Conference 2023_ , pages 1162–1171.
  * Hou et al. (2022) Yupeng Hou, Shanlei Mu, Wayne Xin Zhao, Yaliang Li, Bolin Ding, and Ji-Rong Wen. 2022.  Towards universal sequence representation learning for recommender systems.  In _Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining_ , pages 585–593.
  * Hou et al. (2023b) Yupeng Hou, Junjie Zhang, Zihan Lin, Hongyu Lu, Ruobing Xie, Julian McAuley, and Wayne Xin Zhao. 2023b.  Large language models are zero-shot rankers for recommender systems.  _arXiv preprint arXiv:2305.08845_.
  * Kang and McAuley (2018) Wang-Cheng Kang and Julian McAuley. 2018.  Self-attentive sequential recommendation.  In _2018 IEEE international conference on data mining (ICDM)_ , pages 197–206. IEEE.
  * Kang et al. (2023) Wang-Cheng Kang, Jianmo Ni, Nikhil Mehta, Maheswaran Sathiamoorthy, Lichan Hong, Ed Chi, and Derek Zhiyuan Cheng. 2023.  Do llms understand user preferences? evaluating llms on user rating prediction.  _arXiv preprint arXiv:2305.06474_.
  * Li et al. (2023) Jiacheng Li, Ming Wang, Jin Li, Jinmiao Fu, Xin Shen, Jingbo Shang, and Julian McAuley. 2023.  Text is all you need: Learning language representations for sequential recommendation.  _arXiv preprint arXiv:2305.13731_.
  * Li et al. (2017) Jing Li, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Tao Lian, and Jun Ma. 2017.  Neural attentive session-based recommendation.  In _Proceedings of the 2017 ACM on Conference on Information and Knowledge Management_ , pages 1419–1428.
  * Liu et al. (2023a) Junling Liu, Chao Liu, Renjie Lv, Kang Zhou, and Yan Zhang. 2023a.  Is chatgpt a good recommender? a preliminary study.  _arXiv preprint arXiv:2304.10149_.
  * Liu et al. (2023b) Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2023b.  Lost in the middle: How language models use long contexts.  _arXiv preprint arXiv:2307.03172_.
  * McAuley (2022) Julian McAuley. 2022.  _Personalized machine learning_.  Cambridge University Press.
  * Min et al. (2022) Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2022.  Rethinking the role of demonstrations: What makes in-context learning work?  _arXiv preprint arXiv:2202.12837_.
  * Ni et al. (2019) Jianmo Ni, Jiacheng Li, and Julian McAuley. 2019.  Justifying recommendations using distantly-labeled reviews and fine-grained aspects.  In _Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP)_ , pages 188–197.
  * Nobre and Van Ede (2018) Anna C Nobre and Freek Van Ede. 2018.  Anticipated moments: temporal structure in attention.  _Nature Reviews Neuroscience_ , 19(1):34–48.
  * Rendle et al. (2010) Steffen Rendle, Christoph Freudenthaler, and Lars Schmidt-Thieme. 2010.  Factorizing personalized markov chains for next-basket recommendation.  In _Proceedings of the 19th international conference on World wide web_ , pages 811–820.
  * Robertson et al. (2009) Stephen Robertson, Hugo Zaragoza, et al. 2009.  The probabilistic relevance framework: Bm25 and beyond.  _Foundations and Trends® in Information Retrieval_ , 3(4):333–389.
  * Runco (1991) Mark A Runco. 1991.  _Divergent thinking_.  Ablex Publishing Corporation Norwood, NJ.
  * Singhal et al. (2023) Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, et al. 2023.  Large language models encode clinical knowledge.  _Nature_ , 620(7972):172–180.
  * Sun et al. (2019) Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. 2019.  Bert4rec: Sequential recommendation with bidirectional encoder representations from transformer.  In _Proceedings of the 28th ACM international conference on information and knowledge management_ , pages 1441–1450.
  * Tang and Wang (2018) Jiaxi Tang and Ke Wang. 2018.  Personalized top-n sequential recommendation via convolutional sequence embedding.  In _Proceedings of the eleventh ACM international conference on web search and data mining_ , pages 565–573.
  * Tsay (2005) Ruey S Tsay. 2005.  _Analysis of financial time series_.  John wiley & sons.
  * Wang et al. (2020) Chenguang Wang, Xiao Liu, and Dawn Song. 2020.  Language models are open knowledge graphs.  _arXiv preprint arXiv:2010.11967_.
  * Wang and Lim (2023) Lei Wang and Ee-Peng Lim. 2023.  Zero-shot next-item recommendation using large pretrained language models.  _arXiv preprint arXiv:2304.03153_.
  * Wang et al. (2023) Yancheng Wang, Ziyan Jiang, Zheng Chen, Fan Yang, Yingxue Zhou, Eunah Cho, Xing Fan, Xiaojiang Huang, Yanbin Lu, and Yingzhen Yang. 2023.  Recmind: Large language model powered agent for recommendation.  _arXiv preprint arXiv:2308.14296_.
  * Wei et al. (2021) Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. 2021.  Finetuned language models are zero-shot learners.  _arXiv preprint arXiv:2109.01652_.
  * Wu et al. (2023) Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Shaokun Zhang, Erkang Zhu, Beibin Li, Li Jiang, Xiaoyun Zhang, and Chi Wang. 2023.  Autogen: Enabling next-gen llm applications via multi-agent conversation framework.  _arXiv preprint arXiv:2308.08155_.
  * Wu et al. (2019) Shu Wu, Yuyuan Tang, Yanqiao Zhu, Liang Wang, Xing Xie, and Tieniu Tan. 2019.  Session-based recommendation with graph neural networks.  In _Proceedings of the AAAI conference on artificial intelligence_ , volume 33, pages 346–353.
  * Xu et al. (2024) Lanling Xu, Junjie Zhang, Bingqian Li, Jinpeng Wang, Mingchen Cai, Wayne Xin Zhao, and Ji-Rong Wen. 2024.  Prompting large language models for recommender systems: A comprehensive framework and empirical analysis.  _arXiv preprint arXiv:2401.04997_.
  * Zhang et al. (2019) Tingting Zhang, Pengpeng Zhao, Yanchi Liu, Victor S Sheng, Jiajie Xu, Deqing Wang, Guanfeng Liu, Xiaofang Zhou, et al. 2019.  Feature-level deeper self-attention network for sequential recommendation.  In _IJCAI_ , pages 4320–4326.


# Detecting Anomalous User Behavior in
Remote Patient Monitoring

Deepti Gupta1, Maanak Gupta2, Smriti Bhatt3, and Ali Saman Tosun4
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>14Dept. of Computer Science, University of Texas at San
Antonio, San Antonio, Texas 78249, USA
2Dept. of Computer Science, Tennessee Technological University, Cookeville,
Tennessee 38505, USA
3Dept. of Computing and Cyber Security, Texas A & M University-San Antonio,
San Antonio, Texas 78224, USA

###### Abstract

The growth in Remote Patient Monitoring (RPM) services using wearable and non-
wearable Internet of Medical Things (IoMT) promises to improve the quality of
diagnosis and facilitate timely treatment for a gamut of medical conditions.
At the same time, the proliferation of IoMT devices increases the potential
for malicious activities that can lead to catastrophic results including theft
of personal information, data breach, and compromised medical devices, putting
human lives at risk. IoMT devices generate tremendous amount of data that
reflect user behavior patterns including both personal and day-to-day social
activities along with daily routine health monitoring. In this context, there
are possibilities of anomalies generated due to various reasons including
unexpected user behavior, faulty sensor, or abnormal values from
malicious/compromised devices. To address this problem, there is an imminent
need to develop a framework for securing the smart health care infrastructure
to identify and mitigate anomalies. In this paper, we present an anomaly
detection model for RPM utilizing IoMT and smart home devices. We propose
Hidden Markov Model (HMM) based anomaly detection that analyzes normal user
behavior in the context of RPM comprising both smart home and smart health
devices, and identifies anomalous user behavior. We design a testbed with
multiple IoMT devices and home sensors to collect data and use the HMM model
to train using network and user behavioral data. Proposed HMM based anomaly
detection model achieved over 98% accuracy in identifying the anomalies in the
context of RPM.

###### Index Terms:

Anomaly Detection, Internet of Medical Things, Remote Patient Monitoring,
Security, Cloud Computing, Hidden Markov Model, Behavioral Data.

##  I Introduction and Motivation

Internet of Medical Things (IoMT), also known as health care Internet of
Things (IoT), is a critical data-driven application utilizing smart connected
devices, relevant in context of COVID-19 like pandemic. This domain represents
a connected infrastructure of medical devices, software applications, health
care systems and services. There is an exponential growth in the number of
IoMT devices and applications with utilization and demand across a diverse
user population. According to the Grand View
Research111https://www.healthcareitnews.com/news/asia-pacific/opportunities-
pitfalls-healthcare-iot, IoMT is predicted to reach $534.3 Billion in market
size by 2025. It is expected that the U.S. home-based healthcare
market222https://store.businessinsider.com/products/the-us-home-healthcare-
report will rise by about 7% annually from $103 billion in 2018 to $173
billion by 2026. With recent technological advancements, IoMT has the
potential to revolutionize the future of health care industry. IoMT aims to
escalate progress of the health care industry and enable people to receive
timely care, enhance their treatment plans, manage medications, and lower
health care costs.

In the last few years, data-driven IoMT applications using Machine Learning
(ML) and AI technologies are extensively used for critical functions, such as
drug discovery, Remote Patient Monitoring (RPM), predictive analytic for
hospital resource optimization, interactive medicine delivery, etc., by
providing the right information at the right time in the health care system.
Today, RPM has became a propitious and required need for users and health
practitioners where patients can be monitored remotely, such as home
isolation/quarantine. This RPM ecosystem consists of smart medical devices,
such as thermometer, oximeter, and wearable devices. These smart health
devices collect a large amount of user health data. However, there are several
issues associated with smart health devices and applications, such as security
of devices and applications, privacy of user data which is shared between
devices and applications among many. A recent report by the US National
Institute of Standards and Technology (NIST) [1] highlights the need on
securing RPM ecosystem and discusses how health care delivery organizations
can implement security and privacy controls to build a secure infrastructure
for RPM.

In this paper, we focus on security of RPM including user safety and identify
anomalous behavior in RPM applications based on the data collected from smart
devices while remotely monitoring the patient at home. Research has developed
various methods to secure RPM infrastructure from anomalies, focusing on
wireless sensor networks [2, 3, 4], patient’s behavior monitoring [5, 6, 7, 8,
9], signature and correlation analysis [10, 11]. While there are several
anomaly detection models developed for RPM, elderly care, and smart homes, our
anomaly detection model is designed to identify anomalies in a unique RPM
ecosystem which comprises the intersection of two IoT domains (smart health
care and smart home). Our model aims to solve the problem of anomalous device
data and user behavior which may occur due to malicious or faulty IoT devices
or critical conditions of a user, for instance, when a user loses
consciousness due to a heart attack and needs immediate medical attention.
Proposed anomaly detection can ensure resilient RPM, and save a patient’s
life.

A general RPM environment includes only medical devices, which makes it
challenging to identify and differentiate normal and abnormal data with user
behavior. For instance, a smart oximeter reports the oxygen level of a
diabetic patient as 70%, which is very lower than normal value, thus, it will
identify this as an anomaly. However, it could be a false positive due to
faulty oximeter. In order to accurately identify anomalies in RPM, both data
from medical devices and user activity behavior need to be monitored from home
sensors in a smart home. In this work we design a novel RPM ecosystem with
both medical and smart home devices that can monitor user health and daily
activities within home, and detect anomalies accurately based on the
correlation among user behavior and health readings. Extending the above
scenario before labeling the oximeter reading as an anomaly, we need to
consider these questions: a) Is the user normally moving at home? b) Is the
user able to do normal tasks and respond to any alerts or suggestions on smart
devices (e.g., smart watch)? and c) Is the user even at home when the oximeter
reported 70% oxygen level for the user? Smart home devices capture user’s
activities, for instance, after reported low oxygen level of the diabetic
patient, if motion sensors at home detect no movement of the patient for
certain time, then it will identify it accurately as an anomaly and will send
an alert to health practitioner for remedial measures.

In this paper, we develop an anomaly detection model using Hidden Markov Model
(HMM) [12] approach to identify anomalies in the RPM environment that
comprises of smart home and medical devices. We develop the RPM use case using
our hybrid approach, and deploy it in a real-world scenario to train and test
the model. Using our proposed model, health practitioners can identify
anomalies in RPM and trigger appropriate actions as needed in a particular
situation. Our model simulation results show that our proposed model detects
anomalies in RPM with high accuracy over 98% and generate low false positive
rate.

The main contributions of this paper are as follows.

  * •

We develop a novel use case of Remote Patient Monitoring (RPM) based on the
intersection of two major IoT domains - smart health care and smart home.

  * •

We design a testbed using sensors and smart health devices to implement the
proposed RPM use case.

  * •

We aggregate both network and user behavioral data collected from smart
devices and convert it into temporal sequences for our anomaly detection
model.

  * •

We design an anomaly detection model using Hidden Markov Model (HMM) and test
it based on aggregated training data that represents patient’s normal
behavior.

  * •

We evaluate our proposed model on testing data along with sets of generated
anomalies which achieved over 98% accuracy to identify anomalies.

Figure 1: A Use Case of Remote Patient Monitoring (RPM)

The remainder of this paper is organized as follows. Section II presents the
literature review on anomaly detection models and machine learning approaches
such as HMM. Section III discusses the RPM use case and illustrate steps to
develop the RPM model architecture. Section IV discusses network and
behavioral data collection. Section V discusses anomalous behavior detection
model using HMM approach, and deployment for RPM use case. Section VI presents
the implementation details and results, followed by conclusion in Section VII.

##  II Related Work

We present related work in anomaly detection models based on network and
behavioral data.

###  II-A Anomaly Detection Models for RPM

Extensive research has been done to keep RPM systems secure. Yamauchi et al.
[13] introduced a method to detect attacks due to home appliances based on
user behavior along with conditions at smart home and also presented a
comparative study with HMM. Zhang et al. [14] proposed a medical security
monitor (MedMon) that snoops on all the radio-frequency wireless
communications to/from medical implantable devices to identify anomalies.
Tonchev et al. [15] presented a non-intrusive sleep analyzer for real time
detection of sleep anomalies based on HMM and Viterbi algorithm. The authors
used non-intrusive sensors such as bed pressure sensor, microphone, and
accelerometer. Pham et al. [16] used spectral coherence analysis for
accelerometer data, which is collected from wearable devices to develop an
anomaly detection model. Deep et al. [7] presented a survey on anomalous
behavior detection for elderly case using dense-sensing networks and concluded
that sensor fusion techniques could increase the efficiency of dense sensing
network. Another study [17] showed that anomaly detection model for Ambient
Assisted Living (AAL) focused on user’s activities, such as sit down for
dinner, open stoves etc., not related to user’s body parameters. A study [18]
presented an approach to reduce the false negative rate (FNR), i.e. disease
should not get undetected. In order to protect the devices, security models
are discussed in several research [19, 20, 21, 22, 23].

###  II-B Hidden Markov Models

HMM models have been extensively used in various smart connected domains. HMM
model-based behavior analysis system for assisted living is discussed in [24],
where sensors are located in a smart home, each room with at least one motion
detector, one temperature sensor, one light level detector and two lighting
sensors status. In [25], a general framework is proposed for securing medical
devices based on wireless channel monitoring and anomaly detection model using
Hierarchical Hidden Markov Model (HHMM). Kotevska et al. [26] proposed
coordinated intelligence approach to classify normal and abnormal behavior of
resident using Markov chain at home where geographical co-located multiple
sensors communicate with each other and take decisions based on the collective
co-location in some capacity. Li et al. [27] constructed HMM-based anomaly
detectors and presented a comparative analysis based on several transformation
methods. Research [28] introduced an anomaly detection model based on user’s
behavior using HMM in a smart home environment. Narayanan et al. [29] proposed
OBD SecureAlert model for vehicles, which detects malicious behaviors and
sends alerts while a vehicle is in unsafe state. Ren et al. [30] introduced
anomaly detection approach based on a dynamic Markov model where sequence data
is managed by sliding window. Zhu [31] used HMM approach for detecting
anomalies in a person’s blood glucose levels using historical observations as
a benchmark.

Extensive research has been done for anomaly detection in RPM and use of HMM
in other smart domains for anomaly detection. However, as discussed earlier,
an hybrid approach which uses both smart home and IoMT is still missing to
detect anomalies more accurately (as suggested by our results). In addition,
to our understanding and literature review HMM model has not been used in RPM.
To bridge this gap, we believe our research offers a novel perspective to
detect anomalies in RPM using HMM.

##  III Remote Patient Monitoring (RPM)

In recent years, RPM has received significant attention due to its capability
of providing health care services to patients while having the convenience of
staying at home. Smart home-based health care and patient monitoring can
enhance the quality of service and reduce the cost of health care.

###  III-A RPM Use case

In this section, we present a RPM use case that is aligned with a real-world
scenario. We consider a user Bob who is 34 years old and lives alone at his
home, and has been diagnosed with Obstructive Sleep Apnea (OSA) disease. He is
being monitored by his health practitioners continuously utilizing the RPM
ecosystem enabled by smart devices deployed at his home. This RPM setup
consists of set of devices including Ethernet tag manager, two wireless sensor
tags, and various Bluetooth Low Energy (BLE) based smart health devices such
as iHealth smart oximeter, iHealth smart scale. These specific smart health
devices along with smart home devices are suggested by his health
practitioners. Fridge door sensor and smart scale are used to monitor Bob’s
diet, eating behavior, and weight. Another motion sensor is attached on
bedroom door to track Bob’s activity, specially in night as OSA patient has
disturbed sleep and may wake up multiple times in night. Smart oximeter keeps
track of his oxygen level at regular intervals. In case Bob is not able to
sleep due to shortage of breathe, a notification is sent to the health
practitioners based on data from bedroom door sensor and oxygen level through
home automation cloud service and unified patient care cloud respectively.
Lets us assume that the normal range of Bob’s oxygen level is SpO2 (88% -
99%). If Bob’s oxygen level reaches less than 80%, then the health
practitioner is notified who can also send connected ambulance to Bob’s home.
Figure 1 shows a sequential view of the RPM use case scenario where Bob’s
health condition is continuously monitored by his health practitioners. Bob
checks his oxygen level and weight regularly in specific time intervals. These
BLE based health devices communicate with a gateway device (Bob’s smart phone)
at object abstraction layer of Enhanced ACO architecture (EACO) [32] to allow
interaction with the cloud service and applications. The generated data from
these devices is transmitted through the gateway to their corresponding
manufacturer’s cloud platforms (e.g., iHealth cloud and Wireless sensor cloud
respectively) and is stored in remote cloud. Users rely on the security
mechanisms deployed by smart device manufacturers to ensure collected data
privacy.

In general, anomalous behavior is defined as any violation or deviation from
the normal behavior. We defined the anomalies that could occur in the RPM use
case if a user deviates from his/her behavior or reflects any abnormal
behavior. The following use case presents examples of abnormal behavior and
possible threat scenarios.

  * •

In RPM, the devices are used in a particular order based on user’s daily
routine. Any deviation could be an anomaly; for example, after checking oxygen
level, a user opens the bedroom door within time window, if not that means
user needs emergency assistance.

  * •

When user is not at home, but smart scale or oximeter are still sending data
(vital readings). This could be due to a faulty smart device or possible
intrusion at home.

  * •

An attacker can compromise an IoT device(s), for example, a user is measuring
his oxygen level while other sensors like smart scale and fridge door are also
activated at same time.

  * •

Bedroom door sensor is activated multiple times in a short interval of time
during night and the oxygen level of user is low, i.e. user is not able to
sleep during night. The condition can aggravate to a critical level and a
health practitioner may need to intervene and check the patients health by
sending messages to confirm his well-being.

###  III-B System Architecture

Our RPM model architecture is categorized into 3 phases:

  * •

Data Collection Phase: This is the first step in which the stream of data is
collected from sensors and smart health devices to develop the anomaly
detection model.

  * •

Model Generation Phase: We analyze the collected data and convert into
sequences to feed the HMM model.

  * •

Anomaly Detection Phase: In this final phase, anomalies are detected
automatically using our proposed HMM-based anomaly detection model.

In these phases, we discuss and define normal behavior of a user and devices
associated with that user as patterns in sequences of observations. For
tracking the user behavior and training our anomaly detection model, opening
the bedroom door, opening the fridge door, measuring weight and oxygen level
are the activities of our interest. Each activity and its relevant data is
measured using corresponding smart devices in RPM ecosystem. These phases of
RPM model architecture are described in the following sections.

##  IV Data Collection

In RPM ecosystem, the smart connected devices generate tremendous amount of
data while monitoring the patient at home. To collect this diverse data set,
we set up the RPM testbed at a home by deploying IoT sensors and smart health
devices (architecture shown in Figure 2). We consider that one patient lives
in the smart home and performs his daily tasks, and the testbed captures data
for three weeks. Data is collected from different sensors and smart health
devices, which are used to build anomaly detection model.

We collect data in two different categories - (i) Network data, and (ii)
Behavior data. We integrate both types of data to learn typical behaviors of a
patient who uses smart medical devices and his activities are captured from
home sensors. Network data refers to the TCP/UDP packets from IoT devices and
smart phone. Behavioral data is collected based on the sensors’ readings
showing door status (open/closed) and health devices readings (e.g., 96%
SpO2). This behavioral data is fetched using API from different device
manufacturer cloud data repositories (e.g., iHealth cloud, mytaglist cloud).

Figure 2: Testbed Deployed for Remote Patient Monitoring Figure 3: Snapshot of
Network Data Captured using Wireshark

###  IV-A Network Data Collection

The varieties of data packet including audio and video flow through the
gateway router of a smart home. However, we only require RPM related data from
motion sensors and smart health devices. This required data cannot be
separated from other non essential data inside the network, since the internal
devices IPs are masked during Network Address Translation (NAT) in gateway
router. Our setup consists of ARRIS SB6121 modem, two TP-Link AC1200 wireless
router, NETGEAR ProSAFE Plus GS105Ev2 switch, and a data collection machine.
The data collection machine is connected to the destination port of the switch
and all data coming from the router will be available in it, as shown in
Figure 2. The packets are collected at the switch using
Wireshark333https://www.wireshark.org/ packet sniffer. We use the switch as a
bridge between the gateway and internal router, and enable port mirroring in
the switch. Port mirroring is a method of monitoring network traffic by
setting up one or more source ports to send a copy of every packet received to
a designated destination port. It is important to assign static IP to all
devices in the testbed since IP address filtering is applicable if devices
have a static IP.

As shown in Figure 2, BLE based health devices are connected to patient’s
phone-2 to communicate its cloud service. To avoid other traffic, phone-2 is
used only for data transfer of health devices and has set a static IP during
three weeks of data collection. The motion sensors communicate to their cloud
through Ethernet tag manager, and these wireless motion sensors use
“narrowband FM” technology444Long-range RF communication: Why narrowband is
the de facto standard to communicate to Ethernet tag manager. The static IP
address of this tag manager is fixed via MAC address reservation. Bob’s
phone-1 is used to keep track his presence at his home, where phone-1 connects
to the gateway router. For instance, whenever user enters his home, phone-1 is
connected Wi-Fi automatically. We filter the required network traffic of
phone-1 to know Bob’s presence at home, which is shown in Figure 3. This
figure presents that phone-1 is connected to the gateway router, where mac
address of gateway router is 50:d4:f7:1f:e5:c4 as source address and mac
address of phone-1 is 20:ab:37:db:9a:88 as destination address. In this
research, we define the status of phone-1 as ‘in’ and ‘out’ based on Bob’s
presence.

Figure 4: Snapshot of iHealth Cloud Data

###  IV-B Behavioral Data Collection

The behavioral data represents the status of bedroom door, fridge door, and
vital sign readings of smart health devices. In home, patient’s movement is
captured through sensors, for instance, if a motion sensor is attached to
fridge door which identifies if a fridge door is open or close, i.e. patient
is hungry or not. Another motion sensor is attached on bedroom door which
identifies about sleeping time of patient. These motion sensors are attached
on the doors with 30∘ angle to identify door is open or not. The home sensors
are connected to Ethernet based smart tag manager, and this tag manager pushes
the data into home automation’s cloud. To differentiate between both sensors,
the status of bedroom door are defined as ‘bd_open’ and ‘bd_close’, and the
status of fridge door are defined as ‘fd_open’ and ‘fd_close’. The status of
the sensors and vital sign readings of health devices are retrieved from their
corresponding clouds using API calls. Figure 4 shows body parameter readings
for Bob and from this data we derived the observation status for oximeter and
smart scale. The smart health devices generate numeric reading, however, the
devices’ numeric readings cannot be incorporated into logical calculations. We
use normal distribution to discretize continuous data by using different
thresholds, and calculate its mean $\mu$ and standard deviation $\sigma$ to
define the range. For oximeter, three ranges (ox1, ox2, and ox3) are defined,
where ‘ox1’ comes under range of positive second deviation $\mu+2\sigma$,
which is 97% to 99.3%, and ‘ox2’ comes under range of negative second
deviation $\mu-2\sigma$, which is 96.9% to 94.7%, if readings fall outside the
range [$\mu-2\sigma$, $\mu+2\sigma$], is considered as ‘ox3’. The fourth
status of oximeter is ‘ox_off’. We also define the range of smart scale in
similar way. Patient’s phone-2 has two status ‘on’ and ‘off’, and is connected
to internal router and is only used to connect Bluetooth devices to send the
data (vital readings) to iHealth cloud. Table I lists status of sensors and
devices, which are considered as observations in HMM described in section V).

In our experiment, each sensor (door, fridge) sends its status every 30
seconds, while the status of IoMT devices (oximeter, scale) is measured by
user at regular intervals. The user’s presence is captured through his phone-1
through network traffic. To analyze, both network and behavioral data are
integrated based on time-stamp, as shown in Table II.

TABLE I: List of Observations For Devices Sensors/Devices | Observations
---|---
Bedroom door | bd_open, bd_close
Fridge door | fd_open, fd_close
Weight scale | sc_off, sc1, sc2, sc3
Oximeter | ox_off, ox1, ox2, ox3
Phone-2 | on, off
Phone-1 | in, out
TABLE II: Sample log collected from the sensors in RPM Time | Sensors | Status | Phone-2
---|---|---|---
8:02 | Phone-1 | in |
8:04 | Oximeter | ox1 | on
8:06 | Bedroom door | bd_open |
8:07 | Bedroom door | bd_close |
8:24 | Scale | sc2 | on
8:32 | Fridge door | fd_open |
8:33 | Fridge door | fd_close |

##  V Anomalous Behavior Detection Model

In this section we propose an anomaly detection model for RPM ecosystem. We
define the anomalies as abnormal activities performed by the user and abnormal
values triggered from IoMT devices. This proposed model is based on HMM and
considers the user behavior as specific sequences of data observations to feed
the HMM model. We first present the HMM model, which learns the user’s normal
behavior and later define the proposed anomaly detection model for RPM use
case scenario (discussed in Section III) .

###  V-A HMM for Anomaly Detection in RPM

We first collect diverse data from various IoMT devices and home sensors.
Later, we analyze the collected data and convert into temporal sequences to
feed HMM model. The HMM is an augmentation of Markov chain model, which is
able to detect the sequential relations among hidden states. This is a
probabilistic model where a sequence of observations is generated by visible
observations of internal hidden states. These hidden states are not observed
directly or are not visible. The transitions between hidden states are assumed
to have the form of a (first-order) Markov chain.

The specific parameters used to define the HMM are: $N$, $M$, $A$, $B$, and
$\pi$, which are also mapped to the RPM use case. $N$ refers to the total
number of all possible states in this model. At Bob’s home, we consider his
movement as a sequence of states, where sensors are activated at each state.
Here, we denote an individual state from the set of states $S$ =
${S_{1},S_{2},S_{3},....S_{N}}$, at time t as $q\textsubscript{t}$. $M$ is the
number of unique visible observations for different sensors in a state, $V$ =
${v_{1},v_{2},v_{3},....v_{M}}$ is the set of all possible observations. In
our experiment, observations are simply status of different sensors, as shown
in Table I. Parameter $A$ represents the state transition matrix. It gives the
probability of being able to move from one state i to another state j, as
described in equation 1. If a user does not move to any other state i.e. the
transition probability would be zero. Parameter $B$ represents a probability
distribution of seeing one of the observable status, given that user is in a
particular state and is defined in equation 2. Finally, $\pi$ is the
probability of starting at a particular state, which is randomly chosen.

$\scriptsize\begin{split}&A=[a\textsubscript{i,j}]\\\
&a\textsubscript{i,j}=P(q\textsubscript{t+1}=S\textsubscript{j}|q\textsubscript{t}=S\textsubscript{i})\\\
&where~{}q\textsubscript{t}\text{ is the state at time t,}\\\
&S\textsubscript{i},S\textsubscript{j}\in S(\text{set of all possible
states})\\\ &1\leq i\\\ &j\leq N,t=1,2,3,.......\end{split}$ (1)

$\scriptsize\begin{split}&B=[b\textsubscript{i,j}]\\\
&b\textsubscript{i,j}=P(v\textsubscript{k}\text{ at
t}|q\textsubscript{t}=S\textsubscript{i})\\\ &where~{}q\textsubscript{t}\text{
is the state at time t,}\\\ &v\textsubscript{k}\in V(\text{set of all
observations})\\\ &S\textsubscript{i},S\textsubscript{j}\in S(\text{set of all
possible states})\\\ &1\leq i,j\leq N,t=1,2,3,.......\end{split}$ (2)

###  V-B Learning End User Behavior

In RPM environment, Bob uses medical IoT devices and his activities are
captured through motion sensors. Figure 5 shows the timeline of Bob’s movement
at his home, as he moves from one state to another state, where multiple
sensors and devices can be activated/deactivated at same time. At T1, oximeter
is activated (observation ‘ox2’) along with phone-2 (‘on’), and at T2, the
status of oximeter is changed from ‘ox2’ to ‘ox_off’, and the status of
phone-2 from ‘on’ to ‘off’. Bedroom door is opened at T3, and close at T4.
Then, bedroom door is open (bd_open) again at T5, followed by multiple sensors
(oximeter, bedroom door, scale, phone-2) activating at T6. At T6, oxygen level
is very low, bedroom door is also open, and smart scale is also activated.
According to our model, the three possible sequences of observations will be
generated from T5 to T6, e.g., [bd_open, ox3], [bd_open, sc2], and [bd_open,
off]. At T7, user is at home, phone-2 is still activated, while other sensors,
devices are deactivated. At hidden state T8, status of phone is ‘off’, user is
out.

There are three issues in HMM which must be resolved using the following
approaches. In one approach, given the sequence of observations O = O1 O2…… OT
and a model $\lambda$ = ($A$, $B$, $\pi$) is used to evaluate $P(O|\lambda)$
probability of any sequence of observations using Forward-Backward algorithm
[12]. Next approach is used to adjust the model parameters $\lambda$ = ($A$,
$B$, $\pi$) to maximize $P(O|\lambda)$ using Baum-Welch algorithm [12]. In our
experiment, we collect time-series data from all the devices and sensors, and
convert them into temporal sequences, which will be the training sequences of
the model. The training sequences O = O1, O2,… ,OK are extracted from this
time-series data, where OK is an observation at each state. In our experiment,
we assume that OK is vector as multiple sensors are activated at same time
(shown in Figure 5), where OK = $v_{1}^{(k)}$ $v_{2}^{(k)}$……….$v_{t}^{(k)}$,
1$\leq$ k $\leq$ K. Observation vector (OK) implies to generate different
combinations of sequences. For example, sequence of observations extracted
during state transitions from time T1 to T5 using Figure 5 is
[{ox2,on},{ox_off,off}, bd_open, bd_close, bd_open]. In our experiment, we
extracted 420 sequences of observations from temporal data, and trained the
model with these sequences using Baum-Welch algorithm to determine the HMM
model parameters $\lambda$ = ($A$, $B$, $\pi$). The training process repeats
until the resulting probabilities converge satisfactorily. For testing,
Forward-Backward algorithm is used to evaluate the new sequence of
observations by determining the likelihood P(O$|$$\lambda$), which is the log
probability of new sequence of observations. The high value of log
probabilities indicates that new sequence of observations fit for the model,
and low value of log probabilities implies deviation from normal behavior, and
identify as anomaly.

Figure 5: Sample of Remote Patient Monitoring Events Timeline

###  V-C Anomaly Detection Model

After generating transition, emission and initial probabilities matrices using
Baum-Welch algorithm, we use this generated HMM model to detect anomalies in
RPM environment. In this research, we are detecting both attack states and
unsafe/anomalous states. Here is example of faulty sensors, when user is away
from his home, and smart scale is still sending readings to patient’s care
cloud. To detect the anomalies, we generated a sliding window of hidden states
as shown in Figure 6, where observation vector OK is visible at each state. To
detect anomalies in real-time, a sliding window of “m” observations, Owindow =
O1,O2,….Om is introduced. The sliding window moves every time a new
observation vector is available at a new state and calculate the log
probability of new generated sequences of observations using Forward-Backward
algorithm. As we define that each observation would be a vector of different
sensor values, thus, it will generate a set of log probabilities corresponding
to each sequence of observations. If any log probability in this set is below
a threshold (discussed in Section VI), it identifies it as an anomalous state.
Any sequence of observations is considered anomalous, if it is not accepted by
the proposed model with low probability.

Figure 6: Sliding window to detect anomalies using Forward-Backward Algorithm

##  VI Evaluation and Results

In this section, we present the evaluation of our proposed model and also
discuss the results. We generated 420 sequences of observations from collected
data during the period of three weeks. This data represents Bob’s normal
behavior in RPM environment. We divide collected data into training and
testing datasets, and calculate the threshold value based on the lowest value
of log probability of training sequences in various settings. The threshold
value is used to identify the anomalies in RPM ecosystem. Further, we generate
sets of anomalous scenarios to validate the proposed model. The details of the
experiments are presented in the following subsections.

###  VI-A Analysis of Normal Dataset

For training, we use Baum-Welch algorithm and use Forward-Backward algorithm
for testing. It is required to generate a threshold value to test any sequence
using Forward-Backward algorithm. To validate the threshold value, we divide
the experimental data into three different settings and perform these
experiments. For our first experiment, 70% of data is used for training and
30% is used for testing. We train the model using 70% sequences of
observations and calculate the log probabilities of training sequences, which
are in the range of -5 to -14. For second experiment, we train the HMM model
with the first 60% of data and observed that the log probability of training
sequences ranges from -6 to -14. In third experiment, we divide 420 sequences
into two parts, each set has 210 sequences, where 70% sequences are used for
both training data and provide log probability ranges (-5 to -14, -6 to -13).
Hence, we set the threshold value to be -14, i.e. the log probability value of
any sequence lower than -14 is anomaly. The testing sequences of above defined
three experiments are tested on our proposed model using their corresponding
training datasets, and this model provide different sets of log probability
ranges (-5 to -14.9, -7 to -14, -6.5 to -13.4, -6 to -14). The range of log
probabilities of testing data of first experiment is shown in Figure 7, which
also shows two false positives.

Figure 7: Test data for Remote Patient Monitoring Figure 8: Generated
anomalies injected into test normal data

###  VI-B Abnormal Scenario Detection

To evaluate the model’s performance, we manually generate a set of eight
anomalous scenarios which are representative of real-world scenarios. We
consider abnormal behaviors including unusual device reading, missing
dependency order between devices, and compromised sensors/devices to select
eight anomalous scenarios. First anomalous scenario is User is out from his
home, smart scale is still sending reading to the unified care’s cloud. It may
happen due to faulty/compromised device, which can effect Bob’s diet. To
simulate this scenario, we take a normal sequence of observation, activate the
smart scale and modify the status of phone-1 from ‘in’ to ‘out’. Another
crafted scenario is SpO2 is very low (i.e. ox3) and bedroom door is not
activated., i.e. oxygen level of user is low, and he is not able to get up to
open the bedroom door within a time frame. Third anomalous scenario is when
user is out, scale, oximeter and fridge door sensor activate simultaneously,
that indicates the presence of another user at home or any malicious activity
can be possible. Fourth and fifth anomalous scenarios are when user is at
home, fridge door, oximeter and scale activate simultaneously, and oximeter
reading is reported as ox2, but ox_off is not followed by ox2 within a time
frame respectively. Sixth anomalous scenario is when user is at home, sc3 is
followed by ox3, i.e. only medical devices are activated with unusual vital
readings and seventh scenario is when user is out of his home, bd_open and
fd_open are activated. The last anomalous scenario is bd_open followed by
bd_open along with ox3, i.e. due to low level of oxygen, user is not able to
sleep. These eight generated anomalous scenarios are injected into normal
testing data of first experiment and tested on proposed anomaly detection
model. The threshold values of log probabilities of these eight anomalous
scenarios are lower than -14, which is shown in Figure 8 along with two false
positives. Hence, the proposed model detects all generated anomalies in RPM
environment.

In the next experiment, we generate another set of anomalous scenarios by
fixing or randomly choosing status of specific sensors/devices. For example,
we change the ‘ox2’ to ‘ox3’, ‘on’ to ‘off’, and ‘in’ to ‘out’. In total, we
generated 38 anomalous scenarios randomly, and added eight generated anomalous
scenarios to create a dataset of 46 anomalous scenarios. This dataset is
tested on our trained model, which detects 45 anomalies out of 46. Moreover,
this model can work with any size of dataset. The confusion matrix of the
first experiment (30% of 420) along with the 46 anomalous scenario dataset is
given in Table III, which shows that our proposed HMM based approach provides
98% accuracy.

TABLE III: Confusion Matrix N=172 | Actual:Yes | Actual:Yes | Total
---|---|---|---
| Predicted:Yes | $TP=45$ | $FP=2$ | $47$
| Predicted:No | $FN=1$ | $TN=124$ | $125$
| Total | $46$ | $126$ |

##  VII Conclusion

In this paper, we propose an anomaly detection model for the RPM ecosystem
consisting of smart home and medical IoT devices. We design a testbed of RPM,
collect time-series data, and convert data into sequences of observations
which are used to train the HMM model using Baum-Welch algorithm. This
proposed model detects anomalies based on a single device, and also identifies
anomalies based on the combination of data collected from both IoMT and home-
based devices. Our model evaluation shows that the proposed HMM based approach
can detect anomalies in RPM real-world events/scenarios and randomly generated
anomalies with 98% accuracy. This model can be adopted for a diverse range of
anomalous use cases. For future, we plan to develop a robust anomaly detection
model for multi-users behavior with geographically co-located sensors in home-
based health care applications with more anomalous events.

## Acknowledgement

This research is partially supported by the NSF Grant 2025682 at Tennessee
Technological University, and the Chancellor Research Initiative (CRI) grant
at Texas A&M University-San Antonio. The authors would like to thank Dr. Sudip
Mittal for his suggestions.

## References

  * [1] J. Cawthra _et al._ , “Securing telehealth remote patient monitoring ecosystem,” _NIST SPECIAL PUBLICATION_ , p. 30B, 1800.
  * [2] O. Salem _et al._ , “Anomaly detection scheme for medical wireless sensor networks,” in _Handbook of medical and healthcare technologies_. Springer, 2013, pp. 207–222.
  * [3] V. Vippalapalli _et al._ , “Internet of things (IoT) based smart health care system,” in _2016 IEEE International Conference on Signal Processing, Communication, Power and Embedded System_ , pp. 1229–1233.
  * [4] X. Luo _et al._ , “Design and implementation of a distributed fall detection system based on wireless sensor networks,” _EURASIP Journal on Wireless Communications and Networking_ , vol. 2012, no. 1, p. 118.
  * [5] L. Rachakonda _et al._ , “iLog: an intelligent device for automatic food intake monitoring and stress detection in the IoMT,” _IEEE Transactions on Consumer Electronics_ , vol. 66, no. 2, pp. 115–124, 2020.
  * [6] M. Bi _et al._ , “Anomaly detection model of user behavior based on principal component analysis,” _Journal of Ambient Intelligence and Humanized Computing_ , vol. 7, no. 4, pp. 547–554, 2016.
  * [7] S. Deep _et al._ , “A survey on anomalous behavior detection for elderly care using dense-sensing networks,” _IEEE Communications Surveys & Tutorials_, vol. 22, no. 1, pp. 352–370, 2019.
  * [8] K. N. Swaroop _et al._ , “A health monitoring system for vital signs using IoT,” _Internet of Things_ , vol. 5, pp. 116–129, 2019.
  * [9] D. Gupta _et al._ , “Future smart connected communities to fight covid-19 outbreak,” _Internet of Things_ , p. 100342, 2020.
  * [10] R. Jurdak _et al._ , “Wireless sensor network anomalies: Diagnosis and detection strategies,” in _Intelligence-Based Systems Engineering_. Springer.
  * [11] C. Fu, Q. Zeng, and X. Du, “Hawatcher: Semantics-aware anomaly detection for appified smart homes.”
  * [12] L. R. Rabiner, “A tutorial on hidden Markov models and selected applications in speech recognition,” _Proceedings of the IEEE_ , 1989.
  * [13] M. Yamauchi _et al._ , “Anomaly Detection in Smart Home Operation From User Behaviors and Home Conditions,” _IEEE Transactions on Consumer Electronics_ , vol. 66, no. 2, pp. 183–192, 2020.
  * [14] M. Zhang, A. Raghunathan, and N. K. Jha, “MedMon: Securing medical devices through wireless monitoring and anomaly detection,” _IEEE Transactions on Biomedical circuits and Systems_ , vol. 7, no. 6, 2013.
  * [15] K. Tonchev _et al._ , “Non-intrusive sleep analyzer for real time detection of sleep anomalies,” in _2016 39th International Conference on Telecommunications and Signal Processing (TSP)_. IEEE, 2016, pp. 400–404.
  * [16] T. T. Pham _et al._ , “Wearable healthcare systems: A single channel accelerometer based anomaly detector for studies of gait freezing in parkinson’s disease,” in _IEEE Int. Conf. on Communications_ , 2017.
  * [17] P. Parvin _et al._ , “Anomaly Detection in the Elderly Daily Behavior,” in _IEEE 2018 14th International Conference on Intelligent Environments_.
  * [18] A. Ukil _et al._ , “IoT healthcare analytics: The importance of anomaly detection,” in _2016 IEEE 30th international conference on advanced information networking and applications (AINA)_ , 2016, pp. 994–997.
  * [19] D. Gupta _et al._ , “Access Control Model for Google Cloud IoT,” in _Proc. of IEEE 6th Intl Conf. on Big Data Security on Cloud_ , 2020.
  * [20] M. Gupta _et al._ , “Secure V2V and V2I communication in intelligent transportation using cloudlets,” _IEEE Trans. on Services Comput._ , 2020\.
  * [21] M. Gupta and R. Sandhu, “Towards activity-centric access control for smart collaborative ecosystems,” in _Proceedings of the ACM Symposium on Access Control Models and Technologies_ , 2021, pp. 155–164.
  * [22] D. Gupta _et al._ , “Learner’s Dilemma: IoT Devices Training Strategies in Collaborative Deep Learning,” in _2020 IEEE 6th World Forum on Internet of Things (WF-IoT)_ , pp. 1–6.
  * [23] Ö. Aslan _et al._ , “Intelligent behavior-based malware detection system on cloud computing environment,” _IEEE Access_ , 2021.
  * [24] D. N. Monekosso _et al._ , “Behavior analysis for assisted living,” _IEEE Transactions on Automation science and Engineering_ , 2010.
  * [25] W. Kang _et al._ , “Detecting and predicting of abnormal behavior using hierarchical markov model in smart home network,” in _IEEE Int. Conf. on Industrial Engineering and Engineering Management_ , pp. 410–414.
  * [26] O. Kotevska _et al._ , “Kensor: Coordinated Intelligence from Co-Located Sensors,” in _2019 IEEE International Conference on Big Data_.
  * [27] J. Li _et al._ , “Multivariate time series anomaly detection: A framework of Hidden Markov Models,” vol. 60, pp. 229–240, 2017.
  * [28] S. Ramapatruni _et al._ , “Anomaly detection models for smart home security,” in _IEEE 5th Intl Conference on Big Data Security on Cloud_.
  * [29] S. N. Narayanan _et al._ , “OBD_SecureAlert: An anomaly detection system for vehicles,” in _IEEE Int. Conf. on Smart Computing_ , 2016.
  * [30] H. Ren, Z. Ye, and Z. Li, “Anomaly detection based on a dynamic Markov model,” _Information Sciences_ , vol. 411, pp. 52–65, 2017.
  * [31] Y. Zhu, “Automatic detection of anomalies in blood glucose using a machine learning approach,” _Journal of Communications and Networks_.
  * [32] S. Bhatt _et al._ , “An access control framework for cloud-enabled wearable Internet of Things,” in _2017 IEEE 3rd International Conference on Collaboration and Internet Computing_ , 2017, pp. 328–338.