# A Neurorobotics Approach to Behaviour Selection based on Human Activity
Recognition

Caetano M. Ranieri
Institute of Mathematical and Computer Sciences
University of Sao Paulo
Sao Carlos, SP, Brazil
<EMAIL_ADDRESS>
&Renan C. Moioli
Digital Metropolis Institute
Federal University of Rio Grande do Norte
Natal, RN, Brazil
<EMAIL_ADDRESS>&Patricia A. Vargas
Edinburgh Centre for Robotics
Heriot-Watt University
Edinburgh, Scotland, UK
<EMAIL_ADDRESS>
&Roseli A. F. Romero
Institute of Mathematical and Computer Sciences
University of Sao Paulo
Sao Carlos, SP, Brazil
<EMAIL_ADDRESS>

###### Abstract

Behaviour selection has been an active research topic for robotics, in
particular in the field of human-robot interaction. For a robot to interact
effectively and autonomously with humans, the coupling between techniques for
human activity recognition, based on sensing information, and robot behaviour
selection, based on decision-making mechanisms, is of paramount importance.
However, most approaches to date consist of deterministic associations between
the recognised activities and the robot behaviours, neglecting the uncertainty
inherent to sequential predictions in real-time applications. In this paper,
we address this gap by presenting a neurorobotics approach based on
computational models that resemble neurophysiological aspects of living
beings. This neurorobotics approach was compared to a non-bioinspired,
heuristics-based approach. To evaluate both approaches, a robot simulation is
developed, in which a mobile robot has to accomplish tasks according to the
activity being performed by the inhabitant of an intelligent home. The
outcomes of each approach were evaluated according to the number of correct
outcomes provided by the robot. Results revealed that the neurorobotics
approach is advantageous, especially considering the computational models
based on more complex animals.

_K_ eywords Behaviour selection $\cdot$ Human activity recognition $\cdot$
Robot simulation $\cdot$ Neurorobotics $\cdot$ Bioinspired computational model

##  1 Introduction

Truly autonomous behaviour is still not the norm for robots designed to
interact socially with humans [6]. In general, behaviour selection has been an
active research topic for robotics in general, and human-robot interaction in
particular [19]. In this context, the need for a real-time understanding of
human actions is of paramount importance for the robotic agent to behave
proactively and effectively. Such a requirement could be achieved with
techniques for human activity recognition [32].

When dealing with complex modalities (e.g., videos or data from inertial
units), activity recognition approaches often rely on machine learning. For
instance, video-based activity recognition have been approached by
architectures based on convolutional and recurrent neural networks [15, 29].
For inertial data, similar architectures have been proposed, processing either
raw data [35, 10] or descriptors obtained through feature extraction methods
[47, 1]. To provide a wider range of possibilities, robots may act
symbiotically with other pervasive devices, such as wearable technologies or
ambient sensors in intelligent environments, which may provide additional
capabilities for sensing and acting based on application-specific components
[2]. When synchronised data from different sensors are available, activity
recognition techniques may rely on multiple sensor modalities to provide more
accurate results, giving rise to techniques for multimodal activity
recognition [28, 17, 43].

Although human activity recognition has been a quite fertile field of
research, few approaches have been developed to link the outputs from those
algorithms into actual response behaviours from a robot. Related works usually
consist of direct associations between the recognised activities and the
response behaviours [11, 25, 44]. One of the possibilities consist of
combining computational neuroscience to the robotics scenarios, characterising
the field of neurorobotics [51], which may build upon different biological
aspects that influence the behaviour of living beings.

Li et al. [24] provided a comprehensive survey on neurorobotics systems (NRS)
and the different components that may integrate them. According to the
authors, a generalised framework can be depicted for most NRSs in the
literature, composed of a simulated brain, which is fed with sensory signals
from a body and turns them into control signals for a hierarchical controller,
responsible for decoding these signals into control commands for the body,
which actuates and senses an external environment. Bioinspired strategies may
be introduced to different aspects of the framework, according to the required
task of a particular study.

The basal ganglia, a group of subcortical nuclei present in the vertebrate’s
brain, is known to have an important role in action selection mechanisms,
especially regarding striatal circuits [30]. The so-called direct and indirect
pathways are characterised by competitive or complementary functions that
mediate the excitation of the motor system based on inputs from the
motivational system of an individual, deciding whether to ”go” or to ”stop”
performing a certain behaviour [4]. The potential roles of such a mechanism in
robotic frameworks have also been evaluated, including simulations in which
bioinspired networks receiving different stimuli are expected to respond with
different behaviours, resulting in cooperative interactions that produce robot
behaviours [3].

In this paper, we present a neurorobotics model which embeds computational
models of the basal ganglia-thalamus-cortex (BG-T-C) circuit [22, 40] to
provide a decision-making mechanism for a robot - in this context, we may call
it a neurorobot. The neurorobotics approach has been proposed for enhancing
the decision-making mechanism, as suggested by related researches in
neurorobotics [26, 33]. It consisted of simulating neurophysiological aspects
within a cognitive framework, in which different stimuli was introduced to
certain brain structures within the circuit, according to real-time outputs of
the activity recognition module. The resulting spike trains from the
neurorobotics model were then converted to neural firing frequencies across
brain regions, which would be further decoded using convolutional neural
networks, in order to infer the most suitable response behaviour for the
robot.

The application scenario is a simulated smart home, in which an activity
recognition model, presented in [41] for the HWU-USP activities dataset [38],
was employed in human-robot interaction tasks, using a mobile robot. In
summary, the robot needs to produce response behaviours according to the
contextual information inferred by the user (i.e., the recognised activity).

The neurorobotics approach, which is the central contribution of this work,
was compared to a heuristics approach, in which a deterministic behaviour
selection mechanism was considered using simple heuristics that associate
recognised activities to robot behaviours. This neurorobotics approach
embedded two computational models, one that resembled neurophysiological data
of rodents (i.e., the rat model), and one that resembled data from marmoset
monkeys (i.e., the primate model).

The different factors considered for this study were evaluated according to
the relative number of correct outcomes of the robot simulation. Considering
the activity recognition framework, the results have confirmed that more
accurate classifiers for the activity recognition module led to a greater
number of robot tasks successfully completed. Although the performances of the
heuristic and neurorobotics approaches varied according to the computational
model embedded, the study confirmed that the most complex neurorobotics model
(i.e., the marmoset-based model of the BG-T-C circuit) led to an increased
performance in relation to the heuristic approaches when a more accurate
activity recogniser was considered (i.e., the video-based classifier).

The remainder of this paper is organised as follows. The brain structures
considered for this neurorobotics approach and the computational modelling
adopted are presented in Section 2. In Section 3, are presented the general
aspects of the robotic system, and the integration between each of its
modules. In Section 4, the neurorobotics approach is detailed. In Section 5,
the methods and implementations are depicted. The corresponding results are
presented in Section 6 and discussed in Section 7. Finally, the concluding
remarks and directions for future research are provided in Section 8.

##  2 The BG-T-C Circuit and Original Computational Models

In this section, we present the basic concepts on the brain structures present
in the basal ganglia-thalamus-cortex (BG-T-C) circuit, and the original
computational modelling. The BG-T-C circuit, illustrated in Figure 1, is
formed by the motor cortex (M1), the thalamus (TH), and the basal ganglia
(BG), the latter composed of a subset of structures: the striatum (Str), the
globus pallidus, divided into pars interna (GPi) and pars externa (GPe), the
subthalamic nucleus (STN), and the substantia nigra, divided into pars
compacta (SNc) and pars reticulata (SNr).

In [31], is provided a discussion about the mechanisms of this circuit and
presented models to describe it. The most useful model to explain the
connections within this circuit, especially those affected by PD, is the so-
called classic model, illustrated in Figure 1(a).

(a) Classic model, as described by [31].

(b) Computational model, as designed by [22] for rodent data, and adapted by
[40] for primate data.

Figure 1: Schematic representations of the classic and computational models of
the BG-T-C circuit. In the connections, excitatory synapses are shown as blue
arrows, and inhibitory synapses, as red squares.

The pathways start with an excitatory connection from the cortex to the
striatum, which projects its output neurons, named medium spiny neurons (MSN),
to other structures inside the BG. In the direct pathway, the direct MSN
(dMSN) inhibits the GPi, which reduces its inhibition to the TH. Then, it
excites the motor cortex. In the indirect pathway, the indirect MSN (iMSN)
inhibits the GPe, which reduces its inhibition to the STN, which excites the
GPi. Thus, this results on inhibition of the TH and absence of excitatory
outputs to the motor cortex. In other words, the direct pathway excites the
cortex (i.e., positive feedback loop), while the indirect pathway inhibits it
(i.e., negative feedback loop).

In [22], a computational model of the BG-C-T circuit, originaly developed to
study the underlying mechanisms of Parkinson’s Disease (PD), was proposed and
implemented based on neural data from healthy and PD-induced (i.e., 6-OHDA
lesioned) rats [18]. Eight brain structures were modelled and connected based
on a simplified version of the classic model (see Figure 1(b)). In particular,
the direct and indirect pathways were modelled separately representing the MSN
modulation by D1 and D2 dopamine receptors in the striatum (i.e., StrD1 and
StrD2, respectively). The cortex is represented by regular spiking (RS)
excitatory neurons and fast spiking (FSI) inhibitory interneurons (i.e., CtxRS
and CtxFSI, respectively). A bias current was added in the TH, GPe, and GPi,
accounting for the inputs not explicitly modelled. This model was designed
with the ability to shift from the simulation of healthy to the PD status,
which is done by altering certain conductances.

Although all mammals have a similar set of BG structures that are similarly
connected with thalamic and cortical structures, subtle differences between
species may be found, with primates being more similar to humans than rodents
[27, 21, 7]. A data-driven approach was proposed in [40] to obtain a primate-
based computational model of the BG-T-C circuit and the mechanisms of PD. The
resulting marmoset computational model was evaluated based on the differences
between healthy and PD individuals, with respect to the spectral signature of
the brain activity [49], the dynamics of the firing rates of neurons across
brain regions [50], and the coherence between spike trains [14].

The implementation used in [40] built on a Python translation of the original
computational model of [22], originally made by [46] using the NetPyNE
framework and the libraries from the NEURON simulator [8]. Based on the
results of the machine learning framework, a practical setup of either the rat
or marmoset computational models was made available. The adaptations performed
in this work to the original computational models (see Subsection 4.1) were
based on the code made available by the authors. For all neurorobotics model
evaluations, we considered both the rat and primate computational models,
always with the healthy state set on.

##  3 Integrated System

The modules of the application scenario, and the interactions between them,
are illustrated in Figure 2. In this scenario, the human activities are
inferred by a machine learning algorithm, and the supporting behaviours are
performed by a mobile robot placed in a simulated environment, composing an
ambient assisted living (AAL) application [5].

The general information flow was: given the multimodal data provided by a set
of sensors within a sensed environment, apply an activity recognition module
to classify such data into a set of predefined human activities, and produce
correspondent response behaviours for a mobile robot.

The neurorobotics approach was compared to a heuristics approach. The
heuristics approach (Figure 2(a)) consisted of associating the predictions of
the activity recognition module to response behaviours based on simple
heuristics, presented in Subsection 5.4. In the neurorobotics approach (Figure
2(b)), the predictions from the activity recognition module were employed to
stimulate a bioinspired computational model, whose outputs (i.e., neural
firing frequencies of brain simulated regions) were decoded by a CNN-based
decoder, which provided the decisions for the mobile robot.

(a) Heuristics approach: the predictions from the activity recognition module
are fed directly to the mobile robot.

(b) Neurorobotics approach: the predictions are used as stimuli for the
embedded, bioinspired computational model of the BG-T-C circuit, which
simulates neural activity that is further interpreted by a CNN-based decoder,
responsible for deciding the behaviour to be performed by the robot. Both the
bioinspired computational model and the CNN-based decoder compose the
neurorobotics model presented in this research.

Figure 2: Interaction between modules for the application scenario proposed.

More specifically, a sensed environment consisted of a previously collected
dataset [38], composed by a set of recording sessions $X$, with each $x\in X$
associated to an activity $a\in A=\\{a_{1},\dots,a_{N_{A}}\\}$, where $N_{A}$
is the number of classes (i.e., labels) considered for this dataset. The
function describing these associations is given by Equation 1.

$\begin{split}f_{A}:X\rightarrow A\Longleftrightarrow f_{A}(x)=a,\\\ x\in
X,\quad a\in A\end{split}$ (1)

Each data tuple $x(t)$ comprises a segment, with a previously defined length,
of a recording session $x$ starting at timestep $t\in
T_{x}=\\{1,\dots,N_{T_{X}}\\}$, equally spaced among them, to be segmented
from $x$. The activity recognition module is a machine learning classifier
$g\in G$, which might associate a recording session $x\in X$ at timestep $t\in
T_{X}$ to an activity $a\in A$, through a prediction vector $y_{x}^{t}\in Y$
(see Equation 2). In other words, considering that $a$ is unknown at inference
time, the inference model $g$, learned from labelled samples, provides a
prediction vector $y_{x}^{t}$, where $y_{x}^{t}(a)$ is the probability that a
given input $x(t)$ corresponds to activity $a$.

$\begin{split}g:X\times T\rightarrow Y\Longleftrightarrow g(x,t)=y_{x}^{t},\\\
x\in X,\quad t\in T,\quad y_{x}^{t}\in Y\end{split}$ (2)

The application scenario was designed so that each activity in $A$ was
associated to a desired response for the mobile robot. We defined a set of
response behaviours $B=\\{b1,\dots,b_{N_{B}}\\}$, so that each human activity
$a\in A$ can be, but not necessarily is, associated to a response behaviour of
the robot. The ”no action” behaviour is denoted as $b_{\emptyset}$. Hence, the
function that associates recognised activities to response behaviours is given
by Equation 3.

$\begin{split}f_{B}:A\rightarrow B\cup\\{b_{\emptyset}\\}\Longleftrightarrow
f_{B}(a)=b,\\\ a\in A,\quad b\in B\cup\\{b_{\emptyset}\\}\end{split}$ (3)

The robot simulation would be considered successfully completed if:

  * •

For an activity $a$ being performed in the environment in a session $x$, the
robot completed an expected response behaviour $b\in B$ before $x$ was
finished; or

  * •

No response behaviour was expected (i.e., $f_{B}(a)=b_{\emptyset}$) and the
robot did not complete any of the behaviours in $B$.

It is worth to notice that, according to this evaluation policy, besides an
accuracy requirement (i.e., the correct behaviour must be given in response to
a human activity), there was also a time constraint that must be satisfied
(i.e., if required, the response behaviour must be completed while the human
is still performing the given activity).

Since, by definition, $f_{A}(x)=a$ is not known at runtime, and can only be
inferred by a classifier $g\in G$ as successive prediction vectors $y_{x}^{t}$
are provided, a decision-making mechanism was needed to perform adaptive
decisions based on partial, time-localised predictions. To this aim, we
proposed the neurorobotics model presented in Section 4, and compared it to a
simple heuristics-based approach as described in Section 5.

##  4 The Neurorobotics Model

The neurorobotics model embeds the bioinspired computational model and the
CNN-based decoder (see Figure 2). It consists of simulating and decoding the
neurophysiological mechanisms of the basal ganglia-thalamus-cortex (BG-T-C)
circuit in mammals (see Section 2), responsible for abilities such as motor
control, decision-making, and learning [12, 26, 33]. As already stated in
Section 2, both the rat-based [22] and the marmoset-based [40] computational
models were evaluated as a decision-making mechanism of the neurorobotics
model.

###  4.1 Bioinspired Computational Model

Motivated by the work of [33], two key modifications were introduced to the
computational models of the BG-T-C circuit adopted in this work [22, 40].
First, an additional structure, called prefrontal cortex (PFC), was included
as a variable source of excitatory stimuli towards the striatum (see Figure
3(a)). Second, $N_{C}=N_{B}$ populations of neurons were implemented as
independent channels $c\in C$, each associated to exactly one response
behaviour $b\in B$ (see Figure 3(b)), as defined in Equation 4.

$\begin{split}f_{C}:B\rightarrow C\Longleftrightarrow f_{C}(c)=b,\\\ b\in
B,\quad c\in C\end{split}$ (4)

(a) Schematic representation of the computational model as adapted in this
work. In the connections, excitatory synapses are shown as blue or green
arrows, and inhibitory synapses, as red squares. The blue arrows and red
squares correspond to the original synapses as designed by [22] and adapted by
[40], while the green arrows are the adaptations provided in this work to
allow the stimulation of the circuit in the context of the application
scenario proposed.

(b) Predictions from the activity recognition module are interpreted as
stimulation originated on the prefrontal cortex (PFC), which selectively
stimulates different populations of the computational model, each associated
to one response behaviour of the robot.

Figure 3: Adapted version of the computational model of the BG-T-C circuit.

At each timestep, the channels $c\in C$ received a stimulus $s\in
S=\\{s_{1},\dots,s_{N_{C}}\\}$, whose intensity was based on the linear
combination between a prediction vector $y_{x}^{t}$ and a weight function
$f_{W}$, given by Equation 5. The actual value of $s$ is given by function
$f_{S}$, defined as in Equation 6.

$\begin{split}f_{W}:C\times A\rightarrow\\{0,1\\}\Longleftrightarrow
f_{W}(c,a)=\begin{cases}1,\quad\text{if }f_{B}(a)=f_{C}(c)\\\
0,\quad\text{otherwise}\end{cases}\\\ c\in C,\quad a\in A\end{split}$ (5)

$\begin{split}f_{S}:C\times A\rightarrow S\Longleftrightarrow
f_{S}(c,a)=s=\sum_{a\in A}f_{W}(c,a)\cdot y_{x}^{t}(a)\\\ c\in C,\quad a\in
A,\quad s\in S\end{split}$ (6)

Considering that, as ensured by the softmax activation on the classifiers,
$\sum_{a\in A}y_{x}^{t}(a)=1$, then $s\in[0,1]\mu A/cm^{2}$, which has shown
to be a stable, biologically plausible interval. For a recording sequence $x$,
the set of prediction vectors $y_{x}$ is employed to update periodically each
stimulus $s\in S$, during the course of a corresponding simulation of the
computational model (not to be confused with the robot simulation). For each
simulation, $N_{T_{\text{sim}}}$ subsequent updates would be done for all
$s\in S$, computed for the timesteps in $y_{x}$.

A simulation, after finished, produced a spike train for each of the brain
regions modelled, contemplating all its length (i.e., all $N_{T_{\text{sim}}}$
updates were considered). The neural firing frequencies were computed
according to [23], with the parameters detailed in Subsection 5.3, and summed
across each region of each channel, resulting in $N_{R}\cdot N_{C}$ output
signals for each simulation, each with length $L_{U}$, where $N_{R}=8$ is the
number of regions (see Figure 1b).

Formally, let $u_{x}^{g,m}\in U$ be defined as the output for a given
simulation, where $x\in X$ is a recording session, $g\in G$ is the classifier
employed for activity recognition, and $m\in M$, a computational model.
Therefore, let a simulation be defined as $f_{U}$ (see Equation 7), whose
output is as a multivariate time-series with $N_{R}\cdot N_{C}$ variables and
$L_{U}$ timesteps.

$\begin{split}f_{U}:X\times G\times M\rightarrow U\Longleftrightarrow
f_{U}(x,g,m)=u_{x}^{g,m}\\\ (x,g,m)\in X\times G\times M,\quad u\in
U\end{split}$ (7)

After the simulations were completed, the spike trains at the cortex
populations were converted into temporal signals (i.e., neural firing
frequencies) based on the mean firing rates across brain regions [23]. The
resulting signals were segmented in smaller windows and applied to train and
evaluate a convolutional neural network (CNN), which would be employed to
determine the decision of the robot at each timestep of the robot simulation
(i.e., the CNN-based decoder). More details on the implementation of the CNN-
based decoder are presented in the next section.

###  4.2 CNN-Based Decoder

Each simulation of the computational model provided the summed neural firing
frequencies of each channel and brain region, generating a data structure
$u_{x}^{g,m}$, associated to the whole recording session that generated it. As
a requirement to provide a realistic scenario for the robot simulation, time-
localised decisions were required, which must be taken based only in past
events. In other words, at a timestep $t_{\text{robot}}=i$ of the robot
simulation, only predictions obtained on timesteps $t_{j},j<i$ could be taken
into account when providing a response behaviour to the robot.

To fulfil this requirement, each instance $u_{x}^{g,m}$, correspondent to the
recording session $x\in X$ in the set of conditions $g\in G$ and $m\in M$ (see
Equation 7), was segmented in windows of $N_{V}$ timesteps, with partial
superposition, producing $N_{\text{segs}}$ segments. Considering $N_{X}$
recording sessions in a given set of conditions, the function $f_{v}$ would
generate a total of $N_{X}\cdot N_{\text{segs}}$ instances $v\in V$, as
defined in Equation 8.

$\begin{split}f_{v}:U\times T\rightarrow V\Longleftrightarrow
f_{v}(u_{x}^{g,m},t)=v=u_{x}^{g,m}[t,t+N_{V}]\\\ u_{x}^{g,m}\in U,\qquad t\in
T\quad|\quad t+N_{V}<L_{U},\qquad v\in V\end{split}$ (8)

The resulting segments were employed to train a machine learning decoder
(i.e., the CNN-based decoder). We considered only the cortex regions to
compose the input tuples for the decoder, aiming to preserve biological
plausibility regarding this aspect. Given that each channel of the
computational model has two cortex regions (i.e., cortex RS and FSI), and that
the experiments were performed with $N_{C}$ channels, associated to the
response behaviours $b\in B$, the resulting instances $v$ had shape
$N_{V}\times 2N_{C}$. The decoder $f_{Q}$ might be trained to provide a
decision vector $q_{x}^{t}$, which corresponds to the probability that a given
segment of cortex firing frequencies, given by$v=f_{v}(u_{x}^{g,m},t)$, might
be associated to a behaviour in $B\cup\\{b_{\emptyset}\\}$. This decoding
function may be defined as in Equation 9.

$\begin{split}f_{Q}:V\rightarrow Q\Longleftrightarrow f_{Q}(v)=q_{x}^{t},\\\
v\in V,\quad q_{x}^{t}\in Q\end{split}$ (9)

We have adopted a one-dimensional convolutional neural network (CNN) as
decoder, which has shown to provide state-of-the-art results in related work
[42] (for the architectural choices and hyperparameter settings, see
Subsection 5.3). Classification metrics were provided considering that the
categorical output is chosen according to Equation 10, where $d_{Q}$
corresponds to a response behaviour.

$d_{Q}:Q\rightarrow B\Longleftrightarrow d_{Q}(q_{x}^{t})=argmax(q_{x}^{t})$
(10)

Finally, the decisions decoded would be fed to the robot simulation and turned
into commands, as discussed in Subsection 5.4.

##  5 Methods

In Figure 4, the different factors assessed in this work, already mentioned,
are illustrated. Both the heuristics and the neurorobotics approaches were
evaluated with two different models of the activity recognition module: the
IMU + ambient and the video-based (see Subsection 5.1). For the heuristics
approach, a couple heuristics was considered and compared: the window and the
exponential (see Subsection 5.4). For the neurorobotics approach, the rat and
marmoset computational models were assessed (see Subsections 4.1 and 4.2).

Figure 4: Factors and conditions analysed for the heuristics and neurorobotics
approaches. For both approaches, two models for the activity recognition
module were considered: the IMU + ambient and the video-based. For the
heuristics approach, two approaches were analysed for the decision-making
mechanism: window or exponential (see Subsection 5.4). For the neurorobotics
approach, two computational models of the BG-T-C circuit were considered: the
rat-based and the marmoset-based.

All code was developed in Python language. The machine learning techniques
presented for the activity recognition and the CNN-based decoder were
implemented with the Tensorflow/Keras framework. The computational models were
implemented using the NetPyNE platform [8]. The robot simulation was
implemented in the Gazebo simulator [20] with the Robot Operating System (ROS)
[37] as a middleware. The next subsections will provide the implementation
details of this work.

###  5.1 Dataset and Classifiers

We have adopted the HWU-USP activities dataset [38], a multimodal and
heterogeneous dataset of human activities recorded in the Robotic Assisted
Living Testbed (RALT), at Heriot-Watt University (UK). It is composed by
readings of ambient sensors (e.g., switches at wardrobes and drawers, presence
detectors, power measurements), inertial units attached to the waist and to
the dominant wrist of the subjects, and videos. A set of nine well-defined,
pre-segmented activities of daily living was performed by the 16 participants
of the data collection. A total of $N_{X}=144$ recording sessions were
provided, all of them pre-segmented and labelled (i.e., $X$, $A$ and $f_{A}$
were provided). The length of the recording sessions varied from less than 25
to over 100 seconds, with high variance either between-classes and between-
subjects.

As the activity recognition module, we have employed the framework presented
and evaluated in [41]. This was composed by different time-localised
classifiers based on artificial neural networks, each focused on a particular
modality (i.e., set of similar sensors) or set of modalities. We adopted a
couple pre-trained classifiers (i.e., the IMU + ambient and the video-based
classifiers) from the framework to provide the prediction vectors, respecting
the between-subjects 8-fold approach for training and evaluating. Let those
classifiers be denoted by $g_{\text{I+A}}\in G$ and $g_{\text{video}}\in G$,
respectively. Although both classifiers were described in [41], we give a
brief presentation of their architectures in the next paragraphs, for the sake
of completeness.

Classifier $g_{\text{I+A}}$ was fed with two parallel inputs: a two-seconds-
long (i.e., $100$ timesteps-long) time-window with the raw signals from the
inertial sensors, and the mean values of the ambient sensors in the
correspondent timestamps. The inertial data was processed by a one-dimensional
Convolutional Neural Network (CNN) [53], composed of two convolutional layers
interspersed with pooling layers, followed by a Long Short-Term Memory (LSTM)
recurrent layer [16], generating the feature vector $v_{1}$. The ambient data
was processed by a single fully-connected layer, generating the feature vector
$v_{2}$. Both $v_{1}$ and $v_{2}$ were concatenated and sent to a softmax
output layer.

Classifier $g_{\text{video}}$ has taken, as input, a sequence of $25$ optical
flow pairs, correspondent to two seconds of video, computed with the TVL1
algorithm [52]. The InceptionV3 CNN architecture [48] was trained to classify
each optical flow pair individually. The CNN-LSTM architecture, adopted by the
authors, consisted of feeding each optical flow pair within a sequence to this
pre-trained InceptionV3 module, and feeding the resulting features as inputs
to each timestep of a LSTM layer, whose outputs were connected to a softmax
output layer.

Both above-mentioned classifiers were endowed with softmax activation in their
outputs, which ensured that the prediction vector respects a valid probability
distribution. To provide the prediction vectors, we split each recording
session in $N_{T}=140$ timesteps, regardless to its original length, and used
the referred framework to provide the predictions on each of those timesteps.
The effect is to assume that all activities have similar length, a
simplification that allowed the design of more uniform and comparable
experiments related to the bioinspired computational models (Subsection 4.1),
and the robot simulation (Subsection 5.4).

The output of the activity recognition module is, for a whole recording
session $x$ processed by a classifier $g$, a total of $N_{X}=144$ sets of
prediction vectors $y_{x}=\\{y_{x}^{1},\dots,y_{x}^{N_{T}}\\}$, with
$N_{T}=140$. Outputs from both classifiers $g_{\text{I+A}}$ and
$g_{\text{video}}$ were applied to all simulations, as described in the
following subsections.

###  5.2 Heuristics Model Implementation

Two policies $H$ were considered for the heuristics model, named window or
exponential, that is, $H=\\{h_{\text{window}},h_{\text{exponential}}\\}$. This
experimental setup resulted in a total of four conditions for evaluation in
the neurorobotics approach, given by the space $G\times H$.

The window policy consisted of deriving a wider prediction vector $y_{w}^{t}$,
correspondent to $N_{r_{w}}$ timesteps. This was done by averaging the
$N_{r_{w}}$ most recent prediction vectors in $y_{x}$, from the activity
recognition module, as in Equation 11. We have set $N_{r_{w}}=8$, which
corresponds to windows of four seconds from the recording sessions, because
this was the length of the segments considered for the neurorobotics approach
(see Subsection 4.2). On the other hand, the exponential policy consisted of
deriving a prediction vector $y_{e}^{t}$ that considered the whole sequence of
previous prediction vectors in $y_{x}$, with an exponential decay across
iterations, as in Equation 12. If $R=\\{r_{w},r_{e}\\}$ is the set of the
functions to compute $y_{w}^{t}$ and $y_{e}^{t}$, then the decision $d_{r}$ of
the heuristics approach, for either the window or exponential policies, is
given by Equation 13. It is important to note that, for the window policy of
the heuristics approach, as in the neurorobotics approach, the robot can only
begin to move after the first four seconds of each simulation, in which it is
gathering the number of prediction vectors necessary to compute the first
decision.

$\begin{split}r_{w}:Y\times T\rightarrow Y\Longleftrightarrow
y_{w}^{t}=r_{w}(y_{x},t)=\frac{\sum_{i=0}^{N_{r_{w}}-1}y_{x}^{t-i}}{N_{r_{w}}}\\\
y_{x}\in Y,\quad t\in T_{X}|t>N_{r_{w}},\end{split}$ (11)

$\begin{split}r_{e}:Y\times T\rightarrow Y\Longleftrightarrow
y_{e}^{t}=\begin{cases}r_{e}(y_{x},t=0)=y_{x}^{t}\\\ r_{e}(y_{x},t>0)=0.9\cdot
r_{e}(y_{x},t-1)+y_{x}^{t}\end{cases}\\\ \quad y_{x}\in Y,\quad t\in
T_{X}\end{split}$ (12)

$\begin{split}d_{R}:Y\times T\times R\rightarrow B\Longleftrightarrow
d_{R}(y_{x},t,r)=f_{B}(argmax[r(y_{x},t)])\\\ y_{x}\in Y,\quad t\in
T_{X},\quad r\in R\end{split}$ (13)

As a reference, we introduced an additional approach, a control condition in
which the ground truth labels are directly fed to the robot simulation,
providing a unique decision $d_{GT}$ every timestep, as shown by Equation 14.

$\begin{split}d_{GT}:X\rightarrow B\Longleftrightarrow
d_{GT}(x)=f_{B}(f_{A}(x)),\\\ x\in X\end{split}$ (14)

###  5.3 Neurorobotics Model Implementation

Let $M$ be the bioinspired computational model, which can be rat-based or
marmoset-based, that is, $M=\\{m_{\text{rat}},m_{\text{marmoset}}\\}$. This
experimental setup resulted in a total of four conditions for evaluation in
the neurorobotics approach, given by the space $G\times M$. Each independent
simulation of the computational model (not to be confused with the robot
simulation) was ran for each of the $N_{X}=144$ recording sessions under each
condition being evaluated, that is, the simulations of the computational
models were required to contemplate all instances in the space $X\times
G\times M$. Hence, a total of $576$ simulations of the computational model was
performed.

Each of those simulations ran for $70$ seconds with sampling rate of $1,000$
Hz. The stimuli set $S$ was updated every $0.5$ second (i.e., update frequency
of 2 Hz). This led to an adaptive dynamic that would respond to successive
prediction vectors $y_{x}^{t}$,
$t_{\text{sim}}\in\\{1,\dots,N_{T_{\text{sim}}\\}}$, with
$N_{T_{\text{sim}}}=140$, according to the confidence of each response
behaviour. The resulting spike trains in each neuron population were converted
to neural firing frequencies (for details, see [23]), with bins of size $20$,
which resulted in sequences of length $N_{U}=3,500$. As stated in Subsection
5.1, for the experiments reported in this work, $N_{B}=2$, hence $N_{C}=2$.
Considering that $N_{R}=8$, the multivariate time-series $u_{x}^{g,m}\in U$
had $N_{R}\cdot N_{C}=16$ variables and $L_{U}=3,500$ timesteps, composing a
data structure with dimensions $3,500\times 16$,

The segments for the decoder we set to $N_{V}=200$ timesteps (i.e., four-
seconds-long) with $75\%$ superposition (i.e., a one-second-long step between
the beginning of each segment), resulting in $66$ segments. Considering the
each condition was composed of $N_{x}=144$ recording sessions, these
simulations of the computational models led to a total of $144\cdot 66=9,504$
instances $v\in V$, for each $(g,m)\in G\times M$.

The CNN architecture for decoding these time-series into response behaviours
is depicted in Table 1. It was composed of two convolutional layers, with
$128$ and $256$ filters, respectively, interspersed with max-pooling layers. A
global average pooling operation preceded the softmax output layer, which
produced the decision vector $q_{x}^{t}$.

Table 1: Layers in the CNN-based decoder. The inputs to the neural network are windows of $200$ timesteps from the four cortex channels of the output signals (i.e., neural firing frequencies) of the simulations under a given condition. The output is a decision vector $q_{x}^{t}$ with the confidences for each response behaviour. Layer | Type | Output shape | Free parameters
---|---|---|---
1 | Input | $200\times 4$ | -
2 | Conv1D | $200\times 128$ | $3,712$
3 | MaxPool1D | $100\times 128$ | -
4 | Conv1D | $100\times 256$ | $229,632$
5 | MaxPool1D | $50\times 256$ | -
6 | Global Average Pooling | $256$ | -
7 | Softmax | $3$ | -

For each set of conditions, the CNN was trained in a cross-subject 8-fold
cross-validation scheme, similar to the one adopted for the activity
recognition module [41]. The input data was linearly normalised to the range
$[0,1]$, and the classification models were trained for $40$ epochs with batch
size $32$. The ADAM algorithm was employed, with learning rate $10^{-3}$, to
optimise the categorical cross-entropy loss function. The outputs of the
evaluations were stored and organised, in order to serve as inputs to the next
steps. The resulting sequences $u_{x}^{g,m}$ were then introduced to the
decision-making mechanism.

###  5.4 Robot Behaviours

The behaviours $b\in B$ consisted of transporting an object $o\in O$, from a
starting position $z\in Z$ to a fixed destination $z_{\text{dest}}$. This task
was adopted because it comprises a basic and generic functionality for a
mobile robot in a home environment. The associations between the behaviours
and the objects are given by Equation 15, while the associations between the
objects and their starting positions in the map are given by Equation 16.

$\displaystyle f_{O}:B\rightarrow O\Longleftrightarrow f_{O}(b)=o,\quad b\in
B,\quad o\in O$ (15) $\displaystyle f_{Z}:O\rightarrow Z\Longleftrightarrow
f_{Z}(o)=z,\quad o\in O,\quad z\in Z$ (16)

At each timestep $t_{\text{robot}}$, a decision $d\in
B\cup\\{b_{\emptyset}\\}$ (i.e., a response for each recording session $x\in
X$ of the activity recognition module) was sent to the robot simulation,
composed of a mobile social robot in a home environment (for details on the
platforms and implementations employed, see Subsection 5.5). For the
neurorobotics approach, this decision is given by Equation 10, already
presented in Subsection 4.2. For the heuristics approach, the two policies
mentioned (i.e., window and exponential) were evaluated.

The decisions were turned into commands to the robot following a table of
rules, depicted in Table 2. A decision $d$ is sent to the robot at each
timestep. This decision can be one of the behaviours in $b\in B$ or the ”no
action” behaviour $b_{\emptyset}$. Let $o_{c}$ be the object being carried by
the robot at a certain timestep. Two types of situations might be considered:
$d\in B$ or $d=b_{\emptyset}$.

Table 2: Table of rules associating a response behaviour $f(d)=b$ to an output command at each timestep $t_{\text{robot}}$ of the robot simulation, considering the object being carried and the current robot position. Decision | Object carried | Robot position | Output command
---|---|---|---
$b\in B$ | $o_{c}=\emptyset$ | $z\neq f_{Z}[f_{O}(b)]$ | Move towards $f_{Z}[f_{O}(b)]$
$b\in B$ | $o_{c}=\emptyset$ | $z=f_{Z}[f_{O}(b)]$ | Set $o_{c}=f_{O}(b)$
$b\in B$ | $o_{c}=f_{O}(b)$ | $z\neq z_{\text{dest}}$ | Move towards $z_{\text{dest}}$
$b\in B$ | $o_{c}=f_{O}(b)$ | $z=z_{\text{dest}}$ | Finish behaviour
$b\in B$ | $o_{c}=o_{k}\in O\quad|\quad o_{k}\neq f_{O}(b)$ | $z\neq f_{Z}[o_{k}]$ | Move towards $f_{Z}[o_{k}]$
$b\in B$ | $o_{c}=o_{k}\in O\quad|\quad o_{k}\neq f_{O}(b)$ | $z=f_{Z}[o_{k}]$ | Set $o_{c}=\emptyset$
$b_{\emptyset}$ | $o_{c}=o_{k}\in O$ | $z\neq f_{Z}[o_{k}]$ | Move towards $f_{Z}[o_{k}]$
$b_{\emptyset}$ | $o_{c}=o_{k}\in O$ | $z=f_{Z}[o_{k}]$ | Set $o_{c}=\emptyset$
$b_{\emptyset}$ | $o_{c}=\emptyset$ | $\forall z\in Z$ | Wait

The first type of situation is characterised by $d=b_{\emptyset}$, in which
the robot must return any object that it may be carrying to the corresponding
position, and then stand still, waiting for any further commands. Otherwise,
$d=b\in B$, the second type of situation, in which the robot is supposed to
grab an object $o_{c}=f_{O}(b)\in O$ from position $f_{Z}[f_{O}(b)]$ to a
destination $z_{\text{dest}}$. If the robot is not carrying any object, that
is, $o_{c}=\emptyset$, then it must move to $f_{Z}[f_{O}(b)]$ and take the
object. If it is already carrying the correct object, then it must move
towards the destination $z_{\text{dest}}$. If it is carrying the wrong object,
it is, $o_{c}=o_{k}\in O\quad|\quad o_{k}\neq f_{O}(b)$, then it must return
it to $f_{Z}[o_{k}]$.

###  5.5 Robot Simulator

The simulator adopted for the robotics experiments was previously made
available as part of the LARa framework [39], consisted of a robot and a
software library. The LARa robot was a mobile social robot built on the top of
a Pioneer P3-DX platform, endowed with a Hokuyo laser, a mini computer, a
Microsoft Kinect sensor, a microphone, a screen, and a speaker. The LARa
library was a set of functionalities implemented to control the robot based on
high-level software interfaces, integrated within the Robot Operating System
(ROS) [37]. Besides navigation skills and a framework for human-robot
interaction, this included a platform for simulation, under conditions that
resembled those of the actual robot, deployed to allow offline experiments.
The Gazebo simulator [20] was employed, and a map of a typical home
environment was designed, as reproduced in Figure 5(a). The simulated robot -
a simplified version of the LARa robot - is shown in Figure 5(b), while the
pieces of furniture employed in the experiments are shown in Figure 5(c).

(a) Map of the whole home environment employed for the experiments.

(b) Simulated mobile robot.

(c) In a different camera angle, the section of the map in which the robot
behaviours were performed, with the indications of the robot and the pieces of
furniture involved in the tasks (i.e., shelf 1, shelf 2 and table).

Figure 5: Virtual environment for the robot simulation, using the Gazebo
platform.

This setting comprised a realistic environment, which provided several
challenging aspects resembling those of a real-world scenario, such as
sensors’ noise, communication delays and mechanical issues. The ROS platform
was employed to connect this simulated environment to a navigation stack,
which provided a 2D occupancy grid in which each position (i.e., cell) might
be considered empty, navigable or obstacle. This representation was generated
previously to the robot simulations reported here, via the GMapping algorithm
[13] for Simultaneous Localisation and Mapping (SLAM). The mapping algorithm
ran while the robot was teleoperated through the whole environment, with the
laser readings and the wheels’ encoders combined to gradually compose the
occupancy grid. Once the grid was created, the Augmented Monte Carlo
Localisation (AMCL) and A* algorithms could be employed as a global planner to
perform autonomous navigation. The navigation package was also endowed with a
local planner, responsible for creating adaptable short-term paths for
obstacle avoidance and environmental changes.

For this work, a set of two response behaviours was defined as
$B=\\{b_{1},b_{2}\\}$. In Table 3, are shown the set of daily activities from
the dataset (i.e., $a_{p}\in A$, $p\in\\{1,\dots,N_{A}\\}$), and the expected
response behaviours associated to each of those activities (i.e.,
$f_{B}(a_{p})$). These were chosen respecting semantic relationships between
the activities (i.e., $b_{1}$ is the desired response when the user is
preparing meals, and $b_{2}$, when he is quietly consuming or exchanging
information).

Table 3: List of activities provided by the HWU-USP activities dataset, and expected response behaviours in the application scenario. Activity $a$ | Description | Response behaviour $b$
---|---|---
$a_{1}$ | making a cup of tea | $b_{1}$
$a_{2}$ | making a sandwich | $b_{1}$
$a_{3}$ | making a bowl of cereals | $b_{1}$
$a_{4}$ | using a laptop | $b_{2}$
$a_{5}$ | using a phone | $b_{2}$
$a_{6}$ | reading a newspaper | $b_{2}$
$a_{7}$ | setting the table | $b_{\emptyset}$
$a_{8}$ | cleaning the dishes | $b_{\emptyset}$
$a_{9}$ | tidying the kitchen | $b_{\emptyset}$

These behaviours were based on the assumption that the user is located in the
kitchen, and that the human activities are being monitored by sensors that are
not affected by the robot actions. The starting position for only the first
robot simulation in a battery of experiments is given in Figure 5. However, it
had negligible effect in the overall results, since this position was not
reset for each simulation, as we discuss later in this subsection.

As shown in Figure 5(c), three pieces of furniture are considered. These are
shelf 1, associated to the robot position $z_{\text{s1}}=f_{Z}(o_{1})$,
$o_{1}\in O$; shelf 2, associated to the robot position
$z_{\text{s2}}=f_{Z}(o_{2})$, $o_{2}\in O$; and table, the destination,
associated to the robot position $z_{\text{dest}}$. The two specific
behaviours considered for the experiments performed, $b_{1}$ and $b_{2}$,
consist, respectively, of transporting object $o_{1}$ from $z_{1}$ (i.e.,
shelf 1) to $z_{\text{dest}}$ (i.e., the table), and transporting object
$o_{2}$ from $z_{2}$ (i.e., shelf 2) to $z_{\text{dest}}$ (i.e., the table).
Considering that shelf 2 is closer to the table than shelf 1, then the
distances required for $b_{1}$ are larger than those for $b_{2}$. As a
consequence, it was expected that, on average, $b_{1}$ required more time to
be completed than $b_{2}$.

The maximum robot simulation time was set to $N_{T_{\text{robot}}}=140$
seconds, with each timestep
$t_{\text{robot}}\in\\{1,\dots,N_{T_{\text{robot}}}\\}$ corresponding to one
second in the simulation. Consequently, an expected response behaviour had to
be finished within $N_{T_{\text{robot}}}$ seconds to be considered
successfully completed. We configured $N_{T_{\text{robot}}}=140$, which in
exploratory experiments has shown to give a reasonable margin for the robot
simulations.

A total of $N_{X}=144$ robot simulations was performed for each condition
analysed. The first simulation for each approach began with the robot
positioned as in Figure 5(c). All the next simulations began without resetting
the robot position after the ending of the previous one, with only the object
flag, corresponding to the object $o_{c}$ being carried by the robot, being
cleared. In this scenario, each simulation could be started with the robot in
any of the positions in $Z$, or in locations belonging to the path between
them.

##  6 Results

Concerning the activity recognition module, its classification results are
presented in [41]. The overall accuracy registered for the classifiers were
computed by taking a set of $25$ prediction vectors obtained for a recording
session and averaging it. A categorical classification was provided by
returning the $argmax$ element in the averaged vector. A cross-validation
approach, following the same cross-subject partitioning adopted for evaluating
the CNN-based decoder in this work, have been performed. The accuracy reported
for the modalities considered for the experiments reported here was $74.30\%$
for $g_{I+A}$, and $93.75\%$ for $g_{\text{video}}$.

The other modules in this work relied on important adaptations to frameworks
previously implemented in related work, as happened to the computational
models and the robot simulation, or to components developed from scratch, case
of the CNN-based decoder. The corresponding results are shown in the following
subsections. The classification metrics from the neural firing frequencies
synthesised with the bioinspired computational models are presented in
Subsection 6.1. The outcomes of the robot simulations, in all conditions
analysed, are presented in Subsection 6.2.

###  6.1 Simulated Neural Firing Frequencies

A sample of the segments $v\in V$, provided in the simulations of the
computational models, is shown in Figure 6. This was generated from a rat
model, being stimulated according to an IMU + ambient classifier as the
activity recognition module. A larger stimulus introduced to the striatum is
expected to increase neural firing rates in the BG-T-C circuit, which might be
propagated to the cortex.

Figure 6: Sample output from the bioinspired computational model of the BG-C-T
circuit. For the motor cortex of each channel, RS and FSI, we considered the
overall mean firing rates computed with time bins of size $20$ milliseconds,
and evaluated on two-seconds-long time windows. This data was used as input
for the CNN-based decoder, in the next step of the bioinspired pipeline.

The overall accuracy and F1-score of the decoder, trained and evaluated
according to the 8-fold cross-subject approach described, are shown in the
bars plot of Figure 7. The classifier used in the activity recognition module
and the computational model employed are shown side-by-side.

Figure 7: Accuracy and F1-score for the CNN-based decoder in classifying a MFR
signal into a set of three possible decisions: B1, B2 or "no action". On
choosing the models for evaluation, two factors were analysed: the modalities
and models employed for activity recognition (IMU + ambient sensors or video-
based) [41], and the computational model considered (rat-based or marmoset-
based) [22, 40].

The decoder was applied as a part of the decision-making mechanism,
responsible for providing decision vectors for the robot simulation. Hence,
its results might be correlated to the correct outcomes of the decisions made
during the robot simulation. In other words, a good accuracy of the decoder
might result in more correct decisions of the robot, which may more often
complete the tasks with the correct outcome. The next subsection will present
the experiments performed to validate this statement. These are the outcomes
of the robot simulation not only for each of those conditions, but also for
each policy employed for the heuristics approach.

###  6.2 Outcomes of the Robot Simulations

As it was mentioned before, three possible outcomes were considered for the
robot simulations, with $f_{A}(x)=a$ being the activity associated to a
recording session $x\in X$:

  * •

Correct, if $f_{B}(a)\in B$ and the activity was completed before the end of
the simulation, or if $f_{B}(a)=b_{\emptyset}$ and no behaviour was completed;

  * •

Incorrect, if the robot completed a behaviour $b_{\text{robot}}\in B$
different from $f_{B}(a)$, i.e., $b_{\text{robot}}\neq f_{B}(a)$;

  * •

Unfinished, if a response behaviour $b\in B$ was expected from the robot, but
no behaviour was completed before the end of the simulation.

In Subsection 5.5, a control condition was introduced, with ground truth
decisions being sent for the robot. For this approach, as it was expected, all
robot simulations let to the correct outcome. In Figure 8(a), the outcomes for
the heuristics approach are presented, with each of the policies analysed
(i.e., window and exponential) being represented in different plots, each
illustrating the outcomes for each classifier considered for the activity
recognition module. The outcomes for the neurorobotics approach are shown in
Figure 8(a), with the classifiers for activity recognition (IMU + ambient or
video) and the computational models (rat or marmoset) being represented.

(a) Outcomes for the robot simulations performed with the heuristics approach.
Four batteries of simulations were performed, considering two factors: the
classifiers employed for activity recognition (IMU + ambient sensors and
video-based) and the policy for the decision-making mechanism (window or
exponential) (see Figure 4a).

(b) Outcomes for the robot simulations performed with the neurorobotics
approach. Four batteries of simulations were performed, related to two factors
analysed: the modalities and models employed for activity recognition (IMU +
ambient sensors or video-based) [41], the computational model considered (rat-
based or marmoset-based) [40] (see Figure 4b).

Figure 8: Outcomes for the robot simulations. Three possible outcomes were
considered: the robot completed the expected (correct) behaviour; the robot
concluded the incorrect behaviour; no behaviour was completed (unfinished),
although an action was required from the robot.

The times elapsed for providing the correct outcome, when a response behaviour
was expected from the robot, were also recorded. The mean and standard
deviations, within all simulations performed for each condition, are
represented in Figure 9. Two separate plots were provided, separating the
classifiers employed for the activity recognition module. The ground truth
approach was reproduced in both of them, since it does not depend on
prediction vectors, but in the ground truth activities.

This metric considers only the outcomes completed successfully. An approach
that provides a fast response with poor accuracy would provide a low time
response, though it would not necessarily provide the correct response
behaviours very often. Hence, the fact that the heuristics approach with the
window policy led to a faster average response than the ground-truth condition
is consistent. Since incorrect and unfinished outcomes were not considered for
the computation of this mean value, this result only shows that, for this
model, the correct outcomes were mostly associated to activities that could be
completed in less time (e.g., the behaviour $b_{2}$).

Figure 9: Average times elapsed across the 144 sequences on each simulation in
which the correct behaviour was performed. Incorrect and unfinished outcomes,
as well as correct outcomes when no action was required from the robot, were
not considered in this evaluation. All simulated models, basend on either
heuristics or neurorobotics, are shown.

##  7 Discussion

The results from the CNN-based decoder, shown in Figure 7, confirmed some
expectations regarding the output signals produced by the simulations of the
computational models according to the stimuli provided: it performed better
for the the video-based classifier than for the IMU + ambient, and for the
marmoset-based model, compared to the rat-based. The accuracy and F1-score
metrics were very close, which considering a strictly balanced dataset, points
that the results were not affected by any serious issues regarding the trade-
off between precision and recall.

All evaluations led to an accuracy measure of over $70\%$ for three classes
(i.e., response behaviours $b_{1}$, $b_{2}$ or $b_{\emptyset}$). It is
important to consider that the stimuli came from noisy prediction vectors from
activity recognition algorithms, whose accuracy is variable across successive
segments [41], with overall accuracy values of $74.30\%$, for $g_{I+A}$, and
$93.75\%$, for $g_{\text{video}}$. These results show that the neural activity
provided by the computational models could be reliably interpreted by the
proposed decoder, even considering segments of limited length (i.e., four-
seconds-long segments within a 70-seconds-long sequence). Hence, this
particular technique for brain signals, analysed in previous studies for
processing related neuronal data of the BG-T-C circuit [34, 42], has shown to
be suitable for the decision-making approach proposed.

Since the accuracy measure of the classifier for activity recognition was
significantly higher for the video-based classifier than for the IMU +
ambient, it was expected that it could be more easily decoded by the neural
network, which was confirmed by the decoder results (see Figure 7). Also, the
marmoset-based model led to better decoding performances than the rat-based
model, which also meets the expectations, considering a more sophisticated
morphology and dynamics in the underlying brain structures in primates than in
rodents [27].

Regarding the robot simulations, heuristics approaches were evaluated in
parallel to the neurorobotics approaches. In most experiments performed in
this work, better performances were found for the models fed by the video-
based classifier than those fed by the IMU + ambient classifier, which was
expected, since the video classifier is expressively more accurate [41]. As
shown in Figure 8(a), the window policy led to a lower number of successfully
completed response behaviours, especially when fed with prediction vectors
coming from the IMU + ambient classifier (less accurate). This condition may
be the fairest comparison to the neurorobotics approach since it limits its
decisions to data from the four-seconds-long segment that precedes a given
decision, the same constraint applied to the CNN-based decoder.

In this context, the neurorobotics approach has shown to provide more accurate
outcomes in most conditions, especially for the marmoset model. For the
IMU+ambient modality of the activity recogniser, the window policy of the
heuristics approach led to $79.9\%$ of correct outcomes, which was surpassed
by the $84.7\%$ result for either the rat or marmoset models. For the video
modality, the window policy of the heuristics approach led to $86.0\%$ of
correct outcomes, which was only slightly above the rat model, which hit
$85.3\%$, and expressively below the marmoset model, which hit $93.7\%$. These
results point that the proposed neurorobotics approach, in the conditions
analysed in this study, may lead to better outcomes than a simple heuristics
for a real-time task of an autonomous robot.

For the exponential policy of the heuristics approach, a particularity was
found: it led to similar results for either the video and IMU + ambient
conditions (i.e., $88.2\%$ and $88.8\%$ of correct outcomes, respectively),
both with more correct outcomes than those of the window policy. This result
is relevant, since it reveals that, by performing a long-term aggregation of
prediction vectors obtained subsequently from a single recording session, it
may be possible to compensate lower accuracy values provided by certain
classifiers that work with different sets of sensors. This possibility might
be considered in practical applications, in which more informative modalities
that usually lead to high accuracy, such as videos, may be either difficult to
be obtained, due to privacy concerns [9], or unfeasible to provide real-time
outputs, due to the high computational cost inherent to the operations
required for processing them [45].

Regarding the different conditions considered for the neurorobotics approach
(i.e., the activity recogniser and the computational model), the expectation
was that, when applied to the robot simulation, the number of correct outcomes
would be comparatively proportional to the accuracy measures of the decoder
(see Figure 7). As shown in Figure 8(b), this expectation was met for most
conditions, although some exceptions were found.

Better results for the marmoset model were expected, since the number of
neurons and the connectivity are larger [36, 21]. The results of the decoder,
previously discussed, corroborate to this hypothesis. For the robot
simulations, considering the video modality, the marmoset model led to the
best results found among all of the simulations, with $93.7\%$ of correct
outcomes, against $85.3\%$ achieved by the rat model. However, for the IMU +
ambient modality, the results were similar for both models. A possible
explanation for this result is that such an increased capacity could
compensate the mistakes for a more accurate activity recognise. In other
words, the prediction vectors across successive segments could assign higher
confidence values (i.e., probabilities) to the expected label (i.e., the
ground-truth activity) for the video-based classifier than for the IMU +
ambient, and the marmoset-based model, more sophisticated, was able to take
more advantage on it than the rat-based.

By measuring the time elapsed in the robot simulations with correct outcomes,
we can see only modest variations across conditions. An important observation
regarding this metric is that a fast response is not necessarily an indication
of a good performance, since this result is affected not only for the
assertiveness of the correct outcomes (i.e., few changes of decision within a
simulation), but also to the accuracy of the simulations in a given set of
conditions. For instance, a given condition may lead to fast response when it
provides the correct outcome, but most simulations may lead to an incorrect or
unfinished outcome.

For the neurorobotics approach, the times were approximately similar between
both classifiers, except for the marmoset model, which took significantly
longer to finish, on average, when fed with the video-based classifier.
Considering the heuristics approach, the video-based classifier led to clearly
longer times for completing the behaviours, which was probably because some of
the changes in decisions (i.e., the robot is performing behaviour $b_{1}$, but
the decision-making mechanism changes it to $b_{2}$ after receiving new,
updated prediction vectors) within the simulations allowed for completing more
simulations with the correct outcome. The same reason explains why the correct
outcomes of the window policy for the IMU + ambient classifier led to a faster
response, on average, than the ground-truth value.

##  8 Conclusions and Future Work

In this paper, we employed a neurorobotics approach based on the embodiment of
validated computational models of brain structures for creating a decision-
making mechanism to provide effective response behaviours to a mobile robot in
a simulated environment.

The chosen application scenario was a simulated smart home where data from the
sensed environment was processed with a previously designed activity
recognition framework. The neurorobotics approach was compared to some
heuristics. For this, two simple heuristics were proposed and evaluated to
provide real-time decisions based on the outputs from an activity recognition
classifier.

The neurorobotics model used computational models (CM) of the basal ganglia-
thalamus-cortex (BG-T-C) circuit, originally designed to study the underlying
mechanisms of Parkinson’s Disease. The CM were adapted, so that the outputs
from the activity recognition module were applied as stimuli to the striatum
of the circuit, and spike activity at the cortex was decoded with a
convolutional neural network (CNN) to provide decisions to the robot
simulation. Different conditions were analysed, including whether the
computational models were based on rodent or primate models.

Results were reported with respect to the accuracy obtained for the CNN-based
decoder in each condition for the computational model, and to the outcomes of
the robot simulations, considering the neurorobotics and the heuristics
approaches. The expectations were met for most of the different conditions
regarding the neurorobotics approaches. The primate-based computational model
led to the best outcomes between the simulations analysed.

Hence, one can conclude that the proposed neurorobotics approach is promising
not only as an embedded tool for understanding the neurophysiological aspects
of animal behaviour, but also as a practical component to integrate decision-
making mechanisms for action selection in mobile robots engaged in human-
robot-interaction scenarios.

Future work may consist of providing a real-time simulation of the proposed
application scenario, with a robot placed in a physical environment in which
human participants may be performing activities. This would require the
integration among the different modules shown in the pipelines presented, thus
ensuring that all of them can work in real-time. Such an experiment may
validate our approach in even more challenging conditions and scenarios, which
may foster a wide range of applications.

## Acknowledgement

This work was funded by the Sao Paulo Research Foundation (FAPESP), grants
2017/02377-5, 2017/01687-0 and 2018/25902-0, and the Neuro4PD project - Royal
Society and Newton Fund (NAF\R2\180773). Moioli acknowledge the support from
the Brazilian institutions: INCT INCEMAQ of the CNPq/MCTI, FAPERN, CAPES,
FINEP, and MEC. This research was carried out using the computational
resources from the CeMEAI funded by FAPESP, grant 2013/07375-0. Additional
resources were provided by the Robotics Lab within the ECR, and by the Nvidia
Grants program.

## References

  * [1] Sara Ashry, Tetsuji Ogawa, and Walid Gomaa.  CHARM-Deep: Continuous Human Activity Recognition Model Based on Deep Neural Network Using IMU Sensors of Smartwatch.  IEEE Sensors Journal, 20(15):8757–8770, 8 2020.
  * [2] Davide Bacciu, Maurizio Di Rocco, Mauro Dragone, Claudio Gallicchio, Alessio Micheli, and Alessandro Saffiotti.  An ambient intelligence approach for learning in smart robotic environments.  Computational Intelligence, pages 1–28, 7 2019.
  * [3] Jyotika Bahuguna, Philipp Weidel, and Abigail Morrison.  Exploring the role of striatal D1 and D2 medium spiny neurons in action selection using a virtual robotic framework.  Wiley Online Library, 49(6):737–753, 3 2018.
  * [4] S. Bariselli, W. C. Fobbs, M. C. Creed, and A. V. Kravitz.  A competitive model for striatal action selection, 6 2019.
  * [5] Davide Calvaresi, Daniel Cesarini, Paolo Sernani, Mauro Marinoni, Aldo Franco Dragoni, and Arnon Sturm.  Exploring the ambient assisted living domain: a systematic review.  Journal of Ambient Intelligence and Humanized Computing, 8(2):239–257, 2017.
  * [6] Caitlyn Clabaugh and Maja Matarić.  Escaping Oz: Autonomy in Socially Assistive Robotics.  Annual Review of Control, Robotics, and Autonomous Systems, 2(1):33–61, 5 2019.
  * [7] Ted M. Dawson, Todd E. Golde, and Clotilde Lagier-Tourenne.  Animal models of neurodegenerative diseases.  Nature Neuroscience, 21(10):1370–1379, 2018.
  * [8] Salvador Dura-Bernal, Benjamin A Suter, Padraig Gleeson, Matteo Cantarelli, Adrian Quintana, Facundo Rodriguez, David J Kedziora, George L Chadderdon, Cliff C Kerr, Samuel A Neymotin, Robert A McDougal, Michael Hines, Gordon MG Shepherd, and William W Lytton.  Netpyne, a tool for data-driven multiscale modeling of brain circuits.  eLife, 8:e44494, apr 2019.
  * [9] Francisco Erivaldo Fernandes Junior, Guanci Yang, Ha Manh Do, and Weihua Sheng.  Detection of Privacy-sensitive Situations for Social Robots in Smart Homes.  In Automation Science and Engineering (CASE), 2016 IEEE International Conference on, pages 727–732. IEEE, 2016.
  * [10] Felipe Aparecido Garcia, Caetano Mazzoni Ranieri, and Roseli Aparecida Francelin Romero.  Temporal approaches for human activity recognition using inertial sensors.  In Proceedings - 2019 Latin American Robotics Symposium, 2019 Brazilian Symposium on Robotics and 2019 Workshop on Robotics in Education, LARS/SBR/WRE 2019, pages 121–125. Institute of Electrical and Electronics Engineers Inc., 10 2019.
  * [11] Ilche Georgievski, Tuan Anh Nguyen, Faris Nizamic, Brian Setz, Alexander Lazovik, and Marco Aiello.  Planning meets activity recognition: Service coordination for intelligent buildings.  Pervasive and Mobile Computing, 38:110–139, 7 2017.
  * [12] B. Girard, N. Tabareau, Q. C. Pham, A. Berthoz, and J. J. Slotine.  Where neuroscience and dynamic system theory meet autonomous robotics: A contracting basal ganglia model for action selection.  Neural Networks, 21(4):628–641, 5 2008.
  * [13] Giorgio Grisetti, Cyrill Stachniss, and Wolfram Burgard.  Improved techniques for grid mapping with rao-blackwellized particle filters.  IEEE transactions on Robotics, 23(1):34–46, 2007.
  * [14] P. Halje, Ivani Brys, Juan J. Mariman, Claudio Da Cunha, Romulo Fuentes, and Per Petersson.  Oscillations in cortico-basal ganglia circuits: Implications for parkinson’s disease and other neurologic and psychiatric conditions.  Journal of Neurophysiology, 122(1):203–231, 2019.
  * [15] Samitha Herath, Mehrtash Harandi, and Fatih Porikli.  Going deeper into action recognition: A survey.  Image and Vision Computing, 60:4–21, 4 2017.
  * [16] Sepp Hochreiter and Jürgen Schmidhuber.  Long short-term memory.  Neural Computation, 9(8):1735–1780, 11 1997.
  * [17] Javed Imran and Balasubramanian Raman.  Evaluating fusion of RGB-D and inertial sensors for multimodal human action recognition.  Journal of Ambient Intelligence and Humanized Computing, 11(1):189–208, 1 2020.
  * [18] Hitoshi Kita and Takako Kita.  Cortical stimulation evokes abnormal responses in the dopamine-depleted rat basal ganglia.  Journal of Neuroscience, 31(28):10311–10322, 2011.
  * [19] Woo Ri Ko and Jong Hwan Kim.  Behavior Selection of Social Robots Using Developmental Episodic Memory-Based Mechanism of Thought.  In 2018 IEEE International Conference on Consumer Electronics - Asia, ICCE-Asia 2018. Institute of Electrical and Electronics Engineers Inc., 11 2018.
  * [20] Nathan Koenig and Andrew Howard.  Design and use paradigms for gazebo, an open-source multi-robot simulator.  In 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)(IEEE Cat. No. 04CH37566), volume 3, pages 2149–2154. IEEE, 2004.
  * [21] James B. Koprich, Lorraine V. Kalia, and Jonathan M. Brotchie.  Animal models of alpha-synucleinopathy for Parkinson disease drug development.  Nature Reviews Neuroscience, 18(9):515–529, 8 2017.
  * [22] Karthik Kumaravelu, David T. Brocker, and Warren M. Grill.  A biophysical model of the cortex-basal ganglia-thalamus network in the 6-OHDA lesioned rat model of Parkinson’s disease.  Journal of Computational Neuroscience, 40(2):207–229, 4 2016.
  * [23] Petr Lánský, Roger Rodriguez, and Laura Sacerdote.  Mean Instantaneous Firing Frequency is Always Higher Than the Firing Rate.  Neural Computation, 16(3):477–489, 3 2004.
  * [24] Junjun Li, Zhijun Li, Fei Chen, Antonio Bicchi, Yu Sun, and Toshio Fukuda.  Combined Sensing, Cognition, Learning, and Control for Developing Future Neuro-Robotics Systems: A Survey.  IEEE Transactions on Cognitive and Developmental Systems, 11(2):148–161, 6 2019.
  * [25] Kang Li, Jinting Wu, Xiaoguang Zhao, and Min Tan.  Real-Time Human-Robot Interaction for a Service Robot Based on 3D Human Activity Recognition and Human-Mimicking Decision Mechanism.  In 8th Annual IEEE International Conference on Cyber Technology in Automation, Control and Intelligent Systems, CYBER 2018, pages 498–503. Institute of Electrical and Electronics Engineers Inc., 4 2019.
  * [26] Yabin Liang, Zikai Yan, Qi Zhang, Hongyu Liang, Xiyu Ji, Yin Liu, and Rong Liu.  A decision-making model based on basal ganglia account of action prediction.  In IEEE International Conference on Robotics and Biomimetics, ROBIO 2019, pages 1705–1710. Institute of Electrical and Electronics Engineers Inc., 12 2019.
  * [27] Jean Liénard and Benoît Girard.  A biologically constrained model of the whole basal ganglia addressing the paradoxes of connections and selection.  Journal of Computational Neuroscience, 36(3):445–468, 2014.
  * [28] Yantao Lu and Senem Velipasalar.  Autonomous Human Activity Classification from Wearable Multi-Modal Sensors.  IEEE Sensors Journal, 19(23):11403–11412, 12 2019.
  * [29] Chih Yao Ma, Min Hung Chen, Zsolt Kira, and Ghassan AlRegib.  TS-LSTM and temporal-inception: Exploiting spatiotemporal dynamics for activity recognition.  Signal Processing: Image Communication, 71:76–87, 2 2019.
  * [30] Jeffrey E. Markowitz, Winthrop F. Gillis, Celia C. Beron, Shay Q. Neufeld, Keiramarie Robertson, Neha D. Bhagat, Ralph E. Peterson, Emalee Peterson, Minsuk Hyun, Scott W. Linderman, Bernardo L. Sabatini, and Sandeep Robert Datta.  The Striatum Organizes 3D Behavior via Moment-to-Moment Action Selection.  Cell, 174(1):44–58, 6 2018.
  * [31] Matthew M. McGregor and Alexandra B. Nelson.  Circuit Mechanisms of Parkinson’s Disease, 3 2019.
  * [32] Roghayeh Mojarad, Ferhat Attal, Abdelghani Chibani, Sandro Rama Fiorini, and Yacine Amirat.  Hybrid Approach for Human Activity Recognition by Ubiquitous Robots.  In IEEE International Conference on Intelligent Robots and Systems, pages 5660–5665. Institute of Electrical and Electronics Engineers Inc., 12 2018.
  * [33] Garrett Mulcahy, Brady Atwood, and Alexey Kuznetsov.  Basal ganglia role in learning rewarded actions and executing previously learned choices: Healthy and diseased states.  PLoS ONE, 15(2):1–26, 2020.
  * [34] Shu Lih Oh, Yuki Hagiwara, U. Raghavendra, Rajamanickam Yuvaraj, N. Arunkumar, M. Murugappan, and U. Rajendra Acharya.  A deep learning approach for Parkinson’s disease diagnosis from EEG signals.  Neural Computing and Applications, pages 1–7, 2018.
  * [35] Francisco Ordóñez and Daniel Roggen.  Deep convolutional and LSTM recurrent neural networks for multimodal wearable activity recognition.  Sensors, 16(1):115, 1 2016.
  * [36] Tony J. Prescott, Fernando M. Montes González, Kevin Gurney, Mark D. Humphries, and Peter Redgrave.  A robot model of the basal ganglia: Behavior and intrinsic processing.  Neural Networks, 19(1):31–61, 1 2006.
  * [37] Morgan Quigley, Ken Conley, Brian Gerkey, Josh Faust, Tully Foote, Jeremy Leibs, Rob Wheeler, and Andrew Y Ng.  Ros: an open-source robot operating system.  In ICRA workshop on open source software, volume 3, page 5. Kobe, Japan, 2009.
  * [38] Caetano M. Ranieri, Scott MacLeod, Mauro Dragone, Patricia A. Vargas, and Roseli A. F. Romero.  Human activities with videos, inertial units and ambient sensors.  Dryad Digital Repository, 2021.
  * [39] Caetano M. Ranieri, Guilherme Nardari, Adam H.M. Pinto, Daniel C. Tozadore, and Roseli A.F. Romero.  LARa: A robotic framework for human-robot interaction on indoor environments.  In Proceedings - 15th Latin American Robotics Symposium, 6th Brazilian Robotics Symposium and 9th Workshop on Robotics in Education, LARS/SBR/WRE 2018, pages 383–389. Institute of Electrical and Electronics Engineers Inc., 12 2018.
  * [40] Caetano M. Ranieri, Jhielson M. Pimentel, Marcelo R. Romano, Leonardo A. Elias, Roseli A. F. Romero, Michael Lones, Mariana F. P. Araujo, Patricia A. Vargas, and Renan C. Moioli.  A data-driven biophysical computational model of parkinson’s disease based on marmoset monkeys.  Manuscript submitted for publication., 2021.
  * [41] Caetano Mazzoni Ranieri, Scott MacLeod, Mauro Dragone, Patricia Amancio Vargas, and Roseli Aparecida Francelin Romero.  Activity Recognition for Ambient Assisted Living with Videos, Inertial Units and Ambient Sensors.  Sensors, 21(3):768–, 1 2021.
  * [42] C.M. Ranieri, R.C. Moioli, R.A.F. Romero, M.F.P. De Araujo, M.B. De Santana, J.M. Pimentel, and P.A. Vargas.  Unveiling Parkinson’s Disease Features from a Primate Model with Deep Neural Networks.  In Proceedings of the International Joint Conference on Neural Networks, 2020.
  * [43] C.M. Ranieri, P.A. Vargas, and R.A.F. Romero.  Uncovering Human Multimodal Activity Recognition with a Deep Learning Approach.  In Proceedings of the International Joint Conference on Neural Networks, 2020.
  * [44] Francisco J. Rodriguez Lera, Francisco Martín Rico, Angel Manuel Guerrero Higueras, and Vicente Matellán Olivera.  A context-awareness model for activity recognition in robot-assisted scenarios.  Expert Systems, 37(2):e12481, 4 2020.
  * [45] Itsaso Rodríguez-Moreno, José María Martínez-Otzeta, Basilio Sierra, Igor Rodriguez, and Ekaitz Jauregi.  Video Activity Recognition: State-of-the-Art.  Sensors, 19(14):3160, 7 2019.
  * [46] Marcelo R. Romano, Renan C. Moioli, and Leonardo A. Elias.  Evaluation of Frequency-Dependent Effects of Deep Brain Stimulation in a Cortex-Basal Ganglia-Thalamus Network Model of Parkinson’s Disease.  In Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS, volume 2020-July, pages 3638–3641. Institute of Electrical and Electronics Engineers Inc., 7 2020\.
  * [47] Odongo Steven Eyobu and Dong Han.  Feature Representation and Data Augmentation for Human Activity Classification Based on Wearable IMU Sensor Data Using a Deep LSTM Neural Network.  Sensors, 18(9):2892, 8 2018.
  * [48] Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna.  Rethinking the inception architecture for computer vision.  In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2818–2826, Las Vegas, NV, USA, 2016. IEEE.
  * [49] Gerd Tinkhauser, Alek Pogosyan, Huiling Tan, Damian M. Herz, Andrea A. Kühn, and Peter Brown.  Beta burst dynamics in Parkinson’s disease off and on dopaminergic medication.  Brain, 140(11):2968–2981, 11 2017.
  * [50] S. J. van Albada and P. A. Robinson.  Mean-field modeling of the basal ganglia-thalamocortical system. I. Firing rates in healthy and parkinsonian states.  Journal of Theoretical Biology, 257(4):642–663, 2009.
  * [51] Patrick Van Der Smagt, Michael A. Arbib, and Giorgio Metta.  Neurorobotics: From vision to action.  In Springer Handbook of Robotics, pages 2069–2094. Springer International Publishing, 1 2016.
  * [52] C. Zach, T. Pock, and H. Bischof.  A duality based approach for real-time TV-L 1 optical flow.  In Joint Pattern Recognition Symposium, pages 214–223. Springer, 2007.
  * [53] Matthew D. Zeiler and Rob Fergus.  Visualizing and understanding convolutional networks.  Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 8689 LNCS:818–833, 2014.