# Neuromorphic Face Analysis: a Survey

Author Name Affiliation<EMAIL_ADDRESS>Federico Becattini2∗ Lorenzo
Berlincioni1∗ Luca Cultrera1∗ Alberto Del Bimbo1
1University of Florence
2University of Siena
∗Equal contribution<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Neuromorphic sensors, also known as event cameras, are a class of imaging
devices mimicking the function of biological visual systems. Unlike
traditional frame-based cameras, which capture fixed images at discrete
intervals, neuromorphic sensors continuously generate events that represent
changes in light intensity or motion in the visual field with high temporal
resolution and low latency. These properties have proven to be interesting in
modeling human faces, both from an effectiveness and a privacy-preserving
point of view. Neuromorphic face analysis however is still a raw and
unstructured field of research, with several attempts at addressing different
tasks with no clear standard or benchmark. This survey paper presents a
comprehensive overview of capabilities, challenges and emerging applications
in the domain of neuromorphic face analysis, to outline promising directions
and open issues. After discussing the fundamental working principles of
neuromorphic vision and presenting an in-depth overview of the related
research, we explore the current state of available data, standard data
representations, emerging challenges, and limitations that require further
investigation. This paper aims to highlight the recent process in this
evolving field to provide to both experienced and newly come researchers an
all-encompassing analysis of the state of the art along with its problems and
shortcomings.

##  1 Introduction

Face analysis for decades has been one of the most studied topics in computer
vision. Some tasks involving faces can even be considered to be solved, as
they can be performed effectively in unconstrained scenarios with off-the-
shelf tools. Face detection is likely the most spread-out application, as it
is performed effortlessly by any kind of device, including personal
smartphones. A wide plethora of methods falls under the umbrella of face
analysis: landmark detection, age estimation, lip reading, eye tracking, 3D
reconstruction, just to name a few. An interesting application of face
analysis is the theoretical possibility of estimating human emotions and
feelings just by observing facial micro-expressions Ben et al. (2021). In
fact, micro-movements of the face, triggered by fine muscle movements, have
been mapped directly into emotions and it is known from psychology studies
that such movements can be involuntary and almost impossible to hide Ekman and
Friesen (1978). To this day, several computer vision applications that strive
to estimate emotions by analyzing faces have been proposed, yet such micro-
movements can happen at an extremely fast rate Yan et al. (2013), that is
likely not going to be fully observable with a traditional RGB camera. On the
other hand, faces are arguably the most sensitive biometric data. Analyzing
faces has thus raised privacy-related concerns, that have also been addressed
in the recent AI Act by the European Commission
111https://www.europarl.europa.eu/RegData/etudes/BRIE/2021/698792/EPRS_BRI(2021)698792_EN.pdf.

These issues have recently led to an increasing interest in the usage of
neuromorphic cameras, as they have shown promising results from different
points of view, including effectiveness, latency, power consumption and
privacy-preservation. Unlike conventional cameras that capture entire frames
at fixed intervals, neuromorphic cameras operate on a fundamentally different
principle, mimicking the asynchronous and event-driven nature of biological
vision. Events are generated only when pixel-level changes in luminance exceed
a predefined threshold. This approach enables the efficient use of
computational resources, as only relevant information is transmitted and
processed. The absence of a fixed frame rate means that these cameras can
capture and process events with microsecond precision, a capability that is
especially advantageous in dynamic and fast-paced environments. This novel
paradigm has the potential to open up new approaches for various applications,
ranging from robotics and autonomous vehicles to surveillance and artificial
intelligence.

In this paper, we propose an overview of the relatively recent field of
research involving event cameras and face analysis, which we dub Neuromorphic
Face Analysis. Our goal is to provide a compendium for computer vision
researchers in the field of face analysis, discussing opportunities and
challenges, as well as taking stock of what is possible with event cameras up
to this day. Not many works exist on the subject, yet they address several
topics of primary importance and we firmly believe that neuromorphic face
analysis will quickly raise the bar compared to traditional frame-based face
analysis.

We first provide a brief overview of what an event camera is and how it
differs from a traditional RGB camera and we discuss its advantages and
challenges in the field of facial analysis in Sec. 2. We also stress the
importance of different event representation strategies and the advantages
that event cameras can offer from a privacy-preservation point of view. We
pass to discussing the applications and the state of the art (Sec. 3),
organizing the present research into macro-areas. Finally, we provide an
analysis on the datasets available the literature in Sec. 4 and we discuss
conclusions and future directions in Sec. 5.

|  Face Detection |  Eye Blink Detection |  Pupil/Eye Detection |  Landmark Detection |  Drowsiness Detection | End-to-End
---|---|---|---|---|---|---
Face Detection | - |  Lenz et al. (2020) Ryan et al. (2021) | Ryan et al. (2021) | - | - |  Bissarinova et al. (2023) Ryan et al. (2021) Barua et al. (2016)
Identity Recognition | Moreira et al. (2022) | Chen et al. (2020b) | - | - | - | -
Lip Reading |  Kanamaru et al. (2023) Tan et al. (2022) Li et al. (2019) | - | - | Yu et al. (2022) | - |  Bulzomi et al. (2023) Yoo et al. (2023) Rios-Navarro et al. (2023)
Voice Activity Detection | Savran et al. (2018) | - | - | Savran (2023b) | - | -
Driver Monitoring Systems |  Liu et al. (2022) Ryan et al. (2021) Ryan et al. (2023) | Ryan et al. (2021) | Ryan et al. (2021) | Liu et al. (2022) |  Kielty et al. (2023) Chen et al. (2020a) |  Yang et al. (2022) Shariff et al. (2023)
Emotion/Expr. Recognition |  Becattini et al. (2022) Berlincioni et al. (2023) | - | - | Berlincioni et al. (2023) | - | Guo and Huang (2023)
Gaze Analysis | Ryan et al. (2023) | - | Angelopoulos et al. (2020) | - | - | Banerjee et al. (2022)
VR/AR | Kang et al. (2023) | - | Kang et al. (2023) | Kang et al. (2023) | - | -
Face Pose alignment | Ryan et al. (2023) | - | - | - | - |  Savran (2023c) Savran (2023a) Savran and Bartolozzi (2020)

Table 1: State of the art divided into intermediate modules (columns) and
applications (rows).

##  2 Event Camera and Face Analysis: Advantages and Challenges

Event cameras exhibit unparalleled advantages in face analysis applications,
offering a paradigm shift in capturing and interpreting facial dynamics. Their
low-latency operation and high temporal resolution ensure real-time
responsiveness to facial expressions and lip or eye movements, which are
critical for applications like human-computer interaction and security
systems. This capability or tracking subtle and swift dynamic facial changes
proves invaluable in scenarios where traditional cameras may miss rapid facial
transitions. Furthermore, the wide dynamic range of event cameras allows for
accurate representation of facial features in challenging lighting conditions,
offering improved performance compared to conventional cameras that may
struggle with overexposure or underexposure. However, these advantages come
with challenges. The asynchronous nature of event data, beneficial for real-
time responsiveness, becomes a challenge in the field of face analysis:
developing algorithms tailored to interpret facial expressions and movements
from sporadic events poses a unique computational hurdle. Processing data
produced by these sensors requires innovative approaches in computer vision to
precisely recognize and understand facial dynamics.

It is crucial to note also that existing methods commonly employed for face
analysis, such as face detectors and landmark detectors designed for
traditional frame-based cameras, may not be directly suitable for event
cameras. In Fig. 1 we show the outputs of a commonly used object detector and
landmark detector (respectively DLIB King (2009) and Face Alignment Bulat and
Tzimiropoulos (2017)) trained on RGB frames from traditional cameras. We
tested such models on video sequences captured with a paired set of event and
RGB cameras. The object detector can properly locate targets only when the
face motion is enough to make it visible (odd columns of Fig. 1). However, the
outputs have an extremely low confidence due to the domain shift. The landmark
detector instead fails to produce meaningful outputs, even if the face is
fully visible (even columns of Fig.1). The asynchronous nature of event data
and the lack of continuous frames present a mismatch with the assumptions
underlying traditional face analysis methods. Furthermore, the limited
availability of standardized datasets specifically designed for training
event-based face analysis models is a notable impediment. Even how events are
represented can pose a challenge, since models trained with a given
representation are likely to lose effectiveness when such representation is
changed.

Figure 1: Failure cases of computer vision models over event data. Top Row:
Samples of face detection and landmark estimation on RGB frames. Bottom Row:
Samples of face detection and landmark estimation on the corresponding Event
frames.

### Data Representation

An important challenge when dealing with event data is the fact that learning
end-to-end models from raw events would require processing a huge amount of
information, as millions of events can be generated every second. This poses a
significant difference between the neuromorphic and RGB domains. To bridge
this gap, researchers often engineer frame-like representations so that event
data can be fed to a computer vision model, such as a convolutional neural
network.

At the time, there is no clear standard and each encoding strategy must be
tuned with specific hyperparameters. One of the most important is usually the
accumulation time $\Delta t$, which controls the temporal granularity at which
the information is processed. Based on the task, different accumulation times
can yield extremely different results. For instance, a face detector would
require a sufficiently large $\Delta t$ in order to capture enough information
for the face to be visible; on the contrary, modeling facial micro-expressions
requires a sufficiently small accumulation time, as fast movements might be
lost among other unrelated events.

The most common representation strategies involve quantizing events into
spatio-temporal histograms. However, events can be processed also as raw
individual events Yu et al. (2022). For this family of approaches minimal to
none pre-processing is done. The data keeps its original format or is
quantized in extremely small accumulation times to simulate a continuous
signal. This strategy often relies on the use of Spiking Neural Networks Yu et
al. (2022); Li et al. (2019). Other works have adopted an intermediate
strategy, i.e. do not use a frame-based representation yet increase the
accumulation interval $\Delta t$, obtaining a voxel of quantized events in the
form of a 3D tensor Bulzomi et al. (2023); Yang et al. (2022).

To treat the task as a more standard computer vision problem, it is common to
transform the raw data to a 2D grid (an image). This allows the use of models
originally developed for processing RGB data. This accumulation can be done,
for example, by counting events in a spatio-temporal neighborhood Kielty et
al. (2023), treating sequences of binary events as binary digits of a decimal
number Innocenti et al. (2021); Becattini et al. (2022) or building a motion
history image by applying an exponential decay over the neighborhood of active
events Mueggler et al. (2017).

### Event Cameras as a Privacy Preservation Layer

A notable advantage of neuromorphic vision is that several works have
attributed to the usage of an event camera the benefit of working under the
preservation of privacy Al-obaidi (2020); Delilovic and Salaj (2021); Han et
al. (2023); Dong et al. (2023). Compared to RGB images, in fact, event streams
offer an additional layer of protection, as they are harder to interpret and
since no signal is generated when both the camera and the observed environment
or subject are static. In particular, Al-obaidi (2020) and Dong et al. (2023)
leverage event cameras for action recognition purposes, while protecting the
identity of the analyzed people. This idea is used by Dong et al. (2023) to
work in sensitive environments such as high schools, where minors are filmed
to prevent and recognize episodes of bullying. Nonetheless, whereas it is
certainly true that intensity information is discarded in event data,
sensitive information might still be recoverable from events Rebecq et al.
(2019). The authors of Zhang et al. (2023) have thus addressed the problem of
encrypting events so to be safely transmitted over an untrusted channel. In
particular, they proposed an encryption strategy that can prevent the usage of
a computer vision model directly on the protected data. Adopting an opposite
paradigm, Ahmad et al. (2022) proposed an event scrambling method that makes
the stream impossible to interpret for the human eye but retaining the
possibility of applying computer vision models effectively, showing that it is
even possible to perform person re-identification on protected data. Finally,
Delilovic and Salaj (2021) and Han et al. (2023) studied the advantages of
processing events on the edge and on nodes of a federated network. What these
works propose is to exploit the low power consumption of event cameras to
enable an efficient computation directly on the edge, without the need to
transmit the data in the first place. Despite proposing an edge computing
paradigm, Delilovic and Salaj (2021) also stresses the fact that event cameras
provide an additional layer of privacy due to the lack of intensity
information.

##  3 Present Research

We now present an overview of the current literature in the field of event-
based vision, related to human faces. Neuromorphic Face Analysis has been
applied to study several sub-topics in the recent literature. We identified a
collection of low-level modules, that are commonly used as an intermediate
step to address different applications. Low-level modules solve simpler tasks,
that provide information about the face such as face detection, landmark
detection, pupil detection, etc. The outputs of such modules can be leveraged
for downstream applications, spanning from identity recognition to driving
monitoring systems. The only notable exception is face detection, which we
consider to be both a low-level module and an application, as it is commonly
framed in both declinations. It is also worth noticing that, not all works in
the literature rely on intermediate modules, tackling the problem in an end-
to-end fashion.

We summarize in Tab. 1 the most common modules and applications that have been
addressed in the literature with an event camera so far, explicitly referring
to the corresponding works from the state of the art. As highlighted in Tab.
1, we identified nine macro areas in which faces are analyzed with an event
camera. In the following, we provide a more in-depth analysis of these lines
of research.

### Face Detection

As discussed in Sec. 2, face detection from event streams is not trivial.
However, it is arguably the most important application in neuromorphic face
analysis as it enables most face-related tasks. Some works in the literature,
have developed neuromorphic face detectors Lenz et al. (2020); Ryan et al.
(2021); Bissarinova et al. (2023); Barua et al. (2016).

Barua et al. (2016) was the first to address the problem of face detection
from event streams. The approach is based on a patch-based model that analyzes
small crops of intensity changes and determines whether they are likely to be
caused by a face. The method was able to achieve good performance on a
benchmark dataset of event streams, demonstrating the feasibility of face
detection from event data. Starting from this pioneering work, Lenz et al.
(2020) introduced an alternative approach to face detection by utilizing eye
blinks as a distinctive cue to identify the presence of a face within a scene.

Differently from these approaches, which specifically leveraged motion-based
features derived from the neuromorphic streams, a few attempts addressed the
task in a similar way to standard solutions in the RGB domain. Ryan et al.
(2021) introduced a gated recurrent YOLO (GR-YOLO) architecture for multi-face
and eye detection. However, this work relied on Neuromorphic-Helen (N-Helen),
a synthetically generated dataset, converted from RGB videos. To attempt to
bridge the gap between synthetic and real events, Bissarinova et al. (2023)
collected an annotated neuromorphic face detection dataset. In addition, they
propose also 12 baseline methods, derived from the RGB literature to predict
bounding boxes and facial landmark coordinates.

Other works have followed similar strategies to develop face detection modules
finalized to solve a downstream task, such as identity recognition Moreira et
al. (2022), emotion recognition Berlincioni et al. (2023), gaze analysis Ryan
et al. (2023), among others. We will discuss in-depth these methods in the
following paragraphs.

### Identity Recognition

Identity recognition is a fundamental task in traditional computer vision, as
it can be applied for surveillance and security. Neuromorphic vision offers a
valuable asset in this direction, as its incredibly low latency can provide a
rich characterization of biometric traits. In fact, Chen et al. (2020b)
presented a biometric authentication system that utilized eye blinks as a
unique and distinctive feature for verifying individuals.

To quantify how much identity-related information is carried by facial
movements and how such information can be extracted by an automatic
neuromorphic system, Moreira et al. (2022) proposed a different event-based
approach for face identity recognition: the authors aggregated events in
frames, that were then normalized and grouped into face tokens. These tokens
represent the facial activity in a specific time window and are analyzed with
a spatio-temporal 3D Convolutional Neural Network (3DCNN). The research
demonstrated the crucial role of facial dynamics in identity recognition and
the viability of neuromorphic sensors in detecting subtle facial movements. It
also presented NVSFD, a dataset tailored for speech-induced facial dynamics.

Despite these findings, this remains quite an unexplored field of research, as
to the best of our knowledge no other work on the subject exists yet. Similar
ideas, however, have been explored for event-based re-identification from full
bodies Ahmad et al. (2022).

### Face Pose Alignment

A challenging application in the context of human-robot interaction concerns
estimating face poses. This application can also serve as an important pre-
processing step for face analysis tasks, as it determines the orientation and
position of facial features. A few approaches to estimate face pose alignment
have been recently proposed.

Savran and Bartolozzi (2020) were the first to address this task with event
cameras. The alignment is achieved through a regression cascade of tree
ensembles. Efficiency is also taken into account, as the alignment is
initialized only when a pose change is detected to minimize unnecessary
processing. The authors also recorded a dataset with human subjects exhibiting
large head rotations, varying movement speeds, speaking intervals, and multi-
human annotations.

The same authors also extended this approach in Savran (2023c) and Savran
(2023a), by enhancing it with a multi-timescale event encoding strategy. In
Savran (2023c) multiple timescales are used to improve the efficiency and
quality of face pose alignment. This is achieved by adaptively adjusting the
processing rate based on facial movement intensity. The effect is that the
method generates sparse pose-events, reducing computational demands. Further
analysis was carried out in Savran (2023a), where two different timing
strategies for face pose alignment are used: constant time frame and constant
event count frame. This work addressed the problem of determining the
appropriate accumulation time intervals for the face alignment problem.

Face pose alignment has also been addressed by Ryan et al. (2023) for driver
monitoring system applications.

### LipReading & Voice Activity Detection

Given the capacity of event cameras to model high temporal resolution signals,
several works have leveraged neuromorphic strategies to analyze mouth-related
tasks, such as lip reading and Voice Activity Detection (VAD). VAD is a method
for identifying and isolating periods of speech within an audio stream;
differently, lip-reading is the process of understanding spoken language by
observing the movement of a person’s lips. In both tasks, the micro-movements
of the mouth must be captured to identify specific utterances.

In the exploration of these tasks, researchers have adopted diverse
approaches, some leveraging both audio and video modalities, while others
focus exclusively on video inputs.

A pioneering approach focusing on the task of Voice Activity Detection using
both audio and video was proposed in Savran et al. (2018). The pipeline starts
by jointly locating and detecting lip activity using a probabilistic
estimation technique after applying spatio-temporal filtering. VAD is then
performed, combining visual and audio features. In a similar vein, Savran
(2023b) proposed an event intensity-based approach by constructing a fully
convolutional network for effective neuromorphic VAD. The significance lies in
the development of a purely vision-based VAD model, breaking away from
traditional audio-centric approaches.

Also lip reading approaches have declined the task as a video or an audio-
video approach. Among the video-based approaches, Tan et al. (2022) proposed
MSTP, a Multi-grained Spatio-Temporal features Perceived network. The model
comprises two branches processing low-rate and high-rate event frames: on the
one hand, the low-rate branch accumulates event for a longer time-span, thus
capturing complete spatial features; on the other hand, the high-rate branch
focuses on fine-grained temporal features. The two branches are connected
throughout the network using message flow modules merging the features. The
authors also addressed the lack of event-based lip-reading datasets by
creating DVS-Lip, a dataset with 19,871 word samples.

The idea of separating neuromorphic information in independent streams at
different resolutions has been also followed in Yoo et al. (2023). The authors
propose RN-Net, an architecture specifically designed to process asynchronous
temporal data. It employs simple convolution layers seamlessly integrated with
dynamic temporal encoding reservoirs to effectively detect spatio-temporal
features at both local and global levels. Reservoirs are composed of a large
number of interconnected neurons, and they can be thought of as a kind of
”memory” for the network. Similarly, Bulzomi et al. (2023) and Kanamaru et al.
(2023) have explored the use of event-based cameras for lip reading.
Differently from prior work, Bulzomi et al. (2023) proposed a spiking neural
network architecture for this task, demonstrating an improvement in accuracy.
Kanamaru et al. (2023) instead leveraged a hybrid approach that combines
event-based and frame-based camera data and used a Temporal Convolutional
Network to recognize sounds. They also proposed an original dataset of 15
Japanese consonants from 20 speakers.

Differently from VAD, the idea of combining audio with event cameras in the
field of lip reading is more recent. Among these works, Yu et al. (2022)
introduced a spiking neural network approach for audio-visual speech
recognition based on lip reading. Their innovative use of liquid state
machines and a soft fusion method inspired by the attention mechanism
showcases the effectiveness of a combined audio-visual approach also for the
lip reading task.

Interestingly, the neuromorphic paradigm has been adopted for both the video
and audio modalities. A few works have in fact used neuromorphic audio sensors
along with neuromorphic cameras Rios-Navarro et al. (2023); Li et al. (2019).
Li et al. (2019) presented a multi-modal fusion deep network for event-based
lip reading using spiking sensors, combining Dynamic Video Sensors (DVS) and
Dynamic Audio Sensors (DAS). The fusion model enhances lip reading performance
by integrating visual and auditory information, reflecting the multimodal
nature of the approach. To address the shortage of labeled data, Rios-Navarro
et al. (2023) introduced the LIPSFUS dataset, which comprises both visual and
auditory information. The visual information consists of the lip movement when
a subject articulates a word, while the auditory information refers to the
sound that a subject makes when pronouncing a word. The data has been captured
by a Neuromorphic Auditory Sensor (NAS) and a DVS camera and is synchronized
with the same timing source.

### Facial Expression & Emotion Recognition

One of the most challenging, yet suitable for event-based approaches, is the
analysis of facial expressions and their underlying emotions. The difficulty
of these tasks is due to the extremely high temporal resolution at which
facial micro-movements happen to express an emotion. This poses a natural
limit for standard RGB approaches, which makes the usage of neuromorphic
vision sensors an interesting solution.

In this context, Becattini et al. Becattini et al. (2022) proposed a
pioneering method for modeling expressions with event cameras. The method
employs an event-based convolutional neural network that analyzes temporal
patterns of brightness changes to identify micro-expressions. The study
demonstrated that, by using event cameras, it is possible to understand human
reactions solely by observing facial expressions. A comparison with RGB-based
modeling showed the improved effectiveness of the approach. However, this work
only detected positive, neutral or negative reactions. This idea was then
extended in Berlincioni et al. (2023), where a dataset for Neuromorphic Event-
based Facial Expression Recognition (NEFER) was introduced, containing
annotations for the 7 basic emotions. The dataset consists of paired visual
spectrum images and event camera streams. For each sequence of frames, both
the expected emotion and the self-reported emotion from the user are provided,
allowing for a holistic understanding of emotional expression. They also
proposed a baseline method using a 3D convolutional network for emotion
recognition.

A more fine-grained analysis was recently proposed in Guo and Huang (2023).
The authors developed an approach to micro-expression recognition called
GLEFFN. Two main components are present: an event-based feature extraction
module and a global-local feature fusion network. The event-based feature
extraction module extracts a local count image from an up-sampled video using
SloMo Jiang et al. (2018). The local count image is a binary image where each
pixel is either on (indicating an event) or off (indicating no event). The
global-local feature fusion network combines the local count image with global
dense optical flow to obtain deeper features. By fusing the local and global
features, the network can capture both local and global motion information,
which is crucial for accurate micro-expression recognition. The problem of
facial expression recognition has also been declined as a valence-arousal
estimation problem in Berlincioni et al. (2024).

### Gaze Analysis and AR/VR

Another intriguing application of event cameras is gaze and pupil detection
and tracking. Event cameras are well-suited for this task due to their high
temporal resolution, which allows them to accurately track the movements of
the eye. This application is closely related to augmented and virtual reality
scenarios, as gaze is a natural way of interacting with head-mounted devices.

The challenge behind gaze analysis lies in the fact that eye saccades can
happen abruptly and very quickly. Angelopoulos et al. (2020) presented a
hybrid frame-event near-eye gaze tracking system that delivers high update
rates up to 10,000 Hz. Their system maintained an accuracy comparable to high-
end desktop-mounted commercial trackers. The system utilizes an event camera
that captures both high-frequency events and low-frequency frames to provide a
more efficient and accurate representation of eye motion. The authors
introduced a model-based eye tracking algorithm that operates at the event
rate and an online 2D pupil fitting method that updates a parametric model
every one or few events. The system also employed a polynomial regressor to
estimate the point of gaze from the parametric pupil model in real-time.

Differently, Banerjee et al. (2022) presented an event-based gaze detection
system using retinomorphic events recorded on a DVS camera. The authors
developed a new and compact event-based dataset for gaze detection under
various lighting conditions. They also proposed a novel event encoding
technique that encodes event logs into six-channel images and then used a
Convolutional Neural Network for gaze prediction. The authors evaluated their
method with multiple metrics and found that it achieved high accuracy in gaze
prediction. Finally, Kang et al. (2023) developed a remote pupil-tracking
technique using event cameras for Autostereoscopic 3D displays and AR 3D Head-
up displays (HUDs).

### Driver Monitoring System

Event cameras have proven to be attractive also for developing Driver
Monitoring Systems (DMS). A DMS is a tool that is used to improve safety while
driving as it can detect fatigue, distractions, or altered states of the
driver. This application shares some traits with all the aforementioned
applications since fatigue and distraction indicators can be tied to facial
expressions and movements. Basic modules such as face detectors, landmark
estimators or pupil trackers find a natural applicability in the development
of such applications.

A driving monitoring system can address different issues. The most basic
problem is to localize the driver in the frame. Liu et al. (2022) proposed a
driver face detection framework called NeuroDFD that uses event-based data.
The proposed method involves constructing event representation, incorporating
a shift feature pyramid network and shift context modules that process
temporal information at different scales. This approach also yielded 5 facial
landmarks and posed itself as a preliminary step for developing more complex
monitoring systems. Similarly, Ryan et al. (2023) and Ryan et al. (2021) also
detected the faces of drivers. Ryan et al. (2023) developed a face detection
module jointly estimating head pose, eye gaze and facial occlusions in real-
time. The framework is trained on synthetic event-stream data from
conventional video datasets and validated on real event camera data. Ryan et
al. (2021) instead predicted eye blinks and pupil detections in addition to
facial bounding boxes.

A different take on the problem has been proposed by Kielty et al. (2023) and
Chen et al. (2020a), who proposed to directly infer the drowsiness state of
the driver. Kielty et al. (2023) use event-based data to analyze mouth
movements in search of yawning behaviors that provide a complementary
indicator of tiredness. They also proposed a dataset of 952 video clips
recorded with an RGB camera and converted into neuromorphic images using a
video-to-event converter Hu et al. (2021). Instead of only detecting yawns,
Chen et al. (2020a) proposed to also recognize eye blinks and mouth movements.
Additionally, they provided the EDDD dataset, dedicated to event-based
drowsiness driving detection. Finally, Shariff et al. (2023) analyzed faces of
drivers to estimate distraction. The authors developed a sparse-ResNet, that
extracts features efficiently from event data and classifies the driver’s
distraction. Furthermore, two synthetic event datasets, Drive&Act and DMD,
were created to train and evaluate the proposed model. Interestingly, the same
task has also been addressed by looking at both faces and bodies in Yang et
al. (2022). The paper proposed a real-time driver distraction and action
recognition. The performance of the proposed system was evaluated on a large-
scale simulated event dataset and a self-recorded real event dataset with a
DAVIS346 event camera. Additionally, the authors implemented transfer learning
experiments on real event data, demonstrating promising generalization
capabilities.

Dataset | Video | Users | Resolution | Public | Annotations
---|---|---|---|---|---
Savran and Bartolozzi (2020) | 108 | 30 | 304$\times$204 | X | Eye; Lip
Lenz et al. (2020) | 48 | 10 | 640$\times$480 | ✓ | Eye blink
Becattini et al. (2022) | 455 | 25 | 640$\times$480 | X | Binary Reactions
Berlincioni et al. (2023) | 609 | 29 | 1280$\times$720 | ✓ | Face; Landmk.; Emotion
Bissarinova et al. (2023) | 3889 | 73 | 408$\times$360 | ✓ | Face; Landmk.
Tan et al. (2022) | 200 | 40 | 346$\times$260 | ✓ | Lip; Utterances
Chen et al. (2020a) | 260 | 26 | 346$\times$260 | ✓ | Voice; Eye blink; Yawn
Banerjee et al. (2022) | 3360 | 6 | 1920$\times$1080 | ✓ | Pupil Coordinates
Angelopoulos et al. (2020) | 24 | 24 | 346$\times$260 | ✓ | Gaze
Rios-Navarro et al. (2023) | 22 | 110 | 128$\times$128 | ✓ | Utterances
Barua et al. (2016) | - | 30 | 128$\times$128 | X | Face
Moreira et al. (2022) | 436 | 40 | - | ✓ | Identity
Kanamaru et al. (2023) | 1500 | 20 | 1280$\times$800 | X | Face; Landmk.; Lip
Ryan et al. (2023) | - | 5 | 1280$\times$720 | X | Head pose
Chen et al. (2020b) | 180 | 45 | 346$\times$260 | ✓ | Eye blink
Savran et al. (2018) | 360 | 18 | 304 $\times$ 240 | X | Face; Lip; Voice

Table 2: Several dataset focused on human faces expression and detection
annotated with their respective attributes.

##  4 Datasets

At the time of writing, there are no well-established benchmarks and the lack
of event camera data poses a challenge to the development of new face analysis
models. However, several works that analyze faces with neuromorphic cameras
collected footage for carrying out their experiments. Tab. 2 provides an
overview of the datasets that have been collected in the past years for
neuromorphic face analysis. We would like to stress the fact that most of
these datasets contain a small amount of samples with a lack of diversity. At
the same time, the spatial resolution of these datasets is limited due to
hardware limitations, since only in the last few years event cameras have been
equipped with Full HD sensors (1920 $\times$ 1080). We report in Tab. 2 also
the annotations that are made available along with the recorded videos. These
annotations include low-level labels such as bounding boxes of faces (Face),
lip regions (Lip), eye regions (Eye) and coordinates of specific facial
keypoints (Landmk.). Some datasets also include higher-level annotations such
as positive/negative facial reactions to visual stimuli (Binary reactions),
words pronounced by the subjects (Utterances) and whether a subject is
speaking (Voice).

It is worth noticing that annotating events is not straightforward. First,
instead of annotating frames, the labels must be related to a set of spatio-
temporal events. Then, since an event camera does not produce any signal in
the absence of motion, identifying the entities to annotate is harder. For
instance, one may want to annotate a static face, even when it is not visible.
In addition, events are often accumulated to be processed with a frame-based
model. The accumulation time and the encoding strategy directly affect the
quality or the granularity of the annotations.

### Synthethic Approaches

It must be noticed that all dataset in Tab. 2 are made of real event data,
i.e. recorded with an actual neuromorphic sensor. However, a large crop of
literature Guo and Huang (2023); Kielty et al. (2023); Ryan et al. (2021,
2023); Savran and Bartolozzi (2020); Yu et al. (2022); Berlincioni et al.
(2024) works with synthetic events, obtained by converting RGB videos into
neuromorphic streams. Also some of the methods in Tab. 2 Barua et al. (2016);
Becattini et al. (2022); Chen et al. (2020a); Shariff et al. (2023) leverage
synthetic events to perform additional experiments or train on more data. Some
works, such as Becattini et al. (2022) also convert RGB videos to event
streams to trivially annotate events by transferring any available label
attached to the original data.

Different event camera simulators have been proposed in the literature,
namely, ESIM Rebecq et al. (2018) and V2E Hu et al. (2021). These simulators
are capable of producing neuromorphic counterparts from RGB videos. To this
end, they first perform a temporal upsampling of RGB frames, with a rate that
adapts to the video content and its estimated visual dynamics (the more the
video changes, the more frames are added). Then, synthetic events are
generated by analyzing the differences between adjacent frames.

While these tools have proven to be beneficial for many tasks they also
present drawbacks. As discussed in Berlincioni et al. (2024), from which we
report also here Fig. 2, the RGB to Event simulators are sensible to the
compression of the source video, to the point that the block coding artifacts,
barely visible in the original footage, are exaggerated and turned into block-
sized macro-events. This hinders the applicability of simulators and
underlines the domain shift between real and simulated events.

Figure 2: Compression artifacts after simulation using Hu et al. (2021). Image
courtesy of Berlincioni et al. (2024)

##  5 Conclusions and Future Developments

Neuromorphic Face Analysis is a field of research at a very early stage.
Nonetheless, several works have underlined the effectiveness of neuromorphic
cameras for capturing facial features, offering several benefits compared to
traditional computer vision. First of all the extremely low latency allows to
capture facial micro-movements which might not be fully observable with RGB
cameras. The high dynamic range of the sensors is an aid for stable driver
monitoring systems as well as person identification modules under challenging
conditions. In this paper, we have discussed present research on the subject,
pointing out that there are still many challenges to be addressed. The lack of
annotated data poses a big issue: no standard benchmark exists and new
datasets must be collected even for addressing tasks that are close to being
solved in RGB vision, such as face detection. Many works have bypassed this
problem, relying on synthetic data obtained with a simulator. However, whereas
this could be a suitable solution for many problems, we believe that real
events should be leveraged to exploit the full capacity of neuromorphic
sensors for analyzing faces as micro-movements are lost when converting low-
framerate RGB videos. In particular, we found the field of expression and
emotion recognition to be almost unexplored. This comes as a surprise as we
believe that observing the extremely fast movements that convey emotions is a
perfect application for a sensor with such a low latency as an event camera.
Studying emotions would also allow the development of applications in
healthcare and human-computer interaction. For instance, in the healthcare
field, the development of neuromorphic medical devices could monitor the
patient’s pain expression.

Neuromorphic sensors could also play a crucial role in advancing augmented
reality technologies. Whereas a few works in this direction are present in the
literature, these sensors could enable more immersive and responsive AR
experiences, enhancing how users interact with digital information overlaid on
the real world. The energy-efficient nature of neuromorphic sensors makes them
also well-suited for edge computing applications. Future developments may see
the integration of these sensors into small, resource-constrained devices,
contributing to the growth of decentralized and efficient computing.

In conclusion, the ongoing research and innovations in this area are likely to
shape the next generation of intelligent systems and contribute to the
continued advancement of artificial intelligence.

Acknowledgments This work was supported by the European Commission under
European Horizon 2020 Programme, grant number 951911—AI4Media. This work was
partially supported by the Piano per lo Sviluppo della Ricerca (PSR 2023) of
the University of Siena - project FEATHER: Forecasting and Estimation of
Actions and Trajectories for Human-robot intERactions.

## References

  * Ahmad et al. [2022] Shafiq Ahmad, Gianluca Scarpellini, Pietro Morerio, and Alessio Del Bue.  Event-driven re-id: A new benchmark and method towards privacy-preserving person re-identification.  In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 459–468, 2022.
  * Al-obaidi [2020] Salah Al-obaidi.  Privacy aware human action recognition: an exploration of temporal salience modelling and neuromorphic vision sensing.  PhD thesis, University of Sheffield, 2020.
  * Angelopoulos et al. [2020] Anastasios N Angelopoulos, Julien NP Martel, Amit PS Kohli, Jorg Conradt, and Gordon Wetzstein.  Event based, near eye gaze tracking beyond 10,000 hz.  arXiv preprint arXiv:2004.03577, 2020.
  * Banerjee et al. [2022] Abeer Banerjee, Shyam Sunder Prasad, Naval Kishore Mehta, Himanshu Kumar, Sumeet Saurav, and Sanjay Singh.  Gaze detection using encoded retinomorphic events.  In International Conference on Intelligent Human Computer Interaction, pages 442–453. Springer, 2022.
  * Barua et al. [2016] Souptik Barua, Yoshitaka Miyatani, and Ashok Veeraraghavan.  Direct face detection and video reconstruction from event cameras.  In 2016 IEEE winter conference on applications of computer vision (WACV), pages 1–9. IEEE, 2016.
  * Becattini et al. [2022] Federico Becattini, Federico Palai, and Alberto Del Bimbo.  Understanding human reactions looking at facial microexpressions with an event camera.  IEEE Transactions on Industrial Informatics, 18(12):9112–9121, 2022.
  * Ben et al. [2021] Xianye Ben, Yi Ren, Junping Zhang, Su-Jing Wang, Kidiyo Kpalma, Weixiao Meng, and Yong-Jin Liu.  Video-based facial micro-expression analysis: A survey of datasets, features and algorithms.  IEEE transactions on pattern analysis and machine intelligence, 44(9):5826–5846, 2021.
  * Berlincioni et al. [2023] Lorenzo Berlincioni, Luca Cultrera, Chiara Albisani, Lisa Cresti, Andrea Leonardo, Sara Picchioni, Federico Becattini, and Alberto Del Bimbo.  Neuromorphic event-based facial expression recognition.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4108–4118, 2023.
  * Berlincioni et al. [2024] Lorenzo Berlincioni, Luca Cultrera, Federico Becattini, and Alberto Del Bimbo.  Neuromorphic valence and arousal estimation, 2024.
  * Bissarinova et al. [2023] Ulzhan Bissarinova, Tomiris Rakhimzhanova, Daulet Kenzhebalin, and Huseyin Atakan Varol.  Faces in event streams (fes): An annotated face dataset for event cameras.  Authorea Preprints, 2023.
  * Bulat and Tzimiropoulos [2017] Adrian Bulat and Georgios Tzimiropoulos.  How far are we from solving the 2d & 3d face alignment problem? (and a dataset of 230,000 3d facial landmarks).  In International Conference on Computer Vision, 2017.
  * Bulzomi et al. [2023] Hugo Bulzomi, Marcel Schweiker, Amélie Gruel, and Jean Martinet.  End-to-end neuromorphic lip-reading.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4100–4107, 2023.
  * Chen et al. [2020a] Guang Chen, Lin Hong, Jinhu Dong, Peigen Liu, Jörg Conradt, and Alois Knoll.  Eddd: Event-based drowsiness driving detection through facial motion analysis with neuromorphic vision sensor.  IEEE Sensors Journal, 20(11):6170–6181, 2020.
  * Chen et al. [2020b] Guang Chen, Fa Wang, Xiaoding Yuan, Zhijun Li, Zichen Liang, and Alois Knoll.  Neurobiometric: an eye blink based biometric authentication system using an event-based neuromorphic vision sensor.  IEEE/CAA Journal of Automatica Sinica, 8(1):206–218, 2020.
  * Delilovic and Salaj [2021] Namik Delilovic and Darjan Salaj.  Bio-inspired neuromorphic ai methods enables privacy respecting security and surveillance.  Transactions on Advanced Research, 2021.
  * Dong et al. [2023] Yiting Dong, Yang Li, Dongcheng Zhao, Guobin Shen, and Yi Zeng.  Bullying10k: A large-scale neuromorphic dataset towards privacy-preserving bullying recognition.  In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023.
  * Ekman and Friesen [1978] Paul Ekman and Wallace V Friesen.  Facial action coding system.  Environmental Psychology & Nonverbal Behavior, 1978.
  * Guo and Huang [2023] Cunhan Guo and Heyan Huang.  Gleffn: A global-local event feature fusion network for micro-expression recognition.  In Proceedings of the 3rd Workshop on Facial Micro-Expression: Advanced Techniques for Multi-Modal Facial Expression Analysis, pages 17–24, 2023.
  * Han et al. [2023] Bing Han, Qiang Fu, and Xinliang Zhang.  Towards privacy-preserving federated neuromorphic learning via spiking neuron models.  Electronics, 12(18):3984, 2023.
  * Hu et al. [2021] Y Hu, S C Liu, and T Delbruck.  v2e: From video frames to realistic DVS events.  In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). IEEE, 2021.
  * Innocenti et al. [2021] Simone Undri Innocenti, Federico Becattini, Federico Pernici, and Alberto Del Bimbo.  Temporal binary representation for event-based action recognition.  In 2020 25th International Conference on Pattern Recognition (ICPR), pages 10426–10432. IEEE, 2021.
  * Jiang et al. [2018] Huaizu Jiang, Deqing Sun, Varun Jampani, Ming-Hsuan Yang, Erik Learned-Miller, and Jan Kautz.  Super slomo: High quality estimation of multiple intermediate frames for video interpolation.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 9000–9008, 2018.
  * Kanamaru et al. [2023] Tatsuya Kanamaru, Taiki Arakane, and Takeshi Saitoh.  Isolated single sound lip-reading using a frame-based camera and event-based camera.  Frontiers in Artificial Intelligence, 5:1070964, 2023.
  * Kang et al. [2023] Dongwoo Kang, Youn Kyu Lee, and Jongwook Jeong.  Exploring the potential of event camera imaging for advancing remote pupil-tracking techniques.  Applied Sciences, 13(18):10357, 2023.
  * Kielty et al. [2023] Paul Kielty, Mehdi Sefidgar Dilmaghani, Cian Ryan, Joe Lemley, and Peter Corcoran.  Neuromorphic sensing for yawn detection in driver drowsiness.  In Fifteenth International Conference on Machine Vision (ICMV 2022), volume 12701, pages 287–294. SPIE, 2023.
  * King [2009] Davis E. King.  Dlib-ml: A machine learning toolkit.  Journal of Machine Learning Research, 10:1755–1758, 2009.
  * Lenz et al. [2020] Gregor Lenz, Sio-Hoi Ieng, and Ryad Benosman.  Event-based face detection and tracking using the dynamics of eye blinks.  Frontiers in Neuroscience, 14:587, 2020.
  * Li et al. [2019] Xiaoya Li, Daniel Neil, Tobi Delbruck, and Shih-Chii Liu.  Lip reading deep network exploiting multi-modal spiking visual and auditory sensors.  In 2019 IEEE International Symposium on Circuits and Systems (ISCAS), pages 1–5. IEEE, 2019.
  * Liu et al. [2022] Peigen Liu, Guang Chen, Zhijun Li, Daniel Clarke, Zhengfa Liu, Ruiqi Zhang, and Alois Knoll.  Neurodfd: Towards efficient driver face detection with neuromorphic vision sensor.  In 2022 International Conference on Advanced Robotics and Mechatronics (ICARM), pages 268–273. IEEE, 2022.
  * Moreira et al. [2022] Gonçalo Moreira, André Graça, Bruno Silva, Pedro Martins, and Jorge Batista.  Neuromorphic event-based face identity recognition.  In 2022 26th International Conference on Pattern Recognition (ICPR), pages 922–929. IEEE, 2022.
  * Mueggler et al. [2017] Elias Mueggler, Chiara Bartolozzi, and Davide Scaramuzza.  Fast event-based corner detection.  2017\.
  * Rebecq et al. [2018] Henri Rebecq, Daniel Gehrig, and Davide Scaramuzza.  ESIM: an open event camera simulator.  Conf. on Robotics Learning (CoRL), October 2018.
  * Rebecq et al. [2019] Henri Rebecq, René Ranftl, Vladlen Koltun, and Davide Scaramuzza.  Events-to-video: Bringing modern computer vision to event cameras.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3857–3866, 2019.
  * Rios-Navarro et al. [2023] Antonio Rios-Navarro, Enrique Piñero-Fuentes, Salvador Canas-Moreno, Aqib Javed, Jin Harkin, and Alejandro Linares-Barranco.  Lipsfus: A neuromorphic dataset for audio-visual sensory fusion of lip reading.  arXiv preprint arXiv:2304.01080, 2023.
  * Ryan et al. [2021] Cian Ryan, Brian O’Sullivan, Amr Elrasad, Aisling Cahill, Joe Lemley, Paul Kielty, Christoph Posch, and Etienne Perot.  Real-time face & eye tracking and blink detection using event cameras.  Neural Networks, 141:87–97, 2021.
  * Ryan et al. [2023] Cian Ryan, Amr Elrasad, Waseem Shariff, Joe Lemley, Paul Kielty, Patrick Hurney, and Peter Corcoran.  Real-time multi-task facial analytics with event cameras.  IEEE Access, 2023.
  * Savran and Bartolozzi [2020] Arman Savran and Chiara Bartolozzi.  Face pose alignment with event cameras.  Sensors, 20(24):7079, 2020.
  * Savran et al. [2018] Arman Savran, Raffaele Tavarone, Bertrand Higy, Leonardo Badino, and Chiara Bartolozzi.  Energy and computation efficient audio-visual voice activity detection driven by event-cameras.  In 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), pages 333–340. IEEE, 2018.
  * Savran [2023a] Arman Savran.  Comparison of timing strategies for face pose alignment with event camera.  In 2023 8th International Conference on Computer Science and Engineering (UBMK), pages 97–101. IEEE, 2023.
  * Savran [2023b] Arman Savran.  Fully convolutional event-camera voice activity detection based on event intensity.  In 2023 Innovations in Intelligent Systems and Applications Conference (ASYU), pages 1–6. IEEE, 2023.
  * Savran [2023c] Arman Savran.  Multi-timescale boosting for efficient and improved event camera face pose alignment.  Computer Vision and Image Understanding, 236:103817, 2023.
  * Shariff et al. [2023] Waseem Shariff, Mehdi Sefidgar Dilmaghani, Paul Kielty, Joe Lemley, Muhammad Ali Farooq, Faisal Khan, and Peter Corcoran.  Neuromorphic driver monitoring systems: A computationally efficient proof-of-concept for driver distraction detection.  IEEE Open Journal of Vehicular Technology, 2023.
  * Tan et al. [2022] Ganchao Tan, Yang Wang, Han Han, Yang Cao, Feng Wu, and Zheng-Jun Zha.  Multi-grained spatio-temporal features perceived network for event-based lip-reading.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20094–20103, 2022.
  * Yan et al. [2013] Wen-Jing Yan, Qi Wu, Jing Liang, Yu-Hsin Chen, and Xiaolan Fu.  How fast are the leaked facial expressions: The duration of micro-expressions.  Journal of Nonverbal Behavior, 37:217–230, 2013.
  * Yang et al. [2022] Chu Yang, Peigen Liu, Guang Chen, Zhengfa Liu, Ya Wu, and Alois Knoll.  Event-based driver distraction detection and action recognition.  In 2022 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI), pages 1–7. IEEE, 2022.
  * Yoo et al. [2023] Sangmin Yoo, Eric Yeu-Jer Lee, Ziyu Wang, Xinxin Wang, and Wei D Lu.  Rn-net: Reservoir nodes-enabled neuromorphic vision sensing network.  arXiv e-prints, pages arXiv–2303, 2023.
  * Yu et al. [2022] Xuhu Yu, Lei Wang, Changhao Chen, Junbo Tie, and Shasha Guo.  Multimodal learning of audio-visual speech recognition with liquid state machine.  In International Conference on Neural Information Processing, pages 552–563. Springer, 2022.
  * Zhang et al. [2023] Pei Zhang, Shuo Zhu, and Edmund Y Lam.  Event encryption: Rethinking privacy exposure for neuromorphic imaging.  arXiv preprint arXiv:2306.03369, 2023.


# High-level Features for Resource Economy and Fast Learning in Skill Transfer

Alper Ahmetoglua∗ ∗Corresponding author. Email<EMAIL_ADDRESS>Emre Ugura Minoru Asadab Erhan Oztopb,c
aDepartment of Computer Engineering, Bogazici University, Turkey
bOTRI/SISREC, Osaka University, Osaka, Japan
cDepartment of Computer Science, Ozyegin University, Turkey

###### Abstract

Abstraction is an important aspect of intelligence which enables agents to
construct robust representations for effective and efficient decision making.
In the last decade, deep neural networks are proven to be effective learning
systems that are applicable to a wide range of application domains, in
particular, due to their ability to form increasingly complex abstractions at
successive layers in an end-to-end fashion. However, these abstractions,
unless enforced, are mostly distributed over many neurons, making the re-use
of a learned skill costly and blind to the insights that can be obtained on
the emergent representations. Previous work either enforced formation of
abstractions creating a designer bias, or used a large population of neural
units without investigating how to obtain high-level low-dimensional features
that may more effectively capture the source task. For avoiding designer bias
and unsparing resource use, we propose to exploit neural response dynamics to
form compact representations to use in skill transfer. For this, we consider
two competing methods based on (1) information loss/maximum information
compression principle and (2) the notion that abstract events tend to generate
slowly changing signals, and apply them to the neural signals generated during
task execution. To be concrete, in our simulation experiments, we either apply
principal component analysis (PCA) or slow feature analysis (SFA) on the
signals collected from the last hidden layer of a deep neural network while it
performs a source task, and use these features for skill transfer in a new,
target, task. We then compare the generalization and learning performance of
these alternatives with the baselines of skill transfer with full layer output
and no-transfer settings. Our experimental results on a simulated tabletop
robot arm navigation task show that units that are created with SFA are the
most successful for skill transfer. SFA as well as PCA, incur less resources
compared to usual skill transfer where full layer outputs are used in the new
task learning, whereby many units formed show a localized response reflecting
end-effector-obstacle-goal relations. Finally, SFA units with lowest
eigenvalues resembles symbolic representations that highly correlate with
high-level features such as joint angles and end-effector position which might
be thought of precursors for fully symbolic systems.

###### keywords:

transfer learning, symbol emergence, reinforcement learning

##  1 Introduction

It can be argued that intelligent behavior, both artificial and biological
agents, require the use of abstractions for high competency in the real world
[1]. Abstraction allows an agent to filter out irrelevant aspects of incoming
stimuli, and thus, equips it with a robust representation. From a biological
perspective, it is not clear how concept and symbol representations are formed
by the central nervous system [2]. Yet, behavioral markers of such
representations can be seen even in animals at the lower levels of cognitive
hierarchy. For example, toads have abstract prey and predator feature
detectors that respond according to the high level features present in the
visual stimuli related to the size and shape relative to the direction of
motion [3]. From the robotics point of view, classically, a set of concepts or
symbols can be defined apriori and later grounded to the sensorimotor stimuli
of the agent acting in its environment [4]. More recent approaches aim to
abstract representations directly from the sensorimotor experience of the
agent [5, 6, 7, 8].

In fact, the deep learning approaches form complex feature representations by
combining simpler ones in successive layers [9]. As deep neural networks have
been very successful in many real world problems such as image recognition
[10], machine translation [11], playing the game of Go [12], and protein
folding problem [13], most of the focus has been put on improving performance
and finding newer applications of deep learning. One of the bottlenecks of
learning in deep networks is the large amount of data required for learning
tasks from scratch. To attack this problem, knowledge or skill transfer
methods allowing the reuse of learned representations have started to appear
[14, 15], although similar ideas were present earlier in the robot learning
domain [16]. One straightforward way to achieve knowledge transfer is to
create a new network with access to all the layers of original network that
was used to learned the initial task (e.g. [17, 18]). This ensures that any
representation discovered will have a chance of exploitation during new task
learning. However, in these approaches all the units in the layers are linked
to the new network through adaptable weights, and thus, which of those units
need to be exploited for the new task is left for the learning algorithm to
find. Considering resource economy both in terms of memory and energy, this
seems wasteful. In this sense, a principled task-agnostic way of selecting
which units to use for the next task learning is highly desirable. This study
aims to make a step forward in this direction.

Earlier robotic work indicates that units that capture high level features are
potential candidates to facilitate effective knowledge transfer [19]. From a
neural computation point of view, this also makes sense as compact high-level
representations would be more economical to process and pass around in the
neural circuits of biological systems. Thus, the brain might adopt a strategy
to represent sensorimotor information compactly with minimum information loss,
inline with the information compression ideas [20, 21]. Another hypothesis can
be obtained by generalizing the idea that slowly changing features in sensory
data tend to correspond to more high-level concepts [22] to neural responses.
According to this view, the brain might seek to exploit those neurons that are
more stable over the others that show frequent changes. For example, when a
hand waving action is observed, the earlier sites in the visual processing
pathway (e.g. areas V1, V2) would show temporally changing activity where as
at the end of the processing pipeline (i.e. in area IT), the recognition of
the hand waving action would be represented with a few neurons, which would
show stable activity during the most of the observation period over a variety
hand waving actions.

Motivated from these hypotheses, in this paper, we analyze different ways of
transferring previously learned representations for a new task in an
economical way. More specifically, we consider two methods: (1) principal
component analysis (PCA) for the minimum information loss, and (2) slow
feature analysis (SFA) [22] for creating signals that change slowly. PCA
reduces the dimensionality of the data while preserving the information
maximally which is a suitable candidate for transferring compact features. On
the other hand, SFA, which is not very well explored in the transfer learning
context, creates slowly changing representations which are arguably more
robust because objects of interest in the real world do not make abrupt
changes considering the notion of object permanency. In our experiments, we
first train a deep Q-network [23] on a task, where the goal is to move the
robot arm to a desired target position in the presence of a rectangular
obstacle which may appear at different locations. After training, we create
separate skill transfer scenarios in which either PCA or SFA transformations
are applied on the last hidden layer activations of the network and used in
new task learning by augmenting the features found to the last hidden layer of
the new network. We compare these scenarios with the baselines of skill
transfer with full layer output and no-transfer scenarios. Our experimental
results show that:

  * •

Using features that are constructed with SFA is not only more economical in
terms of the number of units, but also better for skill transfer than the
naive approach of using the full set of layer activations.

  * •

PCA is also helpful for resource economy in skill transfer, but not as good as
SFA in terms of success rate of the new task.

  * •

SFA and PCA capture interpretable high-level features such as joint angles,
tip locations, and the distance from the tip position to the goal position,
solely from the activation history of the network.

In the rest of this paper, we detail our methodology in Section 2, define the
experimental setup in Section 3, give the results in Section 4, and conclude
in Section 5.

###  1.1 Related Work

There has been an increased attention on learning high-level abstractions from
the experience of the robot. Konidaris et al. proved that learning symbols for
the pre- and post-conditions of executed actions are necessary and sufficient
for determining the feasability of a plan [6]. Based on this, they constructed
symbols for pre-conditions and effects of executed actions in a 2d game
environment. In a follow-up work, they moved this framework into the
probabilistic setting and used the learned symbols in a real-world robot to
make plans [24]. Ugur and Piater followed an object-centric strategy and
created symbols from clusters of object features and effects for each action
[19]. A more recent follow-up work uses deep neural networks with a binary
bottleneck layer to predict the effect of executed actions in an end-to-end
fashion [8]. Activations in the bottleneck layer are then treated as symbols
and used for planning. A similar work also employs deep neural networks with a
binary layer but symbols are learned independently from the executed actions
and observed effects. All of these works force symbols to be discrete and this
effectively limits the capability of the system. A more desirable property
would be to allow the system to use continuous representations and observe
discrete symbols only as a by-product of some learning process. To this end,
in this paper, we explore different methods of constructing features in a
generic, task-agnostic way to investigate whether they exhibit a symbol-like
meaning.

PCA is a frequently used method for dimensionality reduction and data analysis
due to its ability to create features that capture the data with a desired
level of information loss. In particular, it allows the tuning of the amount
of information loss by choosing the principal vectors to represent data by
simply examining the eigenvalues corresponding to them. Santello et al. used
PCA to analyze the intrinsic dimensionality of hand posture for grasping
different objects and found that the first two dimensions cover more than 80%
of the variance [25]. Tripathi and Wagatsuma applied PCA to different joint
sets and time segments to impose a prior on the PCA algorithm based on the
control approach of the central nervous system [26]. Promsri and Federolf
analyzed the effect of using PCA on the acceleration of postural changes as
opposed using PCA only on the amplitude [27]. As a non-linear approach, Chen
et al. combined sparse autoencoders with dynamic movement primitives [28] for
50-dimensional human movement data, and showed that new data can be generated
using only a single neuron of the trained system [29].

SFA [22] can be loosely thought as the application of PCA to the first time-
derivative of the data. Therefore, it is particularly useful for extracting
slowly changing signals when the low eigenvalue vectors are chosen. Dawood and
Loo used an incremental version of SFA [30] for action segmentation from high-
dimensional video input. A similar work to ours studied the use of
hierarchical SFA features extracted from raw image data in the context of
reinforcement learning [31]. Our study differs from this work as we are
interested in extracting high-level features from an already formed network,
be it either prewired or learned, and use them in subsequent tasks.

##  2 Methods

Suppose that an agent has developed a neural system to solve a specific task.
When the agent encounters a similar task, neurons in this system will respond
to the sensorimotor input albeit possibly in a different way since the input
will be different. Even though the system does not directly provide the
appropriate motor control output, it would be economically viable to use a
previously learned network instead of re-learning everything from scratch. We
focus on extracting features from this network in an economical way. Note that
it is not important how this network has been formed; it might have been
learned via stochastic gradient descent, pre-wired, or obtained through an
evolutionary algorithm.

Firstly, we train a deep Q-learning agent [23] on a primary task, so to have a
network which we can extract features from. In the primary task, a robot arm
tries to move its arm to the goal position while avoiding a rectangular
obstacle. After training, we extract features from this network in three
different ways and use them in a secondary task to assess the bootstrapping
effect induced for the new task learning. As a baseline we also learn the new
task from scratch without any transfer. So, overall, we have these cases:

  1. 1.

Learning from scratch (transfer:none). The new task is trained without any
transfer to form a baseline for the next three transfer scenarios.

  2. 2.

The last hidden layer activations (transfer:full). This is one of the most
frequently used methods for transferring visual features from networks that
are trained on large-scale data sets [32] and serves as a reference for the
next two methods.

  3. 3.

Transforming the last hidden layer with PCA (transfer:PCA). While the last
layer activations provide useful information, it is mostly distributed in many
units. PCA is especially useful for concentrating the distributed information
into fewer units, effectively reducing the number of neurons, thus reducing
the number of weights that needs to be optimized for learning the new task.
Let us denote our dataset as $X=\\{H_{i}\\}_{i=1}^{N}$ where $N$ is the number
of robot movement trajectories arising from executing the primary task of the
robot, and $H_{i}$ is an $T_{i}\times D$ matrix where $T_{i}$ is the number of
timesteps in $i$th trajectory and $D$ is the dimensionality of the last hidden
layer. We concatenate each $H_{i}$ from the first dimension so that $X$ is a
$(T_{1}+T_{2}+\dots+T_{N})\times D$ matrix. We want to find a projection
$z=Xw$ such that $\text{Var}(z)$ is maximized. This leads to the following
objective:

$\arg\max_{w}\quad\frac{w^{T}X^{T}Xw}{w^{T}w}$ (1)

which is the Rayleigh quotient, and the expression takes its maximum value
when $w$ is equal to the eigenvector with the largest eigenvalue of $X^{T}X$
[33].

  4. 4.

Transforming the last hidden layer with SFA (transfer:SFA). PCA puts an
emphasis on the maximum information preserving units. On the other hand, SFA
tries to minimize the time-derivative of the output signals. More
specifically, SFA creates output features $z=Xw$ with the following objective
and restrictions [22]:

$\min\quad\mathbb{E}[\dot{z}_{i}^{2}]$ (2)

subject to

$\displaystyle\mathbb{E}[z_{i}]$ $\displaystyle=0$ (3)
$\displaystyle\mathbb{E}[z_{i}^{2}]$ $\displaystyle=1$ (4)
$\displaystyle\mathbb{E}[z_{i}z_{j}]$ $\displaystyle=0\quad\forall j<i$ (5)

Here, Equations 3 and 4 prevents the trivial solution of a constant feature
and also forces the output to be normalized. Equation 5 forces features to be
orthogonal to each other. The solution can be found by first whitening the
data, and then applying PCA to the time derivative of $X$. This method is
shown to be useful for extracting the independent factors of an input signal.

(a) transfer:full (b) transfer:sfa

Figure 1: Different transfer methods are depicted. transfer:pca is identical
to 1(b) except that the transformation is done with PCA, instead of SFA.

We transfer the information from a network with these three methods to a new
task by concatenating the features to the input of the last layer as shown if
Figure 1. In our experiments, we define the secondary task to be a
reinforcement learning setup as well. The environment is similar to the
primary task except that there is a different type of obstacle to avoid.

Note that both PCA and SFA defines an affine transformation of the last layer,
and thus, does not bring any advantage in terms of function complexity.
However, re-alignment of the feature space can greatly accelerate the speed of
convergence with stochastic gradient descent [34]. Likewise, we expect that
agents that use these transformations will exhibit a better performance with
fewer samples.

##  3 Experiment Setup

We perform our experiments on CoppeliaSim 4.2.0 simulator [35]. The experiment
setup consists of a UR10 [36] robot arm with a solid cylinder attached to its
end effector and a camera placed at 180cm over the table for a top-down visual
perception. The robot arm can make planar movements in 8 different uniform
directions in a $80$cm $\times$ $100$cm rectangular workspace. The camera
input provides a 64 $\times$ 64 pixels colored image at each timestep (Figure
2). There are rectangular shaped and L-shaped obstacles, a goal marker, and
distractor markers in the environment. The general overview of the setup is
shown in Figure 2.

Figure 2: Left: The experiment setup. The obstacles are red-colored, and the
goal marker is green-colored. Distractors dynamically change their colors in
the RGB range of [(0, 0, 0)-(255, 0, 255)]. The corresponding input from the
camera is shown on the top-right inset. Right: The regions of penalty around
obstacles are visualized in blue (see text).

In this environment, we define two different tasks, a primary task in which
the robot tries to move its end-effector to the goal position while avoiding a
rectangular obstacle, and a secondary task where there is an L-shaped obstacle
instead of a rectangular one to avoid. In both tasks, we included distractors
that change colors and move in random directions with a constant speed to
increase the difficulty of the tasks. The reward function for both
environments is defined as follows:

$\displaystyle R(t)=\begin{cases}\hfil
10\quad&\text{if}\quad\|x_{\text{tip}}(t)-x_{\text{goal}}\|_{2}<5\text{cm}\\\
\hfil 10\Delta_{t}(x_{\text{tip}},x_{\text{goal}})\quad&\text{else
if}\quad\|x_{\text{tip}}(t)-x_{\text{obstacle}}(t)\|_{2}>\tau\\\ \hfil
10(\Delta_{t}(x_{\text{tip}},x_{\text{goal}})+\text{relu}(\Delta_{t}(x_{\text{tip}},x_{\text{obstacle}})))\quad&\text{otherwise}\end{cases}$
(6)

where $\tau$ is a threshold for obstacle proximity penalty and $\Delta_{t}$
denotes the change in the distance between robot end-effector and the target
over consecutive time steps:

$\Delta_{t}(x,y)=\|x(t-1)-y(t-1)\|_{2}-\|x(t)-y(t)\|_{2}$ (7)

We set $\tau$ to 21cm for the primary environment and 28cm for the secondary
environment. The robot gets a penalty whenever it goes towards the obstacle
when it is in the blue region visualized in Figure 2.

We use a convolutional neural network (CNN) architecture, where three
convolutional layers followed by two fully-connected layers are employed in
each architecture. Convolutional layers have 16, 32, and 64 channels
successively. Each layer has a kernel size of $3\times 3$ with a stride of 2
and a padding of 1. Fully-connected layers have 512 and 9 number of units. We
concatenate the transferred features to the last hidden layer except the
learning from scratch case. For example, if we transfer 100 slow features, the
dimensionality of the last hidden layer becomes $512+100=612$.

We train each model as a deep Q-network [23] for 2000 episodes with a replay
buffer of size 50,000. Each episode lasts for at most 200 time-steps, after
then, the episode terminates. We used the reward function defined in Equation
6. The last fully-connected layer is used for the Q-value estimation. There
are 8 different actions for 8 different directions and an additional action
for no operation. For the optimization, we use Adam optimizer [37] with a
learning rate of 0.001 with no learning rate decay.

##  4 Results

In this section, we first test the generalization performance attained by the
transfer of features acquired with different methods to a new task in 4.1.
Next, we analyze correlations between the proposed features and high-level
task related features in Section 4.2. Lastly, in Section 4.3, we visualize the
responses of neurons for varying inputs to get an insight about their
functionalities.

###  4.1 Transfer Performance

The aim of this experiment is to test the bootstrapping effect of using a
previously learned representation on learning a similar task from scratch. To
this end, we concatenate the activations of a previously trained network
(described in the previous section) to the last hidden layer. This method is
one of the basic transfer learning methods [32]. We compare the transfer of
plain activations, PCA features, and SFA features. In addition, we train a
network with no transfer to observe the bootstrapping effect.

Each model is trained with 20, 50, 100, and 200 number of training
configurations for 2000 episodes. Here, a training configuration refers to an
environment setting (i.e. initial position of the objects). Each episode is
initialized with a configuration sampled from this set. After convergence, we
test each model at each 500 episodes on the previously unseen environment
settings (i.e. configurations) to observe the generalization performance. For
testing, we collect 100 runs with each model and calculate the average
percentage of path covered towards the goal, and treat this estimate as one
test result. The average of 15 different test results at best performing
episodes (validated with 5 results) for each model are reported in Figure 3.

Figure 3: Generalization performance measured as distance covered towards the
goal vs. number of configurations experienced during learning in the new task.
Welch’s t-test [38] shows significant difference between transfer:SFA and
transfer:full for 100 and 200 configurations ($p=0.022$ and $p=0.041$,
respectively), and almost significant difference for 50 configurations
($p=0.082$).

We observe that using transfer:SFA condition gives better performance compared
to transfer:PCA, transfer:full and transfer:none conditions. Note that SFA and
PCA features are 100 dimensional vectors while plain activations are 512
dimensional vectors. This result shows that SFA indeed creates more condensed
features that are appropriate for skill transfer. The bootstrapping effect
fades away when we increase the number of training configurations to 200
configurations. This is an expected result as every model gets better when we
increase the variation in the training set. However, the usage of low training
configurations is desirable in many real world settings.

(a) 50 configs  (b) 100 configs  (c) 200 configs

Figure 4: Test performance vs. number of training episodes for different
methods and configuration numbers.

We also report the test results at different episodes in Figure 4. The figure
suggests that using an additional set of features from a previously learned
task helps generalization, especially when there are fewer number of
configurations. We see that transfer:sfa performs on par with transfer:none in
the initial stages of the training, then achieves the peak of its performance
very quickly compared to other methods. On the other hand, the contribution of
transfer:full steadily increases. This might be due to the distribution of
information in the neurons. Only a few SFA units contain useful information
for the task, and when these neurons are discovered with gradient descent in
later steps, the performance increases rapidly. Moreover, transfer:sfa learns
faster compared to other methods for less number of configurations (see 50 and
100 configs in Figure 4), and also requires less computation compared to
transfer:full as there are less number of units, which is an essential
property for small, autonomous systems with no access to a graphics processing
unit (GPU).

###  4.2 Correlation with High-Level Task Properties

In this experiment, we investigate correlations between various units and
high-level features. To this end, we freeze the network and run the policy for
100 episodes to collect hidden layer activations for PCA and SFA calculations.
Then, we apply PCA and SFA with 100 components to the last hidden layer
activations. To understand whether the transformed features capture any high-
level features such as ‘distance from the tip to the goal’ or joint angles
that are helpful to solve the task, we calculate the correlation between those
high-level features and PCA/SFA features. We also compare the correlations
between the last layer activations to see the effect of the these
transformations on the correlations.

(a) $D(x_{tip},x_{goal})$ (b) $D(x_{tip},x_{obstacle})$ (c) ‘path blocked’ (d)
Joint 0  (e) Joint 1  (f) Joint 2  (g) Joint 3  (h) Joint 4 (i) Joint 5

Figure 5: Units are sorted by their correlation. SFA and PCA units are
highlighted in red and green, respectively.

In Figure 5, each subplot shows the distribution of correlations for each
different high-level feature. Units are sorted by their correlation value for
better visualization. After the transformation with PCA and SFA, most of the
units become less correlated while only a few of them become highly correlated
(e.g. Figures 5(d) and 5(e)). This is probably due to PCA and SFA objectives
force components to be decorrelated. From Figure 5, we can say that PCA and
SFA capture the general neural response; they eliminate redundant neural
responses and focus on the important ones by keeping the general response
profile of the hidden layer. Out of the two, SFA unit distribution has more
units with high correlation, especially for joint angles (Figures 5(d)-5(g)).
This might be one of the reasons that the learning performance of transfer:sfa
is comparable to transfer:full, and even better for the case of less number of
configurations since redundant and possibly noisy variations are filtered out
by SFA.

Units that correlate the maximum with high-level features are reported. The
numbers on the left and the right denotes the correlation and the index of the
unit, respectively.  High-level feature Full activations PCA features SFA
features Joint 1 0.69 / 297 0.74 / 2 0.77 / 1 Joint 2 0.55 / 249 0.49 / 5 0.58
/ 3 Joint 3 0.55 / 249 0.50 / 5 0.59 / 3 Joint 4 0.56 / 249 0.52 / 5 0.61 / 3
Joint 5 0.38 / 471 0.20 / 3 0.19 / 2 Joint 6 0.69 / 297 0.74 / 2 0.77 / 1
$D(x_{\text{tip}},x_{\text{goal}})$ 0.35 / 303 0.26 / 2 0.26 / 2
$D(x_{\text{tip}},x_{\text{obstacle}})$ 0.48 / 147 0.47 / 3 0.40 / 3 ‘path
blocked’ 0.35 / 147 0.38 / 3 0.25 / 2

In Table 4.2, we select the most correlated units from each method and report
their correlations. We see that all methods have high correlation with joint
angles and slightly lower correlation with high-level features that relates
the tip position to other objects. The high correlation with joint angles is
an expected result as the agent should know about the location of the arm in
order to navigate it to the goal position.

###  4.3 Visualizing Features

We created heatmaps for the responses of different SFA and PCA units to
varying inputs. In Figure 6, each rectangle represents a unit’s average
response to different tip locations. For example, for the top left unit in
Figure 6(a), the response is high when the tip is located around the bottom
left corner of the table, and it is low around the bottom right corner. To
create these figures, we averaged out over other variables (i.e. the goal
position and the obstacle position). We see that both SFA and PCA units
response to blob-like regions of the table. The first few units are very
compact and can be partially treated as symbolic representations (e.g. in
Figure 6(a), the first and the third units detect the position in $x$-axis and
$y$-axis, respectively.) As the unit number increases, these regions start to
become scattered.

(a) SFA responses to the tip position.  (b) PCA responses to the tip position.

Figure 6: First 20 SFA and PCA units’ responses to different tip positions.

While the tip position is a useful information to solve the task, the agent
should also know the relative position of the goal and the obstacle with
respect to its tip to successfully navigate in the environment. To understand
the responses for relative distance to the goal and the obstacle, we created
heatmaps with the similar procedure in Figures 7 and 8. The center point of
each subplot represents the zero distance (i.e.
$x_{\text{goal}}-x_{\text{tip}}=(0,0)$). In Figure 7, most of the units are
scattered except the first few ones. This figure suggests that both SFA and
PCA are not very successful at covering the relative goal position compactly;
the information is distributed into many units as in plain activations.
However, in Figure 8, we see that both methods generates neuron responses that
are inactive when the tip is close to the obstacle (e.g. the second SFA unit
and the first PCA unit). The response of SFA is more uniformly distributed
compared to PCA which might be one of the reasons for higher transfer
performance when an L-shaped obstacle is introduced.

(a) SFA responses to the relative goal position.  (b) PCA responses to the
relative goal position.

Figure 7: First 20 SFA and PCA features’ responses to the relative goal
position. The center point represents (0, 0), i.e. zero distance between the
tip and the goal.

(a) SFA responses to the relative obstacle position. (b) PCA responses to the
relative obstacle position.

Figure 8: First 20 SFA and PCA features’ responses to the relative obstacle
position. The center point represents (0, 0), i.e. zero distance between the
tip and the obstacle.

To understand whether these two high-level information are present in the
network prior to transformation, we visualized the neurons that correlate the
most with the relative goal position and the relative obstacle position in
Figure 9. For the bottom half in Figure 9(a), we see that the most correlating
neuron partially responds to $x$-axis location of the relative goal position.
On the other hand, in Figure 9(b), the response is similar to SFA and PCA
units; it becomes inactive when distance is below some threshold.

(a) 303rd neuron  (b) 147th neuron

Figure 9: Neurons that correlate the most with 9(a) the relative goal position
and 9(b) the relative obstacle position.

We conclude that even though the plain activations carry a distributed
representation in general, they might represent some high-level information as
a by-product which can be further processed and refined. We see that SFA and
PCA transformations create high-level features to detect the tip position,
however, this is not as compact as for the relative positions. Note that both
SFA and PCA are unsupervised methods, not specifically tailored for creating
more symbolic information. However, since they automatically create low-
complexity signals in a principled way, they are of good candidates of a more
complex system.

##  5 Conclusion

As the literature expands rapidly, the integration of architectures that are
optimized for a specific scenario will gain more importance. In this work, we
provide a principled way for transferring the existing knowledge in a network
to new problems. Our experimental results show that applying SFA, an
unsupervised method, to the last hidden layer of a trained network generates
features that are useful for skill transfer. Transfer with SFA performs better
with less number of units compared to transfer with the full layer. This is a
desirable property for life-long learning systems because of its resource
economy. We see that features that are generated from SFA are more
interpretable and they can be treated as quasi-symbolic information. Moreover,
due to its formulation, the components with lowest eigenvalue will contain the
most useful and symbol-like information. Therefore, one can always tune the
number of units by simply picking the first $k$ components with the lowest
eigenvalues, a procedure familiar to PCA.

As a future work, we plan to extend the experimental setup to a more
continual, open-ended environment so that we can learn representations that
are of progressive complexity to see the limits of the method. Another
possible future work is to integrate the slowness prior to the objective of
learning to make the whole system an end-to-end architecture. The
effectiveness of other dimensionality reduction methods such as independent
component analysis for skill transfer can be also explored as a future work.

## Acknowledgement

This research was supported by the International Joint Research Promotion
Program of Osaka University under the project “Developmentally and
biologically realistic modeling of perspective invariant action understanding”
and TÜBİTAK (The Scientific and Technological Research Council of Turkey)
ARDEB 1001 program (project number: 120E274).

## References

  * [1] Konidaris G. On the necessity of abstraction. Current Opinion in Behavioral Sciences. 2019;29:1–7.
  * [2] Taniguchi T, Ugur E, Hoffmann M, Jamone L, Nagai T, Rosman B, Matsuka T, Iwahashi N, Oztop E, Piater J, et al.. Symbol emergence in cognitive developmental systems: a survey. IEEE Transactions on Cognitive and Developmental Systems. 2019;.
  * [3] Ewert JP. Motion perception shapes the visual world of amphibians. 2004. p. 117–160.
  * [4] Petrick R, Kraft D, Mourao K, Pugeault N, Krüger N, Steedman M. Representation and integration: Combining robot control, high-level planning, and action learning. In: Proceedings of the 6th international cognitive robotics workshop. 2008. p. 32–41.
  * [5] Sun R. Symbol grounding: A new look at an old idea. Philosophical Psychology. 2000;13(149–172).
  * [6] Konidaris G, Kaelbling LP, Lozano-Perez T. Constructing symbolic representations for high-level planning. In: 28th aaai conf. on ai. 2014.
  * [7] Asai M, Fukunaga A. Classical planning in deep latent space: Bridging the subsymbolic-symbolic boundary. In: Proceedings of the aaai conference on artificial intelligence. Vol. 32. 2018.
  * [8] Ahmetoglu A, Seker MY, Sayin A, Bugur S, Piater J, Oztop E, Ugur E. Deepsym: Deep symbol generation and rule learning from unsupervised continuous robot interaction for planning. arXiv preprint arXiv:201202532. 2020;.
  * [9] Bengio Y. Learning deep architectures for ai. Now Publishers Inc. 2009.
  * [10] Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems. 2012;25:1097–1105.
  * [11] Sutskever I, Vinyals O, Le QV. Sequence to sequence learning with neural networks. arXiv preprint arXiv:14093215. 2014;.
  * [12] Silver D, Huang A, Maddison CJ, Guez A, Sifre L, Van Den Driessche G, Schrittwieser J, Antonoglou I, Panneershelvam V, Lanctot M, et al.. Mastering the game of go with deep neural networks and tree search. nature. 2016;529(7587):484–489.
  * [13] Senior AW, Evans R, Jumper J, Kirkpatrick J, Sifre L, Green T, Qin C, Žídek A, Nelson AW, Bridgland A, et al.. Improved protein structure prediction using potentials from deep learning. Nature. 2020;577(7792):706–710.
  * [14] Weiss K, Khoshgoftaar TM, Wang D. A survey of transfer learning. Journal of Big data. 2016;3(1):1–40.
  * [15] Vanschoren J. Meta-learning: A survey. arXiv preprint arXiv:181003548. 2018;.
  * [16] Thrun S, Mitchell TM. Lifelong robot learning. Robotics and Autonomous Systems. 1995;15(1):25–46. Available from: http://www.sciencedirect.com/science/article/pii/092188909500004Y.
  * [17] Rusu AA, Rabinowitz NC, Desjardins G, Soyer H, Kirkpatrick J, Kavukcuoglu K, Pascanu R, Hadsell R. Progressive neural networks. 2016. 1606.04671.
  * [18] Li Z, Hoiem D. Learning without forgetting. In: Leibe B, Matas J, Sebe N, Welling M, editors. Computer vision – eccv 2016. Springer International Publishing. 2016. p. 614–629.
  * [19] Ugur E, Piater J. Bottom-up learning of object categories, action effects and logical rules: From continuous manipulative exploration to symbolic planning. In: 2015 ieee international conference on robotics and automation (icra). IEEE. 2015. p. 2627–2633.
  * [20] Schmidhuber J. Driven by compression progress: A simple principle explains essential aspects of subjective beauty, novelty, surprise, interestingness, attention, curiosity, creativity, art, science, music, jokes. In: Pezzulo G, Butz MV, Sigaud O, Baldassarre G, editors. Anticipatory behavior in adaptive learning systems. Springer Berlin Heidelberg. 2008. p. 48–76.
  * [21] Wolff JG. Information compression, multiple alignment, and the representation and processing of knowledge in the brain. Frontiers in psychology. 2016;7:1584–1584. Available from: https://pubmed.ncbi.nlm.nih.gov/27857695https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5093141/.
  * [22] Wiskott L, Sejnowski TJ. Slow feature analysis: Unsupervised learning of invariances. Neural computation. 2002;14(4):715–770.
  * [23] Mnih V, Kavukcuoglu K, Silver D, Graves A, Antonoglou I, Wierstra D, Riedmiller M. Playing atari with deep reinforcement learning. arXiv preprint arXiv:13125602. 2013;.
  * [24] Konidaris G, Kaelbling LP, Lozano-Perez T. From skills to symbols: Learning symbolic representations for abstract high-level planning. Journal of Artificial Intelligence Research. 2018;61:215–289.
  * [25] Santello M, Flanders M, Soechting JF. Postural hand synergies for tool use. Journal of neuroscience. 1998;18(23):10105–10115.
  * [26] Tripathi GN, Wagatsuma H. Pca-based algorithms to find synergies for humanoid robot motion behavior. International Journal of Humanoid Robotics. 2016;13(02):1550037.
  * [27] Promsri A, Federolf P. Analysis of postural control using principal component analysis: The relevance of postural accelerations and of their frequency dependency for selecting the number of movement components. Frontiers in Bioengineering and Biotechnology. 2020;8:480.
  * [28] Schaal S. Dynamic movement primitives-a framework for motor control in humans and humanoid robotics. In: Adaptive motion of animals and machines. Springer. 2006\. p. 261–280.
  * [29] Chen N, Bayer J, Urban S, Van Der Smagt P. Efficient movement representation by embedding dynamic movement primitives in deep autoencoders. In: 2015 ieee-ras 15th international conference on humanoid robots (humanoids). IEEE. 2015. p. 434–440.
  * [30] Kompella VR, Luciw M, Schmidhuber J. Incremental slow feature analysis. In: Twenty-second international joint conference on artificial intelligence. Citeseer. 2011.
  * [31] Legenstein R, Wilbert N, Wiskott L. Reinforcement learning on slow features of high-dimensional input streams. PLoS Comput Biol. 2010;6(8):e1000894.
  * [32] Goodfellow I, Bengio Y, Courville A, Bengio Y. Deep learning. Vol. 1. MIT press Cambridge. 2016.
  * [33] Trefethen LN, Bau III D. Numerical linear algebra. Vol. 50. Siam. 1997.
  * [34] LeCun YA, Bottou L, Orr GB, Müller KR. Efficient backprop. In: Neural networks: Tricks of the trade. Springer. 2012. p. 9–48.
  * [35] Rohmer E, Singh SPN, Freese M. Coppeliasim (formerly V-REP): a versatile and scalable robot simulation framework. In: Proc. of the international conference on intelligent robots and systems (iros). 2013. www.coppeliarobotics.com.
  * [36] Universal Robots. The UR10 collaborative industrial robot. https://www.universal-robots.com/products/ur10-robot. 2012. online; accessed 10 September 2020.
  * [37] Kingma DP, Ba J. Adam: A method for stochastic optimization. arXiv preprint arXiv:14126980. 2014;.
  * [38] Welch BL. The generalization ofstudent’s’ problem when several different population variances are involved. Biometrika. 1947;34(1/2):28–35.