Motion Classification Based on Harmonic Micro-Doppler Signatures Using a
  Convolutional Neural Network
We demonstrate the classification of common motions of held objects using the
harmonic micro-Doppler signatures scattered from harmonic radio-frequency tags.
Harmonic tags capture incident signals and retransmit at harmonic frequencies,
making them easier to distinguish from clutter. We characterize the motion of
tagged handheld objects via the time-varying frequency shift of the harmonic
signals (harmonic Doppler). With complex micromotions of held objects, the
time-frequency response manifests complex micro-Doppler signatures that can be
used to classify the motions. We developed narrow-band harmonic tags at 2.4/4.8
GHz that support frequency scalability for multi-tag operation, and a harmonic
radar system to transmit a 2.4 GHz continuous-wave signal and receive the
scattered 4.8 GHz harmonic signal. Experiments were conducted to mimic four
common motions of held objects from 35 subjects in a cluttered indoor
environment. A 7-layer convolutional neural network (CNN) multi-label
classifier was developed and obtained a real time classification accuracy of
94.24%, with a response time of 2 seconds per sample with a data processing
latency of less than 0.5 seconds.
Cory Hilton, Steve Bush, Faiz Sherman, Matt Barker, Aditya Deshpande, Steve Willeke, Jeffrey A. Nanzer
Motion Classification Based on Harmonic Micro-Doppler Signatures Using a Convolutional Neural Network

###### Abstract

We demonstrate the classification of common motions of held objects using the harmonic micro-Doppler signatures scattered from harmonic radio-frequency tags. Harmonic tags capture incident signals and retransmit at harmonic frequencies, making them easier to distinguish from clutter. We characterize the motion of tagged handheld objects via the time-varying frequency shift of the harmonic signals (harmonic Doppler). With complex micromotions of held objects, the time-frequency response manifests complex micro-Doppler signatures that can be used to classify the motions. We developed narrow-band harmonic tags at 2.4/4.8 GHz that support frequency scalability for multi-tag operation, and a harmonic radar system to transmit a 2.4 GHz continuous-wave signal and receive the scattered 4.8 GHz harmonic signal. Experiments were conducted to mimic four common motions of held objects from 35 subjects in a cluttered indoor environment. A 7-layer convolutional neural network (CNN) multi-label classifier was developed and obtained a real time classification accuracy of 94.24\(\%\), with a response time of 2 seconds per sample with a data processing latency of less than 0.5 seconds.

 Harmonic radar, Micro-Doppler, Narrow-band antennas, RF tags, Classification, Gesture recognition

## I Introduction

Increasingly rapid developments in human-computer interaction, the internoet of things (IoT), home health, and other areas is leading to more demand for accurate detection and estimation of the motions of people in living spaces. Accurate classification of human motions enables wireless control of devices for interactive entertainment technologies such as augmented reality and virtual reality (AR/VR), determination of human activities to improve home health monitoring, and also differentiation between people and other moving objects. Classifying the motions of people in living spaces is challenging, however, due to the complexity of the environment. Optical systems may be used, however they often entail computationally expensive image processing and have privacy concerns since images of people are formed and processed. Infrared systems generally operate under similar principals and have similar drawbacks. Microwave radar has been increasingly used in short-range motion detection applications due to a number of beneficial aspects that radar holds over optical systems [1, 2]. Radar systems process complex signal returns, so that motion can be directly measured via Doppler frequency shifts, unlike optical systems which are generally intensity-based, requiring techniques like change detection to infer motion. Microwave radar also provides a reliable method of detecting moving objects without forming images of people, alleviating such privacy concerns. Furthermore, when monitoring the motions of objects with rapid variations, such as human movements, the micromotions of the body generate distinguishable signatures in the time-frequency response of the radar return, called micro-Doppler signatures, which can be used for classification of the micromotions [3, 4, 5, 6, 7].

One of the principal challenges in detecting and classifying micro-Doppler signatures in living spaces is the presence of significant signal reflections from the environment. These clutter returns can be significantly higher in power than that returns from a moving person. While the clutter returns are centered at zero Doppler, the phase noise of the radar system contributes to a broader frequency response that overlaps with slow moving targets making them indistinguishable. Since human motions tend to be relatively slow, these micro-Doppler responses can thus be challenging to detect in highly cluttered indoor environments, and even if detected, the presence of system noise may reduce the classification accuracy. Harmonic radars and harmonic tags have been developed to overcome these challenges by moving the signal response to a frequency band without clutter [8, 9, 10, 11]. A harmonic tag accepts the incident radar transmit signal at the fundamental frequency \(f_{c}\) and using a nonlinear component such as a diode generates a higher harmonic signal. Typically the second harmonic \(2f_{c}\), which is then retransmitted and captured by the radar receiver. Since all the clutter returns are present only near \(f_{c}\), the harmonic tag signal can theoretically be detected more easily, however the conversion efficiency of the tag dictates how much return power the radar captures.

While previous works on harmonic tags have principally focused on detection, we recently presented a method of detecting the motion of harmonic tags via the harmonic Doppler return and demonstrated the presence of micro-motion information in harmonic micro-Doppler signatures [12, 13]. When detecting the motion of multiple objects, differentiability of the separate radar returns is necessary. To address this with a low-cost tag, we recently designed a narrow-band harmonic tag that can be used for frequency-selective harmonic Doppler