Activity recognition in smart home environment using wireless ambient sensing is a well-known problem that is being researched very actively . Rapid development in the sensing technologies has made human activity recognition very important for various fields such as health care , home monitoring , surveillance , etc . In this paper , we provide a review on the activity recognition systems that use integrated sensors in the mobile phone with a special focus on the systems that target personal health and well-being applications . Initially , we provide background information about the activity recognition process , such as the sensors used , activities targeted , and the steps of activity recognition using machine learning algorithms , before listing the challenges of activity recognition on mobile phones . Next , we focus on the classification of existing work on the topic together with a detailed taxonomy . Considering the power of binary classification in contrast to direct multiclass approaches , and the capabilities offered by multi-sense environments , we define a novel classification schema based on hierarchical structures composed by weighted decision makers . Remarkable accuracy results are obtained for a particular activity recognition problem in contrast to a traditional multiclass majority voting algorithm . As an important technology in the field of Internet of Things , activity recognition is a well-researched topic . Due to its characteristics of no-invasion , no privacy leakage , high recognition rate , and good user experience , Channel State Information (CSI)-based is a great approach to solve the activity recognition problem . However , commodity WiFi devices for identification are usually in a complex electromagnetic environment , where the interference caused by WiFi devices from channel overlap is the most severe . Not only it estimates delay between two adjacent events , but its first part allows non temporal algorithms to work on temporal databases and reduces the complexity of dealing with temporal data for the others . The algorithm has been implemented to address the problem of activities models creation , the first step in activity recognition process , from sensors history log recorded in a smart home . Experiments , on synthetic data and on real smart home sensors log , have proven the algorithm effectiveness in detecting all frequent activities in an efficient time . In this paper , we present two proof-of-concept , web-based HCGs aimed at enabling game players to annotate accelerometer data . Using results from pilot studies with Amazon Mechanical Turk players , we discuss key challenges , opportunities , and , more generally , the potential of using applied videogames for annotating raw accelerometer data to support activity recognition research . Most of the detection algorithms on board of these devices however are closed-source and the devices do not allow the recording of raw inertial data . This paper presents a project that , faced by these limitations of commercial wearable products , set out to create an open-source recording platform for activity recognition research that ( 1 ) is sufficiently power-efficient , and ( 2 ) remains small and comfortable enough to wear , to be able to record raw inertial data for extended periods of time . We study especially , via high-resolution power profiling , several trade-offs present in the choice for the basic hardware components of our prototype , and contribute with three key design areas that have had a significant impact on our prototype design . In this paper we propose a model that employs both the estimated computation time and power consumption of each application kernel to find the best computing configuration for the whole application . As a case study , our approach is applied to the implementation of an irregular algorithm on a heterogeneous embedded architecture , more precisely an algorithm used in computer vision applications like human action or gait recognition . We analyze two parallelization versions : a non-pipelined version and a pipelined one , and we use our approach to obtain the mapping with least energy consumption . One such area is the diagnosis of local area networks . Diagnostic problems are solved in various fields of human action . Current trends in the field of automation of the solution of diagnostic problems associated with the use of computer technology . A chunk is an object containing arbitrary information that competes for retention in an active memory storage . Transforming the problem of selecting actions to that of retaining chunks , we are able to exploit the same exact mechanism for both retention of chunks and generation of actions , consequently dropping out the RL agent previously in 1 A. M. Franchi , F. Mutti , G. Gini , “ From learning to new goal generation in a bioinspired robotic setup ” , Advanced Robotics , 2016 , DOI 10.1080/01691864.2016.1172732 charge of generating the actions . During each agent-environment interaction , the WM receives from sensors and inner processes the current state and a set of chunks of information proposed for retention . However , ignoring sensory evidence means that externally generated sensations will not be perceived . Conversely , attending to ( proprioceptive and somatosensory ) sensations enables the detection of externally generated events but precludes generation of actions . This conflict can be resolved by attenuating the precision of sensory evidence during movement or , equivalently , attending away from the consequences of self-made acts . Experimental results on the challenging YouTube dataset and HOHA-2 dataset demonstrate that our proposed approach accounting for structural information is more effective in representing realistic human actions . In addition , we also investigate the impact of choosing different local features such as SIFT , HOG , and HOF descriptors in human action representation . It is observed that dense keypoints can better exploit the advantages of our proposed approach . The predicted points are then warped back to the observed time series . We demonstrate the utility of these predictions for human action synthesis , and dynamic texture synthesis . Our main contributions are : multivariate phase space reconstruction for human actions and dynamic textures , a deterministic approach to model dynamics in contrast to the popular noise-driven approaches for dynamic textures , and video synthesis from kernel regression in the phase space . Important place in this advancement takes video generation due to the need for human animation applications , automatic trailer or movie generation . Therefore , taking advantage of various GANs , we proposed own method for human movement video generation GOHAG : GANs Orchestration for Human Actions Generations . GOHAG is an orchestra of three GANs , where Poses generation GAN ( PGAN ) creates a sequence of poses , Poses Optimization GAN ( POGAN ) optimizes them , and Frames generation GAN ( FGAN ) attaches texture for the sequence , creating a video . Human doing actions will result in WiFi distortion , which is widely explored for action recognition , such as the elderly fallen detection , hand sign language recognition , and keystroke estimation . As our best survey , past work recognizes human action by categorizing one complete distortion series into one action , which we term as series-level action recognition . We propose an iterative , probabilistically constrained algorithm for exploring the space of motor commands and show that the algorithm can quickly discover dynamically stable actions for whole-body imitation of human motion . Experimental results based on simulation and subsequent execution by a HOAP-2 humanoid robot demonstrate that our algorithm is able to imitate a human performing actions such as squatting and a one-legged balance . I. INTRODUCTION Imitation learning presents a promising approach to the problem of enabling complex behavior learning in humanoid robots . The probabilistic nature of our model immediately allows the adoption of the prediction entropy and estimated expertise for active selection of data sample to be labeled , and active selection of high quality labelers to label the data , respectively . We apply the proposed model for three visual recognition tasks , i.e. , object category recognition , gender recognition , and multi-modal activity recognition , on three datasets with real crowd-sourced labels from Amazon Mechanical Turk . The experiments clearly demonstrated the efficacy of the proposed model . We identify the key technical challenges in analyzing these behaviors , and describe methods for decoding the interactions . We present experimental results that demonstrate the potential of the dataset to drive interesting research questions , and show preliminary results for multi-modal activity recognition . A Collaborative Map Creation Activity for Deep Comprehension . This paper describes the procedures of a multimodal activity combining different collaborative , communicative modes — the creation of group concept maps and oral presentation of the maps — to assist college-level students in deciphering complex academic texts . Based on the perspective of social constructivism ( Vygotsky , 1978 ) , the authors attempt to explore how the activity provided Japanese university students with an opportunity to confirm and validate their own understanding of the texts and modify and integrate their understanding with that of their classmates with the common goal of attaining a deeper level of comprehension . We present DataPrism , a new interactive visualization tool to aid analysis of multimodal activity data . DataPrism enables analysts to visualize , annotate , and link multiple time-based data streams , including video , log files , and paper-based digital notes . A joint generative system model based on finite mixtures is proposed and the structural properties of the associated PGM are addressed in order to obtain an iterative algorithm for estimation of activity and multimodal segmentation . In a SPECT/MRI digital phantom study , the proposed algorithm outperforms a well established method for multi-modal activity estimation in terms of bias/variance characteristics and identification of lesions . The CRN Toolbox enables fast implementation of activity and context recognition systems , featuring mechanisms for distributed processing and support for mobile and wearable devices . CRN Toolbox is a tool set specifically optimized for implementing multimodal , distributed activity and context recognition systems running on Posix operating systems . Like conventional rapid- prototyping tools , the CRN Toolbox contains a collection of ready-to-use algorithms ( signal processing , pattern classification , and so on ) . Since computers are now interacting with humans , they need to be able to detect these facial cues and accordingly adjust their services based on this feedback . Our proposed method aims at building a Multimodal Group Activity State Detection for Classroom Response System which tries to recognize the learning behavior of a classroom for providing effective feedback and inputs to the teacher . The key challenges dealt here are to detect and analyze as many students as possible for a non-biased evaluation of the mood of the students and classify them into three activity states defined : active , passive , and inactive . The existing activity recognition approaches either rely on domain knowledge or fail to address the spatial-temporal dependencies of the sensing signals . In this paper , we propose a novel attention-based multimodal neural network model called AttnSense for multimodal human activity recognition . AttnSense introduce the framework of combining attention mechanism with a convolutional neural network ( CNN ) and a Gated Recurrent Units ( GRU ) network to capture the dependencies of sensing signals in both spatial and temporal domains , which shows advantages in prioritized sensor selection and improves the comprehensibility . We treat fusion structure optimization as a hyperparameter search and cast it as a discrete optimization problem under the Bayesian optimization framework . We propose two methods to compute structural similarities in the search space of tree-structured multimodal architectures , and demonstrate their effectiveness on two challenging multimodal human activity recognition problems . We explore two inference methods for obtaining the most probable set of activities per window : probability propagation and abduction . In contrast with the standard approach that uses several models , we use a single classifier for multiple activity recognition . We evaluated the model with real image sequences of 6 different activities performed continuously by different people . This hierarchical approach brings more realistic videos with plausible motion and appearance consistency . Our experiments show that our model generates more plausible motion videos and also achieves significantly improved performance for unsupervised action classification in comparison to previous GAN works . In addition , because our model generates videos from two independent information , our model can generate new combinations of motion and attribute that are not seen in training data , such as a video in which a person is doing sit-up in a baseball ground . Based on the human-centric reference frames , a principled analysis of the correspondence problem leads to a novel probabilistic action representation called the correspondence-ambiguous feature histogram array ( CAFHA ) that is robust to variations across similar actions . CAFHA is shown to be effective in unsupervised action classification and quasi real-time action inference . A novel feature selection method is proposed to select the optimal features to improve the CAFHA representation , such that the best discrimination between different action clusters may be found via unsupervised spectral clustering . We propose a new task of unsupervised action detection by action matching . Given two long videos , the objective is to temporally detect all pairs of matching video segments . To address both issues , we propose a novel , regularized Gaussian PLSA ( RG-PLSA ) model that combines Gaussian PLSAs and hierarchical Gaussian mixture models ( HGMM ) . We evaluate our model on supervised human action recognition tasks , using two publicly available datasets . Average classification accuracies of 97.69 % and 93.72 % are achieved on the Weizmann and KTH Action Datasets , respectively , which demonstrate that the RG-PLSA model outperforms Gaussian PLSAs and HGMMs , and is comparable to the state of the art . In the recent years , with a rapid development of video capture devices ( Kinect , etc ) and online video providers ( Youtube , Youku , etc ) , the number of video data , categories of actions and complexities of videos ’ content are explosively increasing . Thus , for the traditional supervised human action recognition , it ’s expensive and intractable to annotate the huge number of videos manually . In the consideration of this situation , researchers dedicate to predict the label of an unseen video using disjoint training videos , which is referred to as zero-shot learning for human action recognition . To this end a novel method , coined as temporal archetypal analysis , is proposed . The performance of the proposed method is assessed by conducting experiments in unsupervised action segmentation . Experimental results on three different real world datasets using single modal and multimodal visual representations indicate the robustness and effectiveness of the proposed methods , outperforming compared state-of-the-art methods by a large , in most of the cases , margin . Our model learns the action/object co-occurrence and action temporal relations in the activity , and uses the learned rich relationships to infer the forgotten action and the related object . We show that our approach not only improves the unsupervised action segmentation and action cluster assignment performance , but also effectively detects the forgotten actions on a challenging human activity RGB-D video dataset . In robotic experiments , we show that our robot is able to remind people of forgotten actions successfully . The architecture comprises growing self-organizing networks equipped with recurrent neurons for processing time-varying patterns . We use a set of hierarchically arranged recurrent networks for the unsupervised learning of action representations with increasingly large spatiotemporal receptive fields . Lifelong learning is achieved in terms of prediction-driven neural dynamics in which the growth and the adaptation of the recurrent networks are driven by their capability to reconstruct temporally ordered input sequences . We show our method ’s success on two different action-object datasets , where our method achieves similar or better results than the supervised methods . We also show that our method can be successfully used as pretraining for a supervised action-object detection task . The Action Verb Corpus ( AVC ) is a multimodal dataset of simple actions for robot learning . The extension introduced here is especially geared to supervised learning of actions from human motion data . Recorded are RGB-D videos of the test scene , grayscale videos from the user ’s perspective , human hand trajectories , object poses and speech utterances . Also , we propose a sparse representation for the human body based on dictionaries in order to recognize actions with rapid body deformations . In the next approach , a low-dimensional representation called action-vector for unsupervised action recognition is presented . Further , linear and non-linear embedding of action-vectors is proposed for addressing inter-action similarity and fine-grained action recognition , respectively . It allows us to model long-range action relations that commonly exist in the complex activity , which is challenging to capture in the previous works . We apply our model to unsupervised action segmentation and recognition , and also to a novel application that detects forgotten actions , which we call action patching . For evaluation , we also contribute a new challenging RGB-D activity video dataset recorded by the new Kinect v2 , which contains several human daily activities as compositions of multiple actions interacted with different objects .
Activity recognition in smart home environment using wireless ambient sensing is a well-known problem that is being researched very actively . Rapid development in the sensing technologies has made human activity recognition very important for various fields such as health care , home monitoring , surveillance , etc .
In this paper , we provide a review on the activity recognition systems that use integrated sensors in the mobile phone with a special focus on the systems that target personal health and well-being applications . Initially , we provide background information about the activity recognition process , such as the sensors used , activities targeted , and the steps of activity recognition using machine learning algorithms , before listing the challenges of activity recognition on mobile phones . Next , we focus on the classification of existing work on the topic together with a detailed taxonomy .
Considering the power of binary classification in contrast to direct multiclass approaches , and the capabilities offered by multi-sense environments , we define a novel classification schema based on hierarchical structures composed by weighted decision makers . Remarkable accuracy results are obtained for a particular activity recognition problem in contrast to a traditional multiclass majority voting algorithm .
As an important technology in the field of Internet of Things , activity recognition is a well-researched topic . Due to its characteristics of no-invasion , no privacy leakage , high recognition rate , and good user experience , Channel State Information (CSI)-based is a great approach to solve the activity recognition problem . However , commodity WiFi devices for identification are usually in a complex electromagnetic environment , where the interference caused by WiFi devices from channel overlap is the most severe .
Not only it estimates delay between two adjacent events , but its first part allows non temporal algorithms to work on temporal databases and reduces the complexity of dealing with temporal data for the others . The algorithm has been implemented to address the problem of activities models creation , the first step in activity recognition process , from sensors history log recorded in a smart home . Experiments , on synthetic data and on real smart home sensors log , have proven the algorithm effectiveness in detecting all frequent activities in an efficient time .
In this paper , we present two proof-of-concept , web-based HCGs aimed at enabling game players to annotate accelerometer data . Using results from pilot studies with Amazon Mechanical Turk players , we discuss key challenges , opportunities , and , more generally , the potential of using applied videogames for annotating raw accelerometer data to support activity recognition research .
Most of the detection algorithms on board of these devices however are closed-source and the devices do not allow the recording of raw inertial data . This paper presents a project that , faced by these limitations of commercial wearable products , set out to create an open-source recording platform for activity recognition research that ( 1 ) is sufficiently power-efficient , and ( 2 ) remains small and comfortable enough to wear , to be able to record raw inertial data for extended periods of time . We study especially , via high-resolution power profiling , several trade-offs present in the choice for the basic hardware components of our prototype , and contribute with three key design areas that have had a significant impact on our prototype design .
In this paper we propose a model that employs both the estimated computation time and power consumption of each application kernel to find the best computing configuration for the whole application . As a case study , our approach is applied to the implementation of an irregular algorithm on a heterogeneous embedded architecture , more precisely an algorithm used in computer vision applications like human action or gait recognition . We analyze two parallelization versions : a non-pipelined version and a pipelined one , and we use our approach to obtain the mapping with least energy consumption .
One such area is the diagnosis of local area networks . Diagnostic problems are solved in various fields of human action . Current trends in the field of automation of the solution of diagnostic problems associated with the use of computer technology .
A chunk is an object containing arbitrary information that competes for retention in an active memory storage . Transforming the problem of selecting actions to that of retaining chunks , we are able to exploit the same exact mechanism for both retention of chunks and generation of actions , consequently dropping out the RL agent previously in 1 A. M. Franchi , F. Mutti , G. Gini , “ From learning to new goal generation in a bioinspired robotic setup ” , Advanced Robotics , 2016 , DOI 10.1080/01691864.2016.1172732 charge of generating the actions . During each agent-environment interaction , the WM receives from sensors and inner processes the current state and a set of chunks of information proposed for retention .
However , ignoring sensory evidence means that externally generated sensations will not be perceived . Conversely , attending to ( proprioceptive and somatosensory ) sensations enables the detection of externally generated events but precludes generation of actions . This conflict can be resolved by attenuating the precision of sensory evidence during movement or , equivalently , attending away from the consequences of self-made acts .
Experimental results on the challenging YouTube dataset and HOHA-2 dataset demonstrate that our proposed approach accounting for structural information is more effective in representing realistic human actions . In addition , we also investigate the impact of choosing different local features such as SIFT , HOG , and HOF descriptors in human action representation . It is observed that dense keypoints can better exploit the advantages of our proposed approach .
The predicted points are then warped back to the observed time series . We demonstrate the utility of these predictions for human action synthesis , and dynamic texture synthesis . Our main contributions are : multivariate phase space reconstruction for human actions and dynamic textures , a deterministic approach to model dynamics in contrast to the popular noise-driven approaches for dynamic textures , and video synthesis from kernel regression in the phase space .
Important place in this advancement takes video generation due to the need for human animation applications , automatic trailer or movie generation . Therefore , taking advantage of various GANs , we proposed own method for human movement video generation GOHAG : GANs Orchestration for Human Actions Generations . GOHAG is an orchestra of three GANs , where Poses generation GAN ( PGAN ) creates a sequence of poses , Poses Optimization GAN ( POGAN ) optimizes them , and Frames generation GAN ( FGAN ) attaches texture for the sequence , creating a video .
Human doing actions will result in WiFi distortion , which is widely explored for action recognition , such as the elderly fallen detection , hand sign language recognition , and keystroke estimation . As our best survey , past work recognizes human action by categorizing one complete distortion series into one action , which we term as series-level action recognition .
We propose an iterative , probabilistically constrained algorithm for exploring the space of motor commands and show that the algorithm can quickly discover dynamically stable actions for whole-body imitation of human motion . Experimental results based on simulation and subsequent execution by a HOAP-2 humanoid robot demonstrate that our algorithm is able to imitate a human performing actions such as squatting and a one-legged balance . I. INTRODUCTION Imitation learning presents a promising approach to the problem of enabling complex behavior learning in humanoid robots .
The probabilistic nature of our model immediately allows the adoption of the prediction entropy and estimated expertise for active selection of data sample to be labeled , and active selection of high quality labelers to label the data , respectively . We apply the proposed model for three visual recognition tasks , i.e. , object category recognition , gender recognition , and multi-modal activity recognition , on three datasets with real crowd-sourced labels from Amazon Mechanical Turk . The experiments clearly demonstrated the efficacy of the proposed model .
We identify the key technical challenges in analyzing these behaviors , and describe methods for decoding the interactions . We present experimental results that demonstrate the potential of the dataset to drive interesting research questions , and show preliminary results for multi-modal activity recognition .
A Collaborative Map Creation Activity for Deep Comprehension . This paper describes the procedures of a multimodal activity combining different collaborative , communicative modes — the creation of group concept maps and oral presentation of the maps — to assist college-level students in deciphering complex academic texts . Based on the perspective of social constructivism ( Vygotsky , 1978 ) , the authors attempt to explore how the activity provided Japanese university students with an opportunity to confirm and validate their own understanding of the texts and modify and integrate their understanding with that of their classmates with the common goal of attaining a deeper level of comprehension .
We present DataPrism , a new interactive visualization tool to aid analysis of multimodal activity data . DataPrism enables analysts to visualize , annotate , and link multiple time-based data streams , including video , log files , and paper-based digital notes .
A joint generative system model based on finite mixtures is proposed and the structural properties of the associated PGM are addressed in order to obtain an iterative algorithm for estimation of activity and multimodal segmentation . In a SPECT/MRI digital phantom study , the proposed algorithm outperforms a well established method for multi-modal activity estimation in terms of bias/variance characteristics and identification of lesions .
The CRN Toolbox enables fast implementation of activity and context recognition systems , featuring mechanisms for distributed processing and support for mobile and wearable devices . CRN Toolbox is a tool set specifically optimized for implementing multimodal , distributed activity and context recognition systems running on Posix operating systems . Like conventional rapid- prototyping tools , the CRN Toolbox contains a collection of ready-to-use algorithms ( signal processing , pattern classification , and so on ) .
Since computers are now interacting with humans , they need to be able to detect these facial cues and accordingly adjust their services based on this feedback . Our proposed method aims at building a Multimodal Group Activity State Detection for Classroom Response System which tries to recognize the learning behavior of a classroom for providing effective feedback and inputs to the teacher . The key challenges dealt here are to detect and analyze as many students as possible for a non-biased evaluation of the mood of the students and classify them into three activity states defined : active , passive , and inactive .
The existing activity recognition approaches either rely on domain knowledge or fail to address the spatial-temporal dependencies of the sensing signals . In this paper , we propose a novel attention-based multimodal neural network model called AttnSense for multimodal human activity recognition . AttnSense introduce the framework of combining attention mechanism with a convolutional neural network ( CNN ) and a Gated Recurrent Units ( GRU ) network to capture the dependencies of sensing signals in both spatial and temporal domains , which shows advantages in prioritized sensor selection and improves the comprehensibility .
We treat fusion structure optimization as a hyperparameter search and cast it as a discrete optimization problem under the Bayesian optimization framework . We propose two methods to compute structural similarities in the search space of tree-structured multimodal architectures , and demonstrate their effectiveness on two challenging multimodal human activity recognition problems .
We explore two inference methods for obtaining the most probable set of activities per window : probability propagation and abduction . In contrast with the standard approach that uses several models , we use a single classifier for multiple activity recognition . We evaluated the model with real image sequences of 6 different activities performed continuously by different people .
This hierarchical approach brings more realistic videos with plausible motion and appearance consistency . Our experiments show that our model generates more plausible motion videos and also achieves significantly improved performance for unsupervised action classification in comparison to previous GAN works . In addition , because our model generates videos from two independent information , our model can generate new combinations of motion and attribute that are not seen in training data , such as a video in which a person is doing sit-up in a baseball ground .
Based on the human-centric reference frames , a principled analysis of the correspondence problem leads to a novel probabilistic action representation called the correspondence-ambiguous feature histogram array ( CAFHA ) that is robust to variations across similar actions . CAFHA is shown to be effective in unsupervised action classification and quasi real-time action inference . A novel feature selection method is proposed to select the optimal features to improve the CAFHA representation , such that the best discrimination between different action clusters may be found via unsupervised spectral clustering .
We propose a new task of unsupervised action detection by action matching . Given two long videos , the objective is to temporally detect all pairs of matching video segments .
To address both issues , we propose a novel , regularized Gaussian PLSA ( RG-PLSA ) model that combines Gaussian PLSAs and hierarchical Gaussian mixture models ( HGMM ) . We evaluate our model on supervised human action recognition tasks , using two publicly available datasets . Average classification accuracies of 97.69 % and 93.72 % are achieved on the Weizmann and KTH Action Datasets , respectively , which demonstrate that the RG-PLSA model outperforms Gaussian PLSAs and HGMMs , and is comparable to the state of the art .
In the recent years , with a rapid development of video capture devices ( Kinect , etc ) and online video providers ( Youtube , Youku , etc ) , the number of video data , categories of actions and complexities of videos ’ content are explosively increasing . Thus , for the traditional supervised human action recognition , it ’s expensive and intractable to annotate the huge number of videos manually . In the consideration of this situation , researchers dedicate to predict the label of an unseen video using disjoint training videos , which is referred to as zero-shot learning for human action recognition .
To this end a novel method , coined as temporal archetypal analysis , is proposed . The performance of the proposed method is assessed by conducting experiments in unsupervised action segmentation . Experimental results on three different real world datasets using single modal and multimodal visual representations indicate the robustness and effectiveness of the proposed methods , outperforming compared state-of-the-art methods by a large , in most of the cases , margin .
Our model learns the action/object co-occurrence and action temporal relations in the activity , and uses the learned rich relationships to infer the forgotten action and the related object . We show that our approach not only improves the unsupervised action segmentation and action cluster assignment performance , but also effectively detects the forgotten actions on a challenging human activity RGB-D video dataset . In robotic experiments , we show that our robot is able to remind people of forgotten actions successfully .
The architecture comprises growing self-organizing networks equipped with recurrent neurons for processing time-varying patterns . We use a set of hierarchically arranged recurrent networks for the unsupervised learning of action representations with increasingly large spatiotemporal receptive fields . Lifelong learning is achieved in terms of prediction-driven neural dynamics in which the growth and the adaptation of the recurrent networks are driven by their capability to reconstruct temporally ordered input sequences .
We show our method ’s success on two different action-object datasets , where our method achieves similar or better results than the supervised methods . We also show that our method can be successfully used as pretraining for a supervised action-object detection task .
The Action Verb Corpus ( AVC ) is a multimodal dataset of simple actions for robot learning . The extension introduced here is especially geared to supervised learning of actions from human motion data . Recorded are RGB-D videos of the test scene , grayscale videos from the user ’s perspective , human hand trajectories , object poses and speech utterances .
Also , we propose a sparse representation for the human body based on dictionaries in order to recognize actions with rapid body deformations . In the next approach , a low-dimensional representation called action-vector for unsupervised action recognition is presented . Further , linear and non-linear embedding of action-vectors is proposed for addressing inter-action similarity and fine-grained action recognition , respectively .
It allows us to model long-range action relations that commonly exist in the complex activity , which is challenging to capture in the previous works . We apply our model to unsupervised action segmentation and recognition , and also to a novel application that detects forgotten actions , which we call action patching . For evaluation , we also contribute a new challenging RGB-D activity video dataset recorded by the new Kinect v2 , which contains several human daily activities as compositions of multiple actions interacted with different objects .
Multi-Resident Activity Recognition using Multi-Label Classification in Ambient Sensing Smart Homes A Review and Taxonomy of Activity Recognition on Mobile Phones Automatic Recognition of Daily Living Activities Based on a Hierarchical Classifier WiAnti: an Anti-Interference Activity Recognition System Based on WiFi CSI New frequent pattern mining algorithm tested for activities models creation Designing Videogames to Crowdsource Accelerometer Data Annotation for Activity Recognition Research Low-power lessons from designing a wearable logger for long-term deployments Calculation of dense trajectory descriptors on a heterogeneous embedded architecture Система мониторинга технического состояния локальной вычислительной сети From Working Memory to Cognitive Control: Presenting a Model for their Integration in a Bio-inspired Architecture Active inference, sensory attenuation and illusions Structure context of local features in realistic human action recognition Time series prediction by chaotic modeling of nonlinear dynamical systems GOHAG: GANs Orchestration for Human Actions Generation Temporal Unet: Sample Level Human Action Recognition using WiFi Dynamic Imitation in a Humanoid Robot through Nonparametric Probabilistic Inference Active Visual Recognition with Expertise Estimation in Crowdsourcing Decoding Children's Social Behavior Fostering Active Readers: A Collaborative Map Creation Activity for Deep Comprehension DataPrism: a tool for visualizing multimodal data Probabilistic Graphical Model of SPECT/MRI Rapid Prototyping of Activity Recognition Applications Multimodal Group Activity State Detection for Classroom Response System Using Convolutional Neural Networks AttnSense: Multi-level Attention Mechanism For Multimodal Human Activity Recognition Bayesian optimization on graph-structured search spaces: Optimizing deep multimodal fusion architectures A Graphical Model for Human Activity Recognition Hierarchical Video Generation from Orthogonal Information: Optical Flow and Texture Human action capturing and classification Unsupervised Human Action Detection by Action Matching Regularized Probabilistic Latent Semantic Analysis with Continuous Observations Zero-shot Action Recognition via Empirical Maximum Mean Discrepancy Temporal Archetypal Analysis for Action Segmentation Watch-Bot: Unsupervised learning for reminding humans of forgotten actions Lifelong learning of human actions with deep neural network self-organization Exploiting Visual-Spatial First-Person Co-Occurrence for Action-Object Detection without Labels Extension of the Action Verb Corpus for Supervised Learning REPRESENTATION LEARNING FOR ACTION RECOGNITION Watch-n-patch: Unsupervised understanding of actions and relations
40cc595dce9fb49c8ca7579cd247344c1291e171 488ea9b2989f278211306b4bae461acb79a97bd6 a917373133905bde38fba221a80a14862480a1d6 83d5de4a3f61c8d4d866a3fb7597cca42923fa4a eef9181403eef3ad3a6cf2fed9d0779c1dc1968f f7ffce549d89b5529bc414a68be47b7795e27fe6 b05fb929a3bc6fc203ef12c8379a06e850c940a8 18f28959388f40902530f51a2fce665a058c50f2 d45d5408e6652b3159df7fc6321e4a2d3eb86b8b f4537684ce28246b8f29e44a4f6277c204399037 6291e49d4e7c30272a43a212e14aec9fe37dac38 377c744076fffde18d9d973fc7c793e8bdc80693 b28af73e426f256e5c05258a94a4b1aa1113bd1e 2ef69c67e3129526ad616985a38fbd51f360d218 61367718a07bafbd696871be25e11815f1dd85c4 2192a07e1581387edd8a3862962846b50c8f5bf0 d360ad80158a86536fcef3baf0ebb95cdd30c439 4b9b4011fd367a3b36444fb79d3f25f2323bd8e4 e3066b0ee3ecf8ddf26e67c1d330122c26d446a0 980a3f7b41df8045202ea912c4730b439259c1ba a6ca38662a4c8a71094f5d02cb6c11a83b131b0f 8734bbee53336a0f1ca6ade80f9e323d227a528e c56c2b144cbf34b2885ebc30ed7e1f417b44aa9f 263d5c0e8e346b794fe97bb1972ef6944da7a69d 4abe861402bff88f239b9664131636adcf5571b6 735616d38e0324fab3fc15097dad718fb5191af8 e25c57a6395b7c8861088699eef1a56b6f1d70ad 8985a40c96a9e4c1d9f1137bd462b0dc5f81f4b2 57c59011614c43f51a509e10717e47505c776389 2287df86c9253c4a785905bf20a8e178156f8ca6 0e50cbd929432ed22b7d81165f177e3135951d77 a0ab4d78d54d14b70179d85e2208ade2053c22c9 2260d9299fac1cefaada690a322cb424b5ce7d16 5bb0a1f387f17a5ec7cef8d36e80e5ea08d6cc92 d91632867822fc95a00348aeca3a3ea7844ab35a e29dcfb090fdba51fae4f21dd34e93e6efa29fdf 62d8bbe48f623f130d243a38674cf77701fbbdb4 c869be0c485d83e80e6409346f20c45791ae2e11
["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science","Engineering"] ["Computer Science","Medicine"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Psychology","Computer Science"] ["Computer Science","Medicine","Psychology"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science","Engineering"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Mathematics","Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science","Engineering"] ["Medicine","Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"]
2019 2013 2011 2018 2014 2019 2015 2015 2014 2016 2013 2011 2009 2020 2019 2006 2013 2013 2012 2010 2011 2008 2019 2019 2018 2004 2017 2006 2017 2012 2018 2017 2015 2017 2016 2018 2018 2015
IWANN (2) iThings/GreenCom/CPSCom/SmartData CICARE CHI PLAY SAS EUCognition ICCV Workshops ICCV ACIIDS (1) Robotics: Science and Systems ICCV CVPR MB MLMI IJCAI CIARP CVPR Workshops ICMLA (1) FG CVPR
https://api.semanticscholar.org/40cc595dce9fb49c8ca7579cd247344c1291e171 https://api.semanticscholar.org/488ea9b2989f278211306b4bae461acb79a97bd6 https://api.semanticscholar.org/a917373133905bde38fba221a80a14862480a1d6 https://api.semanticscholar.org/83d5de4a3f61c8d4d866a3fb7597cca42923fa4a https://api.semanticscholar.org/eef9181403eef3ad3a6cf2fed9d0779c1dc1968f https://api.semanticscholar.org/f7ffce549d89b5529bc414a68be47b7795e27fe6 https://api.semanticscholar.org/b05fb929a3bc6fc203ef12c8379a06e850c940a8 https://api.semanticscholar.org/18f28959388f40902530f51a2fce665a058c50f2 https://api.semanticscholar.org/d45d5408e6652b3159df7fc6321e4a2d3eb86b8b https://api.semanticscholar.org/f4537684ce28246b8f29e44a4f6277c204399037 https://api.semanticscholar.org/6291e49d4e7c30272a43a212e14aec9fe37dac38 https://api.semanticscholar.org/377c744076fffde18d9d973fc7c793e8bdc80693 https://api.semanticscholar.org/b28af73e426f256e5c05258a94a4b1aa1113bd1e https://api.semanticscholar.org/2ef69c67e3129526ad616985a38fbd51f360d218 https://api.semanticscholar.org/61367718a07bafbd696871be25e11815f1dd85c4 https://api.semanticscholar.org/2192a07e1581387edd8a3862962846b50c8f5bf0 https://api.semanticscholar.org/d360ad80158a86536fcef3baf0ebb95cdd30c439 https://api.semanticscholar.org/4b9b4011fd367a3b36444fb79d3f25f2323bd8e4 https://api.semanticscholar.org/e3066b0ee3ecf8ddf26e67c1d330122c26d446a0 https://api.semanticscholar.org/980a3f7b41df8045202ea912c4730b439259c1ba https://api.semanticscholar.org/a6ca38662a4c8a71094f5d02cb6c11a83b131b0f https://api.semanticscholar.org/8734bbee53336a0f1ca6ade80f9e323d227a528e https://api.semanticscholar.org/c56c2b144cbf34b2885ebc30ed7e1f417b44aa9f https://api.semanticscholar.org/263d5c0e8e346b794fe97bb1972ef6944da7a69d https://api.semanticscholar.org/4abe861402bff88f239b9664131636adcf5571b6 https://api.semanticscholar.org/735616d38e0324fab3fc15097dad718fb5191af8 https://api.semanticscholar.org/e25c57a6395b7c8861088699eef1a56b6f1d70ad https://api.semanticscholar.org/8985a40c96a9e4c1d9f1137bd462b0dc5f81f4b2 https://api.semanticscholar.org/57c59011614c43f51a509e10717e47505c776389 https://api.semanticscholar.org/2287df86c9253c4a785905bf20a8e178156f8ca6 https://api.semanticscholar.org/0e50cbd929432ed22b7d81165f177e3135951d77 https://api.semanticscholar.org/a0ab4d78d54d14b70179d85e2208ade2053c22c9 https://api.semanticscholar.org/2260d9299fac1cefaada690a322cb424b5ce7d16 https://api.semanticscholar.org/5bb0a1f387f17a5ec7cef8d36e80e5ea08d6cc92 https://api.semanticscholar.org/d91632867822fc95a00348aeca3a3ea7844ab35a https://api.semanticscholar.org/e29dcfb090fdba51fae4f21dd34e93e6efa29fdf https://api.semanticscholar.org/62d8bbe48f623f130d243a38674cf77701fbbdb4 https://api.semanticscholar.org/c869be0c485d83e80e6409346f20c45791ae2e11