We introduce a generative system for “ avoidance drawings ” , drawings made by virtual drawing robots executing a random walk while simultaneously avoiding the paths of other robots . The random walk method is unique and is based on a curvature controlling scheme initially introduced by Chappell . We assert that current approaches are limited by their ability to capture the temporal and spatial patterns contained within EEG . In order to address these problems , we propose a new generative technique for EEG classification that uses Elman Recurrent Neural Networks . EEG recorded while a subject performs one of several imagined mental tasks is first modeled by training a network to forecast the signal a single step ahead in time . We propose a novel unsupervised topic model for contrastive opinion modeling . It simulates the generative process of how opinion words occur in the documents of different collections . The ad hoc opinion search process can be efficiently accomplished based on the learned parameters in the model . Series editors ' preface Introduction Susan M. Gass and Jacquelyn Schachter Part I. Theories of Acquisition : 1 . Second language acquisition theory : the case for a generative perspective Kevin R. Gregg 2 . What is the logical problem of foreign language learning ? Basing on the up-to date theories in generative grammar the author analyses the emergency of DP ( determiner phrase ) hypothesis in the English language . The investigation highlights main points on the NP structure through generative perspective . The author gives examples of the main models of NP structure using traditional and generative approaches . Once this phase is complete , the model makes decisions without any recursion . The models are both more efficient and more fallible than the above generative ones . Our model also offers a way to reconcile previous expla- nations of the interpretation or production of pragmatically complex utterances : slowly via complex recursive inferences made as each sentence is processed , ( Geurts , 2009 ; Huang & Snedeker , 2009 ) or quickly via inferences that are pre- compiled and cached based on previous interactions ( Levin- son , 2000 ; Grodner et al. , 2010 ; Smith et al. , 2013 ) . Despite the fact that generative models have been frequently employed in a semi-supervised setting since the early days of the statistical revolution in NLP , we advocate the use of discriminative models . The ability of discriminative models to handle complex , high-dimensional feature spaces and their strong theoretical guarantees have made them a very appealing alternative to their generative counterparts . Perhaps more importantly , discriminative models have been shown to offer competitive performance on a variety of sequential and structured learning tasks in NLP that are traditionally tackled via generative models , such as letter-to-phoneme conversion ( Jiampojamarn et al. , 2008 ) , semantic role labeling ( Toutanova et al. , 2005 ) , syntactic parsing ( Taskar et al. , 2004 ) , language modeling ( Roark et al. , 2004 ) , and machine translation ( Liang et al. , 2006 ) . The complexity of this problem reasons from the high correlativity between the editing operations and vision perception . Essentially , a significant requirement that should be emphasized is how to make the object modifications hard to be found visually in the generative counterparts . In this article , we propose a novel dual-domain perceptual framework to generate visual inconspicuous counterparts , which applies the perceptual bidirectional similarity metric ( PBSM ) and appearance similarity metric ( ASM ) to create the dual-domain perception error minimization model . Generative-based models can handle better an open conversation which makes them a more generalizable approach . Promising results have been achieved in generative-based models by applying neural machine translation techniques with the recurrent encoder/decoder architecture . In this project is implemented , compared and analyzed two generative models that constitute the state of the art in neural machine translation applied to chatbots . We found that removing the scale and bias parameters from batchnorm produced better results for both models . We speculate that the noise introduced by batchnorm helps the generative models to better explore and generate from the underlying data distribution . The results are shown in Table [ reference ] which compares our models with other techniques . This inference procedure is differentiable . Other generative models either have non - differentiable inference procedures , making it harder to compute adversarial examples , or require an additional non - generative discriminator model to get good classification accuracy on MNIST . In the case of the MP - DBM , we can be sure that the generative model itself is responding to adversarial examples , rather than the non - generative classifier model on top . In fact , the observations are such that the rating frequencies of users and movies closely follow power law distributions . In our experiment , we assume a simple generative process where users and movies are independently sampled from a power law , that is . This is a very sparse distribution with fixed expected number of observations , so we repeat this process identically times in order to control the overall density . To this end , several works have attempted to factorize a GAN into a series of GANs , decomposing the difficult task into several more tractable sub - tasks . Denton propose a LAPGAN model that factorizes the generative process into multi - resolution GANs , with each GAN generating a higher - resolution residual conditioned on a lower - resolution image . Although both LAPGAN and SGAN consist of a sequence of GANs each working at one scale , LAPGAN focuses on generating multi - resolution images from coarse to fine while our SGAN aims at modeling multi - level representations from abstract to specific . This is susceptible to a phenomenon called spillover , where the error in a single input can propagate through the network , corrupting the coding and biasing the entire reconstructed data . A denoising autoencoder is a more robust variation on the traditional autoencoder , trained to remove noise and build an error-free reconstruction . We created a denoising autoencoder to utilize the noise removal on corrupted inputs , and rebuild from working inputs . The method of infrared small target detection is a crucial technology for infrared early-warning tasks , infrared imaging guidance , and large field of view target monitoring , and it is very important for certain early-warning tasks . In this paper , we propose an end-to-end infrared small target detection model ( called CDAE ) based on denoising autoencoder network and convolutional neural network , which treats small targets as “ noise ” in infrared images and transforms small target detection tasks into denoising problems . In addition , we use the perceptual loss to solve the problem of background texture feature loss in the encoding process , and propose the structural loss to make up for the perceptual loss defect in which small targets appear . Our approach consists of two steps . First , we learn a lower-dimensional representation of human motion using a denoising autoencoder neural network , consisting of a motion encoder MotionE and a motion decoder MotionD. The learned representation preserves the most important aspects of the human pose variation while removing less relevant variation . Second , we train a novel encoder network SpeechE to map from speech to a corresponding motion representation with reduced dimensionality . Pointing to the noise problems , this paper proposed a denoising autoencoder neural network ( DAE ) algorithm which can not only oversample minority class sample through misclassification cost , but it can denoise and classify the sampled dataset . Through experiments , compared with the denoising autoencoder neural network ( DAE ) with oversampling process and traditional fully connected neural networks , the results showed the proposed algorithm improves the classification accuracy of minority class of imbalanced datasets . After watching each video clips , their facial expressions and speech responses are collected when they are interviewing with a clinician . In mood disorder detection , the facial action unit ( AU ) profiles and speech emotion profiles ( EPs ) are obtained , respectively , by using the support vector machines ( SVMs ) which are built via facial features and speech features adapted from two selected databases using a denoising autoencoder-based method . Finally , a Coupled Hidden Markov Model (CHMM)-based fusion method is proposed to characterize the temporal information . This has resulted in the interest in applying deep learning methods to recommender systems . This study also proposes another novel method based on the denoising autoencoder architecture , which is referred to as the Attentive Contextual Denoising Autoencoder ( ACDA ) . The ACDA model augments the basic denoising autoencoder with a context-driven attention mechanism to provide personalized recommendation . In this paper , we introduce a new pre-training approach for neural networks , applied in LSTM-based postfilters for speech synthesis , with the objective of enhancing the quality of the synthesized speech in a more efficient manner . Our approach begins with an auto-regressive training of one LSTM network , whose is used as an initialization for postfilters based on a denoising autoencoder architecture . We show the advantages of this initialization on a set of multi-stream postfilters , which encompass a collection of denoising autoencoders for the set of MFCC and fundamental frequency parameters of the artificial voice . Through the choice of an appropriate A-matrix , the capacity of the model can be steered towards modeling relevant information for the classification task . Comparative experiments with the proposed denoising autoencoder variant show that this way of proceeding yields extracted features that achieve better classification performance on several datasets . Second , we propose a Relational Autoencoder model for unsupervised learning of features for action and scene retrieval . Finally , a Denoising Autoencoder variant is presented to infer missing people in the scene from their context . Empirical results demonstrate that this approach learns relational feature representations that can effectively discriminate person and group activity classes . In this paper , a denoising autoencoder approach is proposed for the training process for neural networks . The denoising-autoencoder-based neural network model is then applied to credit risk analysis , and the performance is evaluated . In this paper , we explore the potential of using deep learning for extracting speaker-dependent features for noise robust speaker identification . More specifically , an SNR-adaptive denoising classifier is constructed by stacking two layers of restricted Boltzmann machines ( RBMs ) on top of a denoising deep autoencoder , where the top-RBM layer is connected to a soft-max output layer that outputs the posterior probabilities of speakers and the top-RBM layer outputs speaker-dependent bottleneck features . Both the deep autoencoder and RBMs are trained by contrastive divergence , followed by backpropagation fine-tuning . subsubsection : Autoencoder Learning To evaluate ELU networks at unsupervised settings , we followed and and trained a deep autoencoder on the MNIST dataset . The encoder part consisted of four fully connected hidden layers with sizes 1000 , 500 , 250 and 30 , respectively . [ reference ] . That literature has mostly focused on exploiting the principle of robust representations , based on the denoising autoencoder paradigm [ reference ] . Concurrently , multiple methods of matching the feature distributions in the source and the target domains have been proposed for unsupervised domain adaptation . Among these methods , [ reference ] does this in a " deep " way by the layerwise training of a sequence of deep autoencoders , while gradually replacing source - domain samples with target - domain samples . This improves over a similar approach of [ reference ] that simply trains a single deep autoencoder for both domains . In both approaches , the actual classifier / predictor is learned in a separate step using the feature representation learned by autoencoder ( s ) . As the only difference between DANN and NN is the domain adaptation regularizer , we conclude that our approach successfully helps to find a representation suitable for the target domain . section : Combining DANN with Denoising Autoencoders We now investigate on whether the DANN algorithm can improve on the representation learned by the state - of - the - art Marginalized Stacked Denoising Autoencoders ( mSDA ) proposed by [ reference ] . It takes the unlabeled parts of both source and target samples to learn a feature map from input space X to a new representation space . As a denoising autoencoders algorithm , it finds a feature representation from which one can ( approximately ) reconstruct the original features of an example from its noisy counterpart . [ reference ] showed that using mSDA with a linear SVM classifier reaches state - of - the - art performance on the Amazon reviews data sets . The training objective is to reconstruct the input after passing through a low - dimensional bottleneck , referred to as the latent representation with : The per - sample loss is simply a sum over the pixel - wise L2 distance The resulting latent space can , for example , be used for unsupervised clustering . Denoising Autoencoders have a modified training procedure . Here , artificial random noise is applied to the input images while the reconstruction target stays clean . Some approaches have attempted to minimize human involvement by developing weakly - supervised schemes , making use of image silhouettes to conduct 3D object reconstruction . Of the few unsupervised neural - based approaches that exist , the T - L network is one of the most important , combining a convolutional autoencoder with an image regressor to encode a unified vector representation of a given 2D image . However , one fundamental issue with the T - L Network is its three - phase training procedure , since jointly training the system components proves to be too difficult . • For the denoising and variational autoencoder , the multinomial likelihood compares favorably over the more common Gaussian and logistic likelihoods . • Aiming to remove this dependency on parallel data , a recent research line has managed to train unsupervised machine translation systems using monolingual corpora only . The first such systems were based on Neural Machine Translation ( NMT ) , and combined denoising autoencoding and back - translation to train a dual model initialized with cross - lingual embeddings artetxe2018unmt , lample2018unsupervised . Nevertheless , these early systems were later superseded by Statistical Machine Translation ( SMT ) based approaches , which induced an initial phrase - table through cross - lingual embedding mappings , combined it with an n - gram language model , and further improved the system through iterative back - translation lample2018phrase , artetxe2018usmt . The main goal is to encode temporal data into a global and low-dimensional latent vector that captures the dynamics of the spatio-temporal signal . To this end , we incorporate auto-regressive RNNs , Wasserstein GAN loss , spectral norm weight constraints and a semi-supervised learning scheme into InfoGAN , a method for retrieval of meaningful latents in adversarial learning . To demonstrate the modeling capability of our method , we encode full-body skeletal human motion from a large dataset representing 60 classes of daily activities , recorded in a multi-Kinect setup . In this paper , we address the classification of heartbeats images in different cardiovascular diseases . We propose a two-dimensional Convolutional Neural Network for classification after using a InfoGAN architecture for generating synthetic images to unbalanced classes . We call this proposal Adversarial Oversampling and compare it with the classical oversampling methods as SMOTE , ADASYN , and Random Oversampling . While images are commonly generated randomly for a content-based augmentation , we leverage ideas from the domain of active learning . Using a network which is based on the InfoGAN architecture allows mapping existing vehicle images to a latent space representation . After mapping the complete training dataset , we perform the augmentation in the latent space . If any interpretable representation is obtained for the latent space , we can characterize the abnormality by the latent variable . In this study , Causal InfoGAN is trained by normal time series data to acquire temporal state variables in the latent space . Then , a temporal abnormality can be characterized by an abnormal transition in the state space . The models learn a small set of intuitive parameters without the need for labeled data and use them to generate new realistic trajectories . The neural networks , which base on the InfoGAN and beta-VAE architectures , are adapted from the image domain to the time series domain . Although our methods are generally applicable , our experiments focus on lane change maneuvers on highways . While VAEs tend to produce excellent reconstructions but blurry samples , GANs generate sharp but slightly distorted images . In this paper we propose a new model called Variational InfoGAN ( ViGAN ) . Our aim is two fold : ( i ) To generated new images conditioned on visual descriptions , and ( ii ) modify the image , by fixing the latent representation of image and varying the visual description . This paper proposes a novel speech enhancement model based on generative adversarial network ( GAN ) . The proposed method additionally uses cycle-consistency loss for learning on non-parallel datasets , where the InfoGAN mechanism is used to cluster noise information in an unsupervised learning manner . The proposed model can form cluster-specific mapping by using the obtained clustering information . In recent years , progress in deep generative models has produced methods that learn to ` imagine ' plausible images from data . Building on the recent Causal InfoGAN generative model , in this work we learn to imagine goal-directed object manipulation directly from raw image data of self-supervised interaction of the robot with the object . After learning , given a goal observation of the system , our model can generate an imagined plan -- a sequence of images that transition the object into the desired goal . Recently , the unsupervised setting has been explored . InfoGAN achieves disentanglement by maximizing the mutual information between latent variables and data variation . Similar to DrNet that separates time - independent and time - varying components with an adversarial loss , we apply a content adversarial loss to disentangle an image into domain - invariant and domain - specific representations to facilitate learning diverse cross - domain mappings . However , in these approaches the representations follow a simple prior , e.g. a Gaussian or uniform distribution , and do not exhibit any disentangled properties . Our model , the Bidirectional - InfoGAN , integrates some of these approaches by extending traditional GANs with an encoder that learns disentangled representations in an unsupervised setting . After training , the encoder can map data points to meaningful , disentangled representations which can potentially be used for different tasks such as classification , clustering , or image captioning . Compared to the InfoGAN we introduce an encoder to mitigate the problems of using a discriminator for both the adversarial loss and the inference task . Unlike the Structured GAN our training procedure is completely unsupervised , can detect unknown data - generating factors , and only introduces one additional inference network ( the encoder ) . In addition , presented to analyze the GAN from information theoretical perspective , and they seek to minimize the variational estimate of f - divergence , and show that the classic GAN is included as a special case of f - GAN . In contrast , InfoGAN proposed another information - theoretic GAN to learn disentangled representations capturing various latent concepts and factors in generating samples . Most recently , propose to minimize the Earth - Mover distance between the density of generated samples and the true data density , and they show the resultant Wasserstein GAN ( WGAN ) can address the vanishing gradient problem that the classic GAN suffers . We then computed the test error based on the assigned class labels to each cluster . As shown in the first column of le : semi ] Table [ reference ] , the performance of PixelGAN autoencoders is on par with other GAN - based clustering algorithms such as CatGAN catgan , InfoGAN infogan and adversarial autoencoders aae . subsection : Semi - supervised Classification StyleGAN2 is a state-of-the-art network in generating realistic images . Besides , it was explicitly trained to have disentangled directions in latent space , which allows efficient image manipulation by varying latent factors . We provide results of human faces ' transformation : gender swap , aging/rejuvenation , style transfer and image morphing . We show that the quality of generation using our method is comparable to StyleGAN2 backpropagation and current state-of-the-art methods in these particular tasks . We use the Frchet Inception Distance ( FID ) metric , Domain Frchet Distance ( DFD ) , and blind tests with a radiology fellow for evaluation . We demonstrate that StyleGAN2 provides the best performance in generating wbMRI images with respect to all three metrics . This results in the meaningful manipulation of sets of features that correspond to the generation of a broad array of semantically significant aspects of the generated images . We demonstrate these transformations on the official pre-trained StyleGAN2 model trained on the FFHQ dataset . In doing so , we lay the groundwork for future interactive multimedia systems where the inner representation of deep generative models are manipulated for greater creative expression , whilst also increasing our understanding of how such " black-box systems " can be more meaningfully interpreted .
We introduce a generative system for “ avoidance drawings ” , drawings made by virtual drawing robots executing a random walk while simultaneously avoiding the paths of other robots . The random walk method is unique and is based on a curvature controlling scheme initially introduced by Chappell .
We assert that current approaches are limited by their ability to capture the temporal and spatial patterns contained within EEG . In order to address these problems , we propose a new generative technique for EEG classification that uses Elman Recurrent Neural Networks . EEG recorded while a subject performs one of several imagined mental tasks is first modeled by training a network to forecast the signal a single step ahead in time .
We propose a novel unsupervised topic model for contrastive opinion modeling . It simulates the generative process of how opinion words occur in the documents of different collections . The ad hoc opinion search process can be efficiently accomplished based on the learned parameters in the model .
Series editors ' preface Introduction Susan M. Gass and Jacquelyn Schachter Part I. Theories of Acquisition : 1 . Second language acquisition theory : the case for a generative perspective Kevin R. Gregg 2 . What is the logical problem of foreign language learning ?
Basing on the up-to date theories in generative grammar the author analyses the emergency of DP ( determiner phrase ) hypothesis in the English language . The investigation highlights main points on the NP structure through generative perspective . The author gives examples of the main models of NP structure using traditional and generative approaches .
Once this phase is complete , the model makes decisions without any recursion . The models are both more efficient and more fallible than the above generative ones . Our model also offers a way to reconcile previous expla- nations of the interpretation or production of pragmatically complex utterances : slowly via complex recursive inferences made as each sentence is processed , ( Geurts , 2009 ; Huang & Snedeker , 2009 ) or quickly via inferences that are pre- compiled and cached based on previous interactions ( Levin- son , 2000 ; Grodner et al. , 2010 ; Smith et al. , 2013 ) .
Despite the fact that generative models have been frequently employed in a semi-supervised setting since the early days of the statistical revolution in NLP , we advocate the use of discriminative models . The ability of discriminative models to handle complex , high-dimensional feature spaces and their strong theoretical guarantees have made them a very appealing alternative to their generative counterparts . Perhaps more importantly , discriminative models have been shown to offer competitive performance on a variety of sequential and structured learning tasks in NLP that are traditionally tackled via generative models , such as letter-to-phoneme conversion ( Jiampojamarn et al. , 2008 ) , semantic role labeling ( Toutanova et al. , 2005 ) , syntactic parsing ( Taskar et al. , 2004 ) , language modeling ( Roark et al. , 2004 ) , and machine translation ( Liang et al. , 2006 ) .
The complexity of this problem reasons from the high correlativity between the editing operations and vision perception . Essentially , a significant requirement that should be emphasized is how to make the object modifications hard to be found visually in the generative counterparts . In this article , we propose a novel dual-domain perceptual framework to generate visual inconspicuous counterparts , which applies the perceptual bidirectional similarity metric ( PBSM ) and appearance similarity metric ( ASM ) to create the dual-domain perception error minimization model .
Generative-based models can handle better an open conversation which makes them a more generalizable approach . Promising results have been achieved in generative-based models by applying neural machine translation techniques with the recurrent encoder/decoder architecture . In this project is implemented , compared and analyzed two generative models that constitute the state of the art in neural machine translation applied to chatbots .
We found that removing the scale and bias parameters from batchnorm produced better results for both models . We speculate that the noise introduced by batchnorm helps the generative models to better explore and generate from the underlying data distribution . The results are shown in Table [ reference ] which compares our models with other techniques .
This inference procedure is differentiable . Other generative models either have non - differentiable inference procedures , making it harder to compute adversarial examples , or require an additional non - generative discriminator model to get good classification accuracy on MNIST . In the case of the MP - DBM , we can be sure that the generative model itself is responding to adversarial examples , rather than the non - generative classifier model on top .
In fact , the observations are such that the rating frequencies of users and movies closely follow power law distributions . In our experiment , we assume a simple generative process where users and movies are independently sampled from a power law , that is . This is a very sparse distribution with fixed expected number of observations , so we repeat this process identically times in order to control the overall density .
To this end , several works have attempted to factorize a GAN into a series of GANs , decomposing the difficult task into several more tractable sub - tasks . Denton propose a LAPGAN model that factorizes the generative process into multi - resolution GANs , with each GAN generating a higher - resolution residual conditioned on a lower - resolution image . Although both LAPGAN and SGAN consist of a sequence of GANs each working at one scale , LAPGAN focuses on generating multi - resolution images from coarse to fine while our SGAN aims at modeling multi - level representations from abstract to specific .
This is susceptible to a phenomenon called spillover , where the error in a single input can propagate through the network , corrupting the coding and biasing the entire reconstructed data . A denoising autoencoder is a more robust variation on the traditional autoencoder , trained to remove noise and build an error-free reconstruction . We created a denoising autoencoder to utilize the noise removal on corrupted inputs , and rebuild from working inputs .
The method of infrared small target detection is a crucial technology for infrared early-warning tasks , infrared imaging guidance , and large field of view target monitoring , and it is very important for certain early-warning tasks . In this paper , we propose an end-to-end infrared small target detection model ( called CDAE ) based on denoising autoencoder network and convolutional neural network , which treats small targets as “ noise ” in infrared images and transforms small target detection tasks into denoising problems . In addition , we use the perceptual loss to solve the problem of background texture feature loss in the encoding process , and propose the structural loss to make up for the perceptual loss defect in which small targets appear .
Our approach consists of two steps . First , we learn a lower-dimensional representation of human motion using a denoising autoencoder neural network , consisting of a motion encoder MotionE and a motion decoder MotionD. The learned representation preserves the most important aspects of the human pose variation while removing less relevant variation . Second , we train a novel encoder network SpeechE to map from speech to a corresponding motion representation with reduced dimensionality .
Pointing to the noise problems , this paper proposed a denoising autoencoder neural network ( DAE ) algorithm which can not only oversample minority class sample through misclassification cost , but it can denoise and classify the sampled dataset . Through experiments , compared with the denoising autoencoder neural network ( DAE ) with oversampling process and traditional fully connected neural networks , the results showed the proposed algorithm improves the classification accuracy of minority class of imbalanced datasets .
After watching each video clips , their facial expressions and speech responses are collected when they are interviewing with a clinician . In mood disorder detection , the facial action unit ( AU ) profiles and speech emotion profiles ( EPs ) are obtained , respectively , by using the support vector machines ( SVMs ) which are built via facial features and speech features adapted from two selected databases using a denoising autoencoder-based method . Finally , a Coupled Hidden Markov Model (CHMM)-based fusion method is proposed to characterize the temporal information .
This has resulted in the interest in applying deep learning methods to recommender systems . This study also proposes another novel method based on the denoising autoencoder architecture , which is referred to as the Attentive Contextual Denoising Autoencoder ( ACDA ) . The ACDA model augments the basic denoising autoencoder with a context-driven attention mechanism to provide personalized recommendation .
In this paper , we introduce a new pre-training approach for neural networks , applied in LSTM-based postfilters for speech synthesis , with the objective of enhancing the quality of the synthesized speech in a more efficient manner . Our approach begins with an auto-regressive training of one LSTM network , whose is used as an initialization for postfilters based on a denoising autoencoder architecture . We show the advantages of this initialization on a set of multi-stream postfilters , which encompass a collection of denoising autoencoders for the set of MFCC and fundamental frequency parameters of the artificial voice .
Through the choice of an appropriate A-matrix , the capacity of the model can be steered towards modeling relevant information for the classification task . Comparative experiments with the proposed denoising autoencoder variant show that this way of proceeding yields extracted features that achieve better classification performance on several datasets .
Second , we propose a Relational Autoencoder model for unsupervised learning of features for action and scene retrieval . Finally , a Denoising Autoencoder variant is presented to infer missing people in the scene from their context . Empirical results demonstrate that this approach learns relational feature representations that can effectively discriminate person and group activity classes .
In this paper , a denoising autoencoder approach is proposed for the training process for neural networks . The denoising-autoencoder-based neural network model is then applied to credit risk analysis , and the performance is evaluated .
In this paper , we explore the potential of using deep learning for extracting speaker-dependent features for noise robust speaker identification . More specifically , an SNR-adaptive denoising classifier is constructed by stacking two layers of restricted Boltzmann machines ( RBMs ) on top of a denoising deep autoencoder , where the top-RBM layer is connected to a soft-max output layer that outputs the posterior probabilities of speakers and the top-RBM layer outputs speaker-dependent bottleneck features . Both the deep autoencoder and RBMs are trained by contrastive divergence , followed by backpropagation fine-tuning .
subsubsection : Autoencoder Learning To evaluate ELU networks at unsupervised settings , we followed and and trained a deep autoencoder on the MNIST dataset . The encoder part consisted of four fully connected hidden layers with sizes 1000 , 500 , 250 and 30 , respectively .
[ reference ] . That literature has mostly focused on exploiting the principle of robust representations , based on the denoising autoencoder paradigm [ reference ] . Concurrently , multiple methods of matching the feature distributions in the source and the target domains have been proposed for unsupervised domain adaptation .
Among these methods , [ reference ] does this in a " deep " way by the layerwise training of a sequence of deep autoencoders , while gradually replacing source - domain samples with target - domain samples . This improves over a similar approach of [ reference ] that simply trains a single deep autoencoder for both domains . In both approaches , the actual classifier / predictor is learned in a separate step using the feature representation learned by autoencoder ( s ) .
As the only difference between DANN and NN is the domain adaptation regularizer , we conclude that our approach successfully helps to find a representation suitable for the target domain . section : Combining DANN with Denoising Autoencoders We now investigate on whether the DANN algorithm can improve on the representation learned by the state - of - the - art Marginalized Stacked Denoising Autoencoders ( mSDA ) proposed by [ reference ] .
It takes the unlabeled parts of both source and target samples to learn a feature map from input space X to a new representation space . As a denoising autoencoders algorithm , it finds a feature representation from which one can ( approximately ) reconstruct the original features of an example from its noisy counterpart . [ reference ] showed that using mSDA with a linear SVM classifier reaches state - of - the - art performance on the Amazon reviews data sets .
The training objective is to reconstruct the input after passing through a low - dimensional bottleneck , referred to as the latent representation with : The per - sample loss is simply a sum over the pixel - wise L2 distance The resulting latent space can , for example , be used for unsupervised clustering . Denoising Autoencoders have a modified training procedure . Here , artificial random noise is applied to the input images while the reconstruction target stays clean .
Some approaches have attempted to minimize human involvement by developing weakly - supervised schemes , making use of image silhouettes to conduct 3D object reconstruction . Of the few unsupervised neural - based approaches that exist , the T - L network is one of the most important , combining a convolutional autoencoder with an image regressor to encode a unified vector representation of a given 2D image . However , one fundamental issue with the T - L Network is its three - phase training procedure , since jointly training the system components proves to be too difficult .
• For the denoising and variational autoencoder , the multinomial likelihood compares favorably over the more common Gaussian and logistic likelihoods . •
Aiming to remove this dependency on parallel data , a recent research line has managed to train unsupervised machine translation systems using monolingual corpora only . The first such systems were based on Neural Machine Translation ( NMT ) , and combined denoising autoencoding and back - translation to train a dual model initialized with cross - lingual embeddings artetxe2018unmt , lample2018unsupervised . Nevertheless , these early systems were later superseded by Statistical Machine Translation ( SMT ) based approaches , which induced an initial phrase - table through cross - lingual embedding mappings , combined it with an n - gram language model , and further improved the system through iterative back - translation lample2018phrase , artetxe2018usmt .
The main goal is to encode temporal data into a global and low-dimensional latent vector that captures the dynamics of the spatio-temporal signal . To this end , we incorporate auto-regressive RNNs , Wasserstein GAN loss , spectral norm weight constraints and a semi-supervised learning scheme into InfoGAN , a method for retrieval of meaningful latents in adversarial learning . To demonstrate the modeling capability of our method , we encode full-body skeletal human motion from a large dataset representing 60 classes of daily activities , recorded in a multi-Kinect setup .
In this paper , we address the classification of heartbeats images in different cardiovascular diseases . We propose a two-dimensional Convolutional Neural Network for classification after using a InfoGAN architecture for generating synthetic images to unbalanced classes . We call this proposal Adversarial Oversampling and compare it with the classical oversampling methods as SMOTE , ADASYN , and Random Oversampling .
While images are commonly generated randomly for a content-based augmentation , we leverage ideas from the domain of active learning . Using a network which is based on the InfoGAN architecture allows mapping existing vehicle images to a latent space representation . After mapping the complete training dataset , we perform the augmentation in the latent space .
If any interpretable representation is obtained for the latent space , we can characterize the abnormality by the latent variable . In this study , Causal InfoGAN is trained by normal time series data to acquire temporal state variables in the latent space . Then , a temporal abnormality can be characterized by an abnormal transition in the state space .
The models learn a small set of intuitive parameters without the need for labeled data and use them to generate new realistic trajectories . The neural networks , which base on the InfoGAN and beta-VAE architectures , are adapted from the image domain to the time series domain . Although our methods are generally applicable , our experiments focus on lane change maneuvers on highways .
While VAEs tend to produce excellent reconstructions but blurry samples , GANs generate sharp but slightly distorted images . In this paper we propose a new model called Variational InfoGAN ( ViGAN ) . Our aim is two fold : ( i ) To generated new images conditioned on visual descriptions , and ( ii ) modify the image , by fixing the latent representation of image and varying the visual description .
This paper proposes a novel speech enhancement model based on generative adversarial network ( GAN ) . The proposed method additionally uses cycle-consistency loss for learning on non-parallel datasets , where the InfoGAN mechanism is used to cluster noise information in an unsupervised learning manner . The proposed model can form cluster-specific mapping by using the obtained clustering information .
In recent years , progress in deep generative models has produced methods that learn to ` imagine ' plausible images from data . Building on the recent Causal InfoGAN generative model , in this work we learn to imagine goal-directed object manipulation directly from raw image data of self-supervised interaction of the robot with the object . After learning , given a goal observation of the system , our model can generate an imagined plan -- a sequence of images that transition the object into the desired goal .
Recently , the unsupervised setting has been explored . InfoGAN achieves disentanglement by maximizing the mutual information between latent variables and data variation . Similar to DrNet that separates time - independent and time - varying components with an adversarial loss , we apply a content adversarial loss to disentangle an image into domain - invariant and domain - specific representations to facilitate learning diverse cross - domain mappings .
However , in these approaches the representations follow a simple prior , e.g. a Gaussian or uniform distribution , and do not exhibit any disentangled properties . Our model , the Bidirectional - InfoGAN , integrates some of these approaches by extending traditional GANs with an encoder that learns disentangled representations in an unsupervised setting . After training , the encoder can map data points to meaningful , disentangled representations which can potentially be used for different tasks such as classification , clustering , or image captioning . Compared to the InfoGAN we introduce an encoder to mitigate the problems of using a discriminator for both the adversarial loss and the inference task . Unlike the Structured GAN our training procedure is completely unsupervised , can detect unknown data - generating factors , and only introduces one additional inference network ( the encoder ) .
In addition , presented to analyze the GAN from information theoretical perspective , and they seek to minimize the variational estimate of f - divergence , and show that the classic GAN is included as a special case of f - GAN . In contrast , InfoGAN proposed another information - theoretic GAN to learn disentangled representations capturing various latent concepts and factors in generating samples . Most recently , propose to minimize the Earth - Mover distance between the density of generated samples and the true data density , and they show the resultant Wasserstein GAN ( WGAN ) can address the vanishing gradient problem that the classic GAN suffers .
We then computed the test error based on the assigned class labels to each cluster . As shown in the first column of le : semi ] Table [ reference ] , the performance of PixelGAN autoencoders is on par with other GAN - based clustering algorithms such as CatGAN catgan , InfoGAN infogan and adversarial autoencoders aae . subsection : Semi - supervised Classification
StyleGAN2 is a state-of-the-art network in generating realistic images . Besides , it was explicitly trained to have disentangled directions in latent space , which allows efficient image manipulation by varying latent factors .
We provide results of human faces ' transformation : gender swap , aging/rejuvenation , style transfer and image morphing . We show that the quality of generation using our method is comparable to StyleGAN2 backpropagation and current state-of-the-art methods in these particular tasks .
We use the Frchet Inception Distance ( FID ) metric , Domain Frchet Distance ( DFD ) , and blind tests with a radiology fellow for evaluation . We demonstrate that StyleGAN2 provides the best performance in generating wbMRI images with respect to all three metrics .
This results in the meaningful manipulation of sets of features that correspond to the generation of a broad array of semantically significant aspects of the generated images . We demonstrate these transformations on the official pre-trained StyleGAN2 model trained on the FFHQ dataset . In doing so , we lay the groundwork for future interactive multimedia systems where the inner representation of deep generative models are manipulated for greater creative expression , whilst also increasing our understanding of how such " black-box systems " can be more meaningfully interpreted .
Avoidance Drawings Evolved Using Virtual Drawing Robots Classification of EEG during imagined mental tasks by forecasting with Elman Recurrent Neural Networks Mining contrastive opinions on political texts using cross-perspective topic model Linguistic Perspectives on Second Language Acquisition: Frontmatter Гіпотеза детермінантної фрази як функціональної проекції в генеративній парадигмі Learning to Reason Pragmatically with Cognitive Limitations Discriminative Models for Semi-Supervised Natural Language Learning A Dual-Domain Perceptual Framework for Generating Visual Inconspicuous Counterparts Implementing ChatBots using Neural Machine Translation techniques Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks Explaining and Harnessing Adversarial Examples Matrix Completion on Graphs Stacked Generative Adversarial Networks Denoising autoencoder anomaly detection for correlated data Infrared Dim and Small Target Detection Based on Denoising Autoencoder Network Analyzing Input and Output Representations for Speech-Driven Gesture Generation Credit Card Fraud Detection Using Autoencoder Neural Network Coupled HMM-based multimodal fusion for mood disorder detection through elicited audio–visual signals Machine Learning Models for Context-Aware Recommender Systems Pre-training Long Short-term Memory Neural Networks for Efficient Regression in Artificial Speech Postfiltering Semi Supervised Autoencoders: Better Focusing Model Capacity during Feature Extraction Hierarchical Relational Networks for Group Activity Recognition and Retrieval A Denoising Autoencoder Approach for Credit Risk Analysis Bottleneck features from SNR-adaptive denoising deep classifier for speaker identification Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs) Domain-Adversarial Training of Neural Networks Implicit 3D Orientation Learning for 6D Object Detection from RGB Images Learning a Hierarchical Latent-Variable Model of 3D Shapes Variational Autoencoders for Collaborative Filtering An Effective Approach to Unsupervised Machine Translation Classification of sparsely labeled spatio-temporal data through semi-supervised adversarial learning Heartbeat Anomaly Detection using Adversarial Oversampling VeGAN: Using GANs for Augmentation in Latent Space to Improve the Semantic Segmentation of Vehicles in Images From an Aerial Perspective Analysis of Time Series Anomalies Using Causal InfoGAN and Its Application to Biological Data Data-Driven Maneuver Modeling using Generative Adversarial Networks and Variational Autoencoders for Safety Validation of Highly Automated Vehicles Image Generation and Editing with Variational Info Generative AdversarialNetworks Cycle-Consistent InfoGAN for Speech Enhancement in Various Background Noises Learning Robotic Manipulation through Visual Planning and Acting Diverse Image-to-Image Translation via Disentangled Representations Inferencing based on unsupervised learning of disentangled representations Loss-Sensitive Generative Adversarial Networks on Lipschitz Densities PixelGAN Autoencoders StyleGAN2 Distillation for Feed-forward Image Manipulation Using Generative Models for Pediatric wbMRI Network Bending: Manipulating The Inner Representations of Deep Generative Models
53b325149d6173750ea78afc48def5f874f5fa57 a48b3342710d7e26a29f69efae1d297ef706614e ef401fdf8e856989e7704e3bde7e991a79675d32 aa7cb945d62043589491cb58b1ca70d195ed4e27 939eaceb3fc8b08b4b20e55805595f4b523f25eb 7536b750c6cb972cc2d4e0f9ff39bea4b56ff94c 2889257678cf009c1ca2970408afc4ed90fa2449 0c2122cd651f6499c8b864f04eff68fdd75cdb7f af1dbc6616edcdec69f782a1e36b3e825b04a486 18fa48463e3df6b218256e902dfeebb638787c55 2efde85050f538c3911d062c60a1f8ffad10075b 4c0d70af27e7b1b28c4492b6a1b3798f0ab5d96f 51c5cfa4fa3cacc5dc321fd7b2ae2cdbf9616cfb 2b38d0a6ecef18eb981a567103e4990279d625fd 90a9987fd437740f86c8421f1f147b634d27dbc8 5aa6ec189728b4e46ed4090b9980322683a1aaf0 f3ad764e559d3589063d31efcdf2baac64e6ebfc b984c9815fd556cc845adae1f9a206d2a0fa2d33 e245cf90ae5056a120026d7e322f22a08c310ac4 0c9bf48cc7f702312cbb6c0bdd5d3cdeff6a4a60 b3c7f8947e8b1c85775732d2ae59fb6c680a807c c3d67e96d3e6840948d9c9479b9a8e2fd7f71392 ce1a50c3df6b908846f4d3f750820277ba2def7c b864e0fb46ee3028c8dd57e2893cd9a039d547dd 922b80a8b504b990880058cfac1096f313df556b ea7fbfd02bf17b310e1e7f4be12d106b4990c33d 3de3da897abe7129782af2bcad2f61ec731781b9 0eca090aebd6b5867c523e04df2b6347ee824eab f0ea41e56ab7ac1276f8e639ebad4d0ca472d62d 5a1ccb7c3ecc18d27157966f39ed258ab0d473bf c08fefcd7abd5b4451ecef3dcfcf85aba57e8eae
["Computer Science","Engineering"] ["Computer Science"] ["Computer Science"] ["Psychology","Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Mathematics","Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Mathematics","Computer Science"] ["Computer Science","Mathematics"] ["Computer Science"] ["Computer Science"] ["Engineering","Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science","Engineering"] ["Computer Science"] ["Engineering","Computer Science","Medicine"] ["Computer Science"]
2015 2011 2012 1989 2015 2014 2009 2017 2018 2016 2015 2014 2017 2018 2020 2019 2019 2017 2018 2018 2013 2018 2018 2015 2016 2016 2018 2018 2018 2019 2018 2019 2019 2019 2018 2017 2019 2019 2018 2018 2019 2017 2020 2020 2020
EvoMUSART IJCNN WSDM CogSci ICLR ICLR NIPS 2014 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) IVA IWOBI ICONIP (1) ECCV (3) ICCAI APSIPA ICLR J. Mach. Learn. Res. ECCV 2018 International Conference on 3D Vision (3DV) WWW ACL IJCNN WACV ICNC-FSKD ITSC SITIS Robotics: Science and Systems ECCV ESANN International Journal of Computer Vision NIPS
https://api.semanticscholar.org/53b325149d6173750ea78afc48def5f874f5fa57 https://api.semanticscholar.org/a48b3342710d7e26a29f69efae1d297ef706614e https://api.semanticscholar.org/ef401fdf8e856989e7704e3bde7e991a79675d32 https://api.semanticscholar.org/aa7cb945d62043589491cb58b1ca70d195ed4e27 https://api.semanticscholar.org/939eaceb3fc8b08b4b20e55805595f4b523f25eb https://api.semanticscholar.org/7536b750c6cb972cc2d4e0f9ff39bea4b56ff94c https://api.semanticscholar.org/2889257678cf009c1ca2970408afc4ed90fa2449 https://api.semanticscholar.org/0c2122cd651f6499c8b864f04eff68fdd75cdb7f https://api.semanticscholar.org/af1dbc6616edcdec69f782a1e36b3e825b04a486 https://www.semanticscholar.org/paper/8388f1be26329fa45e5807e968a641ce170ea078 https://www.semanticscholar.org/paper/bee044c8e8903fb67523c1f8c105ab4718600cdb https://www.semanticscholar.org/paper/4c86d29f67ed537d81f6f7714d4d12eab603b128 https://www.semanticscholar.org/paper/9ec59d3b0320fc8abfaf297be45f0da6fdeef3d7 https://api.semanticscholar.org/18fa48463e3df6b218256e902dfeebb638787c55 https://api.semanticscholar.org/2efde85050f538c3911d062c60a1f8ffad10075b https://api.semanticscholar.org/4c0d70af27e7b1b28c4492b6a1b3798f0ab5d96f https://api.semanticscholar.org/51c5cfa4fa3cacc5dc321fd7b2ae2cdbf9616cfb https://api.semanticscholar.org/2b38d0a6ecef18eb981a567103e4990279d625fd https://api.semanticscholar.org/90a9987fd437740f86c8421f1f147b634d27dbc8 https://api.semanticscholar.org/5aa6ec189728b4e46ed4090b9980322683a1aaf0 https://api.semanticscholar.org/f3ad764e559d3589063d31efcdf2baac64e6ebfc https://api.semanticscholar.org/b984c9815fd556cc845adae1f9a206d2a0fa2d33 https://api.semanticscholar.org/e245cf90ae5056a120026d7e322f22a08c310ac4 https://api.semanticscholar.org/0c9bf48cc7f702312cbb6c0bdd5d3cdeff6a4a60 https://www.semanticscholar.org/paper/f63e917638553414526a0cc8550de4ad2d83fe7a https://www.semanticscholar.org/paper/1d5972b32a9b5a455a6eef389de5b7fca25771ad https://www.semanticscholar.org/paper/3fc26991557d28ff720edef2d284ef2663e1e530 https://www.semanticscholar.org/paper/3a8d537bcec370d37990d39eab01c729496ad057 https://www.semanticscholar.org/paper/23f5854b38a15c2ae201e751311665f7995b5e10 https://www.semanticscholar.org/paper/aecddd82840323e5bd43f9c73a32fed88ee93c8c https://api.semanticscholar.org/b3c7f8947e8b1c85775732d2ae59fb6c680a807c https://api.semanticscholar.org/c3d67e96d3e6840948d9c9479b9a8e2fd7f71392 https://api.semanticscholar.org/ce1a50c3df6b908846f4d3f750820277ba2def7c https://api.semanticscholar.org/b864e0fb46ee3028c8dd57e2893cd9a039d547dd https://api.semanticscholar.org/922b80a8b504b990880058cfac1096f313df556b https://api.semanticscholar.org/ea7fbfd02bf17b310e1e7f4be12d106b4990c33d https://api.semanticscholar.org/3de3da897abe7129782af2bcad2f61ec731781b9 https://api.semanticscholar.org/0eca090aebd6b5867c523e04df2b6347ee824eab https://www.semanticscholar.org/paper/3d5d9d8e74b215609eabba80ef79a35ebf460e49 https://www.semanticscholar.org/paper/148eb2b649ce0f66214fceba5c061b417ab88d79 https://www.semanticscholar.org/paper/4b71ab38d3ac6eee0da5e68ef666d33d1028bad8 https://www.semanticscholar.org/paper/1130d8fdd931225c2d7563c3808367726cfa1c3a https://api.semanticscholar.org/f0ea41e56ab7ac1276f8e639ebad4d0ca472d62d https://api.semanticscholar.org/5a1ccb7c3ecc18d27157966f39ed258ab0d473bf https://api.semanticscholar.org/c08fefcd7abd5b4451ecef3dcfcf85aba57e8eae