Domains such as speech recognition or neural machine translation , work with inherently temporal data and , today , achieve impressive results by using sequential encoder-decoder structures . Inspired by these sequence-to-sequence models , we adapt an encoder structure with convolutional recurrent layers in order to approximate a phenological model for vegetation classes based on a temporal sequence of Sentinel 2 ( S2 ) images . In our experiments , we visualize internal activations over a sequence of cloudy and non-cloudy images and find several recurrent cells that reduce the input activity for cloudy observations . It contains 470 erroneous sentences and their 470 error-free counterparts . This is an Arabic parallel corpus that can be used as a linguistic resource for Arabic natural language processing ( NLP ) mainly to train sequence-to-sequence models for grammar checking . Sentences were manually collected from a book that has been prepared as a guide for correctly writing and using Arabic grammar and other linguistic features . To address the above difficulties , we propose a novel Deep Hybrid Trajectory Recovery model . Our model extends the classic sequence-to-sequence generation framework by implementing a subsequence-to-sequence recovery model tailored for the current task , named subseq2seq . In order to effectively capture spatiotemporal correlations , we adopt both spatial and temporal attentions for enhancing the model performance . The sequence-to-sequence paradigm employed by neural text-to-SQL models typically performs token-level decoding and does not consider generating SQL hierarchically from a grammar . Grammar-based decoding has shown significant improvements for other semantic parsing tasks , but SQL and other general programming languages have complexities not present in logical formalisms that make writing hierarchical grammars difficult . AMR - to - text generation is a problem recently introduced to the NLP community , in which the goal is to generate sentences from Abstract Meaning Representation ( AMR ) graphs . Sequence - to - sequence models can be used to this end by converting the AMR graphs to strings . Approaching the problem while working directly with graphs requires the use of graph - to - sequence models that encode the AMR graph into a vector representation . More recent models for distributed sentence representation fall into two categories according to the form of input sentence : sequence - based models and tree - structured models . Sequence - based models construct sentence representations from word sequences by taking in account the relationship between successive words . Tree - structured models treat each word token as a node in a syntactic parse tree and learn sentence representations from leaves to the root in a recursive manner . To avoid the problem of gradient exploding or vanishing in the standard RNN , Long Short - term Memory RNN ( LSTM ) and other variants were designed for better remembering and memory accesses . Along with the sequence - based or the tree - structured models , RNNs have achieved remarkable results in sentence or document modeling . To conclude , CNN is able to learn local response from temporal or spatial data but lacks the ability of learning sequential correlations ; on the other hand , RNN is specilized for sequential modelling but unable to extract features in a parallel way . BART ( Versley et al. , 2008 ) is a highly modular toolkit for coreference resolution that supports state-of-the-art statistical approaches and enables efficient feature engineering . For the SemEval task 1 on Coreference Resolution , BART runs have been submitted for German , English , and Italian . Bayesian Additive Regression Trees ( BART ) is a Bayesian machine learning algorithm in which the conditional distribution is modeled as a sum of regression trees . We extend BART into a semiparametric generalized linear model framework so that a portion of the covariates are modeled nonparametrically using BART and a subset of the covariates have parametric form . This presents an attractive option for research in which only a few covariates are of scientific interest but there are other covariates must be controlled for . Efficient computation is available via an MCMC algorithm , which provides a mechanism for direct adaptation of successful Bayesian methods for continuous data to the integer-valued data setting . Using the STAR framework , we develop new linear regression models , additive models , and Bayesian Additive Regression Trees ( BART ) for integer-valued data , which demonstrate substantial improvements in performance relative to existing regression models for a variety of simulated and real datasets . Paper aims : This paper presents a comparison of the performances of the Bayesian additive regression trees ( BART ) , Random Forest ( RF ) and the logistic regression model ( LRM ) for the development of credit scoring models . Originality The model performances varied from country to country , and from spatial to spatio-temporal setting . In general , BGM , GAM and BART models performed well , with BGM being the most consistent . The inclusion of past data is not always beneficial : the predictive performance of GAM and GBM increased under spatio-temporal setting , but BGM ’s performance decreased in most of the countries . However , the publicly available implementation of the BART algorithm in the R package BayesTree is not fast enough to be considered interactive with over a thousand observations , and is unlikely to even run with 50,000 to 100,000 observations . In this article we show how the BART algorithm may be modified and then computed using single program , multiple data ( SPMD ) parallel computation implemented using the Message Passing Interface ( MPI ) library . The approach scales nearly linearly in the number of processor cores , enabling the practitioner to perform statistical inference on massive datasets . This chapter describes the BART case study , that will be used in all this book as a common problem to be solved by various analysis methods . The following chapters pick-up pieces from this example to illustrate possibilities of the presented techniques . This chapter describes the BART case study , that will be used in all this book as a common problem to be solved by various analysis methods . The following chapters pick-up pieces from this example to illustrate possibilities of the presented techniques . BART also provides a 1.1 BLEU increase over a back-translation system for machine translation , with only target language pretraining . We also report ablation experiments that replicate other pretraining schemes within the BART framework , to better measure which factors most influence end-task performance . We propose a new nonlinear classification method based on a Bayesian ' ' sum-of-trees ' ' model , the Bayesian Additive Classification Tree ( BACT ) , which extends the Bayesian Additive Regression Tree ( BART ) method into the classification context . Like BART , the BACT is a Bayesian nonparametric additive model specified by a prior and a likelihood in which the additive components are trees , and it is fitted by an iterative MCMC algorithm . Further , the paper contributes by discussing how to interpret the estimates produced by BART and similar bandwidth estimation tools relying on self-induced congestion when used in wireless scenarios . An analysis show that the BART estimates produced are correct - but corresponds to a fair share of the wireless link rather than to the unused capacity . However , the estimates do indicate how much bandwidth an application or device in the wireless network can expect when sending and/or receiving network traffic . However , the current online learning approaches often rely on heuristic methods that lack a formal statistical basis and exhibit limited scalability in the face of large data stream . In light of these issues , we develop a new Sequential Probabilistic Learning for Adaptive Fuzzy Inference System ( SPLAFIS ) that synergizes the Bayesian Adaptive Resonance Theory ( BART ) and Rule-Wise Decoupled Extended Kalman Filter ( RDEKF ) to generate the rule base structure and refine its parameters , respectively . The marriage of the BART and RDEKF methods , both of which are built upon the maximum a posteriori ( MAP ) principle rooted in the Bayes ’ rule , offers a comprehensive probabilistic treatment and an efficient way for online structural and parameter learning suitable for large , dynamic data stream . To manage the model complexity without sacrificing its predictive accuracy , SPLAFIS also includes a simple procedure to prune inconsequential rules that have little contribution over time . - nearest neighbor ( - NN ) , Targeted Maximum Likelihood , which is a doubly robust method ( TMLE ) , Bayesian Additive Regression Trees ( BART ) , Random Forests ( Rand . For . ) , Causal Forests ( Caus . However , the attribute of cyberspace benchmarks is different from that of images . In this paper , we propose using sequence-based generative models such as Sequence Generative Adversarial Nets ( SeqGAN ) and Sequence to Sequence ( Seq2Seq ) to augment the ADFA-LD dataset , a sequence call based benchmark . Experimental results show that the performance is better when training ADFA-LD with augmented data from SeqGAN and Seq2Seq than training only the original dataset . Most of the current abstractive text summarization models are based on the sequence-to-sequence model ( Seq2Seq ) . The source content of social media is long and noisy , so it is difficult for Seq2Seq to learn an accurate semantic representation . Compared with the source content , the annotated summary is short and well written . The classical sequence-to-sequence ( Seq2Seq ) methods can hardly satisfy this requirement . In this study , we propose a novel deep neural network model based on the Seq2Seq framework for the review response generation task in e-commerce platforms , which can incorporate product information by a gated multi-source attention mechanism and a copy mechanism . Moreover , we employ a reinforcement learning technique to reduce the exposure bias problem . It plays an important role in Mongolian related natural language processing tasks . To improve performance and avoid the tedious work of rule-making and statistics over large-scale corpus in early methods , this work takes a Seq2Seq framework to realize Mongolian word segmentation . Since each Mongolian word consisted of several sequential characters , we map Mongolian word segmentation to character-level Seq2Seq task , and further propose three different models from three different prospective to achieve the segmentation goal . There have been some success in formulating the multi-label problem as sequence-to-sequence ( Seq2Seq ) learning , where the positive class labels of each input instance are used as the corresponding output sequence . Seq2Seq methods , nonetheless , have not yet been scalable to the XMC setting due to the softmax bottleneck . In this paper , we propose a semantic coding framework , namely SeCSeq , for a Seq2Seq approach to the XMC problem . Bidirectional Seq2Seq models show an average improvement of 17.6 % in forecasting performance over their unidirectional counterparts . We also demonstrate the effect of employing an attention mechanism in the context of Seq2Seq architectures and find that it provides an average improvement of 57.12 % in the case of unidirectional Seq2Seq architectures while causing a performance decline in the case of bidirectional Seq2Seq architectures . Finally , we also find that scheduled sampling helps in training better models that yield significantly lower forecasting error . We examine a set of deep learning models constructed based on the Fully Connected Network ( FCN ) , Variational AutoEncoder ( VAE ) , and Long Short-Term Memory with Sequence to Sequence ( LSTM Seq2Seq ) structures , with two public traffic data sets that have distinctive properties with respect to the distribution of normal and attack populations . Our experimental results confirm the potential of deep learning models for network anomaly detection , and the model based on the LSTM Seq2Seq structure shows a highly promising performance , yielding 99 % of binary classification accuracy on the public data sets . Specifically , a sequence to sequence ( Seq2Seq ) learning architecture that leverages Long Short-Term Memory ( LSTM ) and Self-Attention mechanism ( SA ) is proposed for dance generation . The work in this article is interesting in the following aspects : 1 ) A cross-domain Seq2Seq learning framework is proposed for realistic dance generation ; 2 ) A set of evaluation criterion is proposed for synthetization evaluation which do not have source for reference ; 3 ) A dance dataset that including both music and corresponding dance motions collected , and very competitive results have been obtained against the-state-of-the-arts . However , existing task-oriented dialogue systems depend on handcrafted actions and states or extra semantic labels , which sometimes degrades user experience despite the intensive human intervention . Moreover , current user simulators have limited expressive ability so that deep reinforcement Seq2Seq models have to rely on selfplay and only work in some special cases . To address those problems , we propose a uSer and Agent Model IntegrAtion ( SAMIA ) framework inspired by an observation that the roles of the user and agent models are asymmetric . This paper presents a Generative Adversarial Network ( GAN ) to model singleturn short-text conversations , which trains a sequence-to-sequence ( Seq2Seq ) network for response generation simultaneously with a discriminative classifier that measures the differences between human-produced responses and machinegenerated ones . In addition , the proposed method introduces an approximate embedding layer to solve the non-differentiable problem caused by the sampling-based output decoding procedure in the Seq2Seq generative model . The GAN setup provides an effective way to avoid noninformative responses ( a.k.a “ safe responses ” ) , which are frequently observed in traditional neural response generators . In our system , we use the Attention-based Sequence-to-Sequence ( Seq2Seq ) method as our basic structure to generate emotional responses . This paper introduces ( 1 ) an emotion-aware Seq2Seq model and ( 2 ) several features to boost the performance of emotion consistency . Official results show that our model performs the best in terms of the overall results across the five given emotion categories . We also explore two approaches to accomplish the conditional discriminator : ( 1 ) phredGAN_a , a system that passes the attribute representation as an additional input into a traditional adversarial discriminator , and ( 2 ) phredGAN_d , a dual discriminator system which in addition to the adversarial discriminator , collaboratively predicts the attribute(s ) that generated the input utterance . To demonstrate the superior performance of phredGAN over the persona Seq2Seq model , we experiment with two conversational datasets , the Ubuntu Dialogue Corpus ( UDC ) and TV series transcripts from the Big Bang Theory and Friends . Performance comparison is made with respect to a variety of quantitative measures as well as crowd-sourced human evaluation . To show that the MoS is a generic structure that can be used to model other context - dependent distributions , we additionally conduct experiments in the dialog domain . We use the Switchboard dataset godfrey1997switchboard preprocessed by zhao2017learning to train a Seq2Seq sutskever2014sequence model with MoS added to the decoder RNN . Then , a Seq2Seq model using Softmax and another one augmented by MoC with comparable parameter sizes are used as baselines . The increased amount of information in a profile versus a single sequence can lead to more accurate alignments and more sensitive homolog detection in database searches . Several profile-profile alignment methods have been proposed and have been shown to improve sensitivity and alignment quality compared with sequence-sequence methods ( such as BLAST ) and profile-sequence methods ( e.g. PSI-BLAST ) . Here we present a new approach to profile-profile alignment we call Comparison of Alignments by Constructing Hidden Markov Models ( HMMs ) ( COACH ) . Results . From Table [ reference ] , we can see that the LSTM model fails on this task while our model makes perfect predictions , which underlines the importance of the use of graph encoder to directly encode a graph instead of using sequence model on the converted inputs from a graph . Comparing to GGS - NN that uses carefully designed initial embeddings for different types of nodes such as START and END , our model uses a purely end - to - end approach which generates the initial node feature vectors based on random initialization of the embeddings for words in text attributes . mou also explores convolutional models on tree - structured sentences . As a sequence model , RNN is able to deal with variable - length input sequences and discover long - term dependencies . Various variants of RNN have been proposed to better store and access memories . As the loss function is usually calculated only for frequency-domain acoustic features , that does n't directly control the quality of the generated time-domain waveform . To address this problem , we propose a new training scheme for Tacotron-based TTS , referred to as WaveTTS , that has 2 loss functions : 1 ) time-domain loss , denoted as the waveform loss , that measures the distortion between the natural and generated waveform ; and 2 ) frequency-domain loss , that measures the Mel-scale acoustic feature loss between the natural and generated acoustic features . WaveTTS ensures both the quality of the acoustic features and the resulting speech waveform . To our best knowledge , this is the first implementation of Tacotron with joint time-frequency domain loss .
Domains such as speech recognition or neural machine translation , work with inherently temporal data and , today , achieve impressive results by using sequential encoder-decoder structures . Inspired by these sequence-to-sequence models , we adapt an encoder structure with convolutional recurrent layers in order to approximate a phenological model for vegetation classes based on a temporal sequence of Sentinel 2 ( S2 ) images . In our experiments , we visualize internal activations over a sequence of cloudy and non-cloudy images and find several recurrent cells that reduce the input activity for cloudy observations .
It contains 470 erroneous sentences and their 470 error-free counterparts . This is an Arabic parallel corpus that can be used as a linguistic resource for Arabic natural language processing ( NLP ) mainly to train sequence-to-sequence models for grammar checking . Sentences were manually collected from a book that has been prepared as a guide for correctly writing and using Arabic grammar and other linguistic features .
To address the above difficulties , we propose a novel Deep Hybrid Trajectory Recovery model . Our model extends the classic sequence-to-sequence generation framework by implementing a subsequence-to-sequence recovery model tailored for the current task , named subseq2seq . In order to effectively capture spatiotemporal correlations , we adopt both spatial and temporal attentions for enhancing the model performance .
The sequence-to-sequence paradigm employed by neural text-to-SQL models typically performs token-level decoding and does not consider generating SQL hierarchically from a grammar . Grammar-based decoding has shown significant improvements for other semantic parsing tasks , but SQL and other general programming languages have complexities not present in logical formalisms that make writing hierarchical grammars difficult .
AMR - to - text generation is a problem recently introduced to the NLP community , in which the goal is to generate sentences from Abstract Meaning Representation ( AMR ) graphs . Sequence - to - sequence models can be used to this end by converting the AMR graphs to strings . Approaching the problem while working directly with graphs requires the use of graph - to - sequence models that encode the AMR graph into a vector representation .
More recent models for distributed sentence representation fall into two categories according to the form of input sentence : sequence - based models and tree - structured models . Sequence - based models construct sentence representations from word sequences by taking in account the relationship between successive words . Tree - structured models treat each word token as a node in a syntactic parse tree and learn sentence representations from leaves to the root in a recursive manner .
To avoid the problem of gradient exploding or vanishing in the standard RNN , Long Short - term Memory RNN ( LSTM ) and other variants were designed for better remembering and memory accesses . Along with the sequence - based or the tree - structured models , RNNs have achieved remarkable results in sentence or document modeling . To conclude , CNN is able to learn local response from temporal or spatial data but lacks the ability of learning sequential correlations ; on the other hand , RNN is specilized for sequential modelling but unable to extract features in a parallel way .
BART ( Versley et al. , 2008 ) is a highly modular toolkit for coreference resolution that supports state-of-the-art statistical approaches and enables efficient feature engineering . For the SemEval task 1 on Coreference Resolution , BART runs have been submitted for German , English , and Italian .
Bayesian Additive Regression Trees ( BART ) is a Bayesian machine learning algorithm in which the conditional distribution is modeled as a sum of regression trees . We extend BART into a semiparametric generalized linear model framework so that a portion of the covariates are modeled nonparametrically using BART and a subset of the covariates have parametric form . This presents an attractive option for research in which only a few covariates are of scientific interest but there are other covariates must be controlled for .
Efficient computation is available via an MCMC algorithm , which provides a mechanism for direct adaptation of successful Bayesian methods for continuous data to the integer-valued data setting . Using the STAR framework , we develop new linear regression models , additive models , and Bayesian Additive Regression Trees ( BART ) for integer-valued data , which demonstrate substantial improvements in performance relative to existing regression models for a variety of simulated and real datasets .
Paper aims : This paper presents a comparison of the performances of the Bayesian additive regression trees ( BART ) , Random Forest ( RF ) and the logistic regression model ( LRM ) for the development of credit scoring models . Originality
The model performances varied from country to country , and from spatial to spatio-temporal setting . In general , BGM , GAM and BART models performed well , with BGM being the most consistent . The inclusion of past data is not always beneficial : the predictive performance of GAM and GBM increased under spatio-temporal setting , but BGM ’s performance decreased in most of the countries .
However , the publicly available implementation of the BART algorithm in the R package BayesTree is not fast enough to be considered interactive with over a thousand observations , and is unlikely to even run with 50,000 to 100,000 observations . In this article we show how the BART algorithm may be modified and then computed using single program , multiple data ( SPMD ) parallel computation implemented using the Message Passing Interface ( MPI ) library . The approach scales nearly linearly in the number of processor cores , enabling the practitioner to perform statistical inference on massive datasets .
This chapter describes the BART case study , that will be used in all this book as a common problem to be solved by various analysis methods . The following chapters pick-up pieces from this example to illustrate possibilities of the presented techniques .
This chapter describes the BART case study , that will be used in all this book as a common problem to be solved by various analysis methods . The following chapters pick-up pieces from this example to illustrate possibilities of the presented techniques .
BART also provides a 1.1 BLEU increase over a back-translation system for machine translation , with only target language pretraining . We also report ablation experiments that replicate other pretraining schemes within the BART framework , to better measure which factors most influence end-task performance .
We propose a new nonlinear classification method based on a Bayesian ' ' sum-of-trees ' ' model , the Bayesian Additive Classification Tree ( BACT ) , which extends the Bayesian Additive Regression Tree ( BART ) method into the classification context . Like BART , the BACT is a Bayesian nonparametric additive model specified by a prior and a likelihood in which the additive components are trees , and it is fitted by an iterative MCMC algorithm .
Further , the paper contributes by discussing how to interpret the estimates produced by BART and similar bandwidth estimation tools relying on self-induced congestion when used in wireless scenarios . An analysis show that the BART estimates produced are correct - but corresponds to a fair share of the wireless link rather than to the unused capacity . However , the estimates do indicate how much bandwidth an application or device in the wireless network can expect when sending and/or receiving network traffic .
However , the current online learning approaches often rely on heuristic methods that lack a formal statistical basis and exhibit limited scalability in the face of large data stream . In light of these issues , we develop a new Sequential Probabilistic Learning for Adaptive Fuzzy Inference System ( SPLAFIS ) that synergizes the Bayesian Adaptive Resonance Theory ( BART ) and Rule-Wise Decoupled Extended Kalman Filter ( RDEKF ) to generate the rule base structure and refine its parameters , respectively . The marriage of the BART and RDEKF methods , both of which are built upon the maximum a posteriori ( MAP ) principle rooted in the Bayes ’ rule , offers a comprehensive probabilistic treatment and an efficient way for online structural and parameter learning suitable for large , dynamic data stream . To manage the model complexity without sacrificing its predictive accuracy , SPLAFIS also includes a simple procedure to prune inconsequential rules that have little contribution over time .
- nearest neighbor ( - NN ) , Targeted Maximum Likelihood , which is a doubly robust method ( TMLE ) , Bayesian Additive Regression Trees ( BART ) , Random Forests ( Rand . For . ) , Causal Forests ( Caus .
However , the attribute of cyberspace benchmarks is different from that of images . In this paper , we propose using sequence-based generative models such as Sequence Generative Adversarial Nets ( SeqGAN ) and Sequence to Sequence ( Seq2Seq ) to augment the ADFA-LD dataset , a sequence call based benchmark . Experimental results show that the performance is better when training ADFA-LD with augmented data from SeqGAN and Seq2Seq than training only the original dataset .
Most of the current abstractive text summarization models are based on the sequence-to-sequence model ( Seq2Seq ) . The source content of social media is long and noisy , so it is difficult for Seq2Seq to learn an accurate semantic representation . Compared with the source content , the annotated summary is short and well written .
The classical sequence-to-sequence ( Seq2Seq ) methods can hardly satisfy this requirement . In this study , we propose a novel deep neural network model based on the Seq2Seq framework for the review response generation task in e-commerce platforms , which can incorporate product information by a gated multi-source attention mechanism and a copy mechanism . Moreover , we employ a reinforcement learning technique to reduce the exposure bias problem .
It plays an important role in Mongolian related natural language processing tasks . To improve performance and avoid the tedious work of rule-making and statistics over large-scale corpus in early methods , this work takes a Seq2Seq framework to realize Mongolian word segmentation . Since each Mongolian word consisted of several sequential characters , we map Mongolian word segmentation to character-level Seq2Seq task , and further propose three different models from three different prospective to achieve the segmentation goal .
There have been some success in formulating the multi-label problem as sequence-to-sequence ( Seq2Seq ) learning , where the positive class labels of each input instance are used as the corresponding output sequence . Seq2Seq methods , nonetheless , have not yet been scalable to the XMC setting due to the softmax bottleneck . In this paper , we propose a semantic coding framework , namely SeCSeq , for a Seq2Seq approach to the XMC problem .
Bidirectional Seq2Seq models show an average improvement of 17.6 % in forecasting performance over their unidirectional counterparts . We also demonstrate the effect of employing an attention mechanism in the context of Seq2Seq architectures and find that it provides an average improvement of 57.12 % in the case of unidirectional Seq2Seq architectures while causing a performance decline in the case of bidirectional Seq2Seq architectures . Finally , we also find that scheduled sampling helps in training better models that yield significantly lower forecasting error .
We examine a set of deep learning models constructed based on the Fully Connected Network ( FCN ) , Variational AutoEncoder ( VAE ) , and Long Short-Term Memory with Sequence to Sequence ( LSTM Seq2Seq ) structures , with two public traffic data sets that have distinctive properties with respect to the distribution of normal and attack populations . Our experimental results confirm the potential of deep learning models for network anomaly detection , and the model based on the LSTM Seq2Seq structure shows a highly promising performance , yielding 99 % of binary classification accuracy on the public data sets .
Specifically , a sequence to sequence ( Seq2Seq ) learning architecture that leverages Long Short-Term Memory ( LSTM ) and Self-Attention mechanism ( SA ) is proposed for dance generation . The work in this article is interesting in the following aspects : 1 ) A cross-domain Seq2Seq learning framework is proposed for realistic dance generation ; 2 ) A set of evaluation criterion is proposed for synthetization evaluation which do not have source for reference ; 3 ) A dance dataset that including both music and corresponding dance motions collected , and very competitive results have been obtained against the-state-of-the-arts .
However , existing task-oriented dialogue systems depend on handcrafted actions and states or extra semantic labels , which sometimes degrades user experience despite the intensive human intervention . Moreover , current user simulators have limited expressive ability so that deep reinforcement Seq2Seq models have to rely on selfplay and only work in some special cases . To address those problems , we propose a uSer and Agent Model IntegrAtion ( SAMIA ) framework inspired by an observation that the roles of the user and agent models are asymmetric .
This paper presents a Generative Adversarial Network ( GAN ) to model singleturn short-text conversations , which trains a sequence-to-sequence ( Seq2Seq ) network for response generation simultaneously with a discriminative classifier that measures the differences between human-produced responses and machinegenerated ones . In addition , the proposed method introduces an approximate embedding layer to solve the non-differentiable problem caused by the sampling-based output decoding procedure in the Seq2Seq generative model . The GAN setup provides an effective way to avoid noninformative responses ( a.k.a “ safe responses ” ) , which are frequently observed in traditional neural response generators .
In our system , we use the Attention-based Sequence-to-Sequence ( Seq2Seq ) method as our basic structure to generate emotional responses . This paper introduces ( 1 ) an emotion-aware Seq2Seq model and ( 2 ) several features to boost the performance of emotion consistency . Official results show that our model performs the best in terms of the overall results across the five given emotion categories .
We also explore two approaches to accomplish the conditional discriminator : ( 1 ) phredGAN_a , a system that passes the attribute representation as an additional input into a traditional adversarial discriminator , and ( 2 ) phredGAN_d , a dual discriminator system which in addition to the adversarial discriminator , collaboratively predicts the attribute(s ) that generated the input utterance . To demonstrate the superior performance of phredGAN over the persona Seq2Seq model , we experiment with two conversational datasets , the Ubuntu Dialogue Corpus ( UDC ) and TV series transcripts from the Big Bang Theory and Friends . Performance comparison is made with respect to a variety of quantitative measures as well as crowd-sourced human evaluation .
To show that the MoS is a generic structure that can be used to model other context - dependent distributions , we additionally conduct experiments in the dialog domain . We use the Switchboard dataset godfrey1997switchboard preprocessed by zhao2017learning to train a Seq2Seq sutskever2014sequence model with MoS added to the decoder RNN . Then , a Seq2Seq model using Softmax and another one augmented by MoC with comparable parameter sizes are used as baselines .
The increased amount of information in a profile versus a single sequence can lead to more accurate alignments and more sensitive homolog detection in database searches . Several profile-profile alignment methods have been proposed and have been shown to improve sensitivity and alignment quality compared with sequence-sequence methods ( such as BLAST ) and profile-sequence methods ( e.g. PSI-BLAST ) . Here we present a new approach to profile-profile alignment we call Comparison of Alignments by Constructing Hidden Markov Models ( HMMs ) ( COACH ) .
Results . From Table [ reference ] , we can see that the LSTM model fails on this task while our model makes perfect predictions , which underlines the importance of the use of graph encoder to directly encode a graph instead of using sequence model on the converted inputs from a graph . Comparing to GGS - NN that uses carefully designed initial embeddings for different types of nodes such as START and END , our model uses a purely end - to - end approach which generates the initial node feature vectors based on random initialization of the embeddings for words in text attributes .
mou also explores convolutional models on tree - structured sentences . As a sequence model , RNN is able to deal with variable - length input sequences and discover long - term dependencies . Various variants of RNN have been proposed to better store and access memories .
As the loss function is usually calculated only for frequency-domain acoustic features , that does n't directly control the quality of the generated time-domain waveform . To address this problem , we propose a new training scheme for Tacotron-based TTS , referred to as WaveTTS , that has 2 loss functions : 1 ) time-domain loss , denoted as the waveform loss , that measures the distortion between the natural and generated waveform ; and 2 ) frequency-domain loss , that measures the Mel-scale acoustic feature loss between the natural and generated acoustic features . WaveTTS ensures both the quality of the acoustic features and the resulting speech waveform . To our best knowledge , this is the first implementation of Tacotron with joint time-frequency domain loss .
Multi-Temporal Land Cover Classification with Sequential Recurrent Encoders A7׳ta: Data on a monolingual Arabic parallel corpus for grammar checking Deep Trajectory Recovery with Fine-Grained Calibration using Kalman Filter Grammar-based Neural Text-to-SQL Generation Structural Neural Encoders for AMR-to-text Generation A C-LSTM Neural Network for Text Classification BART: A Multilingual Anaphora Resolution System Bayesian Nonparametric Methods For Causal Inference And Prediction A Simultaneous Transformation and Rounding Approach for Modeling Integer-Valued Data Application of bayesian additive regression trees in the development of credit scoring models in Brazil Comparison of national level spatial and spatio-temporal models of malaria Parallel Bayesian Additive Regression Trees The BRAT case study The BART Case Study BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension The Bayesian Additive Classification Tree applied to credit risk modelling On measuring available bandwidth in wireless networks Online probabilistic learning for fuzzy inference system Estimating individual treatment effect: generalization bounds and algorithms Anomaly Dataset Augmentation Using the Sequence Generative Models Autoencoder as Assistant Supervisor: Improving Text Representation for Chinese Social Media Text Summarization Review Response Generation in E-Commerce Platforms with External Product Information Mongolian Word Segmentation Based on Three Character Level Seq2Seq Models SeCSeq: Semantic Coding for Sequence-to-Sequence based Extreme Multi-label Classification Multivariate Long-Term State Forecasting in Cyber-Physical Systems: A Sequence to Sequence Approach An Empirical Evaluation of Deep Learning for Network Anomaly Detection Music-Driven Dance Generation Integrating User and Agent Models: A Deep Task-Oriented Dialogue System Neural Response Generation via GAN with an Approximate Embedding Layer A Hybrid Framework of Emotion-Aware Seq2Seq Model for Emotional Conversation Generation An Adversarial Learning Framework For A Persona-Based Multi-Turn Dialogue Model Breaking the Softmax Bottleneck: A High-Rank RNN Language Model COACH: profile-profile alignment of protein families using hidden Markov models Graph2Seq: Graph to Sequence Learning with Attention-based Neural Networks A C-LSTM Neural Network for Text Classification WaveTTS: Tacotron-based TTS with Joint Time-Frequency Domain Loss
900630185a31b573fefd7fd2195b93c80df8cdb3 f60fae7ad61f33eec5d81b5c7195085760df2903 e65e068aa247b3b7494500c44c377f4bfa0f9010 6a9cab267526de228a51f70ae56f1e07ecfc7bf8 db51c9525f49b4a2c9711b45f31f2f6b6522c1f2 a5a30c2b7166f2d204e3ba693854ef3c5eb0b81e 4ef72e65e0e080a532082d89b9e3cab19d6aa9d5 7268ba5effaba4ed8f7f88921a0ab80eedcdb53c ae502bc97cdcf3f47aa29db69b0ac747c98fa68f 7748b87663351d6e2b045aa3147e83a2880531e0 e842146c2d3734dca6e687e5061b8030a68b248d f730762e4f2f61e093f46492ccea48157b748d41 395de0bd3837fdf4b4b5e5f04835bcc69c279481 21809ac58b8b63c5b9ea981ac96bcf35afdade1c 8718971926808781f464103e4c3a022d215cd86c 6f6eb2b7c34f48fc78caeb65a652a865685fa604 d3167d71de5e345efc5bb6dfd4b9a3d9c4f74084 d4038429315874a851e5f39b0d6c7cd0d6189d82 2aa0c96074cf0baef3d130803382ae4a3d37566f 45668027a30a8e05a7bbafdf4ca510dc5315e26a d994be4c1d9e711523cad825931b5eb5cb9074c5 414f38f0a64d6d8fb3fa23ee6542918b7c731b1e 793fa365083be47b0e92e35cf19400347acd7cbd 1f94e9a19484de7a92d025a371b24c196fe1b117 b34f2bcce6069e9c4173ca1fa3bd32d6fdc8b196 f4a67539e254dcbf8d662f6ab7435ceb81a61990 2ece257d957ccf0cb722f695354cecb97d44788c 6e173c90553ea9e77c105069f4aa18ed85b736c3 8b4a5d5d09bb6fe56940869a312c3261938020c8 6bd4e38477543226b402cc3c1949c6cf08df0552
["Computer Science"] ["Biology","Medicine"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Mathematics"] ["Computer Science"] ["Biology","Medicine"] ["Computer Science","Mathematics"] ["Computer Science"] ["Computer Science"] ["Computer Science","Mathematics"] ["Mathematics","Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science","Mathematics"] ["Computer Science","Medicine"] ["Computer Science","Engineering"]
2018 2019 2019 2019 2019 2015 2010 2017 2019 2018 2019 2014 2004 2004 2019 2010 2008 2014 2017 2019 2018 2019 2018 2018 2019 2018 2019 2017 2017 2019 2019 2018 2004 2018 2015 2020
NAACL-HLT ArXiv SemEval@ACL Formal Methods for Embedded Distributed Systems LCN ICML ICMLA WWW ICONIP (5) BigData ICNC EMNLP NTCIR ICLR ArXiv ArXiv
https://api.semanticscholar.org/900630185a31b573fefd7fd2195b93c80df8cdb3 https://api.semanticscholar.org/f60fae7ad61f33eec5d81b5c7195085760df2903 https://api.semanticscholar.org/e65e068aa247b3b7494500c44c377f4bfa0f9010 https://api.semanticscholar.org/6a9cab267526de228a51f70ae56f1e07ecfc7bf8 https://www.semanticscholar.org/paper/0fc47b976051366cc1b1ec00e0f3d512deb3232a https://www.semanticscholar.org/paper/10f62af29c3fc5e2572baddca559ffbfd6be8787 https://api.semanticscholar.org/db51c9525f49b4a2c9711b45f31f2f6b6522c1f2 https://api.semanticscholar.org/a5a30c2b7166f2d204e3ba693854ef3c5eb0b81e https://api.semanticscholar.org/4ef72e65e0e080a532082d89b9e3cab19d6aa9d5 https://api.semanticscholar.org/7268ba5effaba4ed8f7f88921a0ab80eedcdb53c https://api.semanticscholar.org/ae502bc97cdcf3f47aa29db69b0ac747c98fa68f https://api.semanticscholar.org/7748b87663351d6e2b045aa3147e83a2880531e0 https://api.semanticscholar.org/e842146c2d3734dca6e687e5061b8030a68b248d https://api.semanticscholar.org/f730762e4f2f61e093f46492ccea48157b748d41 https://api.semanticscholar.org/395de0bd3837fdf4b4b5e5f04835bcc69c279481 https://api.semanticscholar.org/21809ac58b8b63c5b9ea981ac96bcf35afdade1c https://api.semanticscholar.org/8718971926808781f464103e4c3a022d215cd86c https://api.semanticscholar.org/6f6eb2b7c34f48fc78caeb65a652a865685fa604 https://www.semanticscholar.org/paper/17409ae85c237983dc22a5e4750d8054e4b6edc9 https://api.semanticscholar.org/d3167d71de5e345efc5bb6dfd4b9a3d9c4f74084 https://api.semanticscholar.org/d4038429315874a851e5f39b0d6c7cd0d6189d82 https://api.semanticscholar.org/2aa0c96074cf0baef3d130803382ae4a3d37566f https://api.semanticscholar.org/45668027a30a8e05a7bbafdf4ca510dc5315e26a https://api.semanticscholar.org/d994be4c1d9e711523cad825931b5eb5cb9074c5 https://api.semanticscholar.org/414f38f0a64d6d8fb3fa23ee6542918b7c731b1e https://api.semanticscholar.org/793fa365083be47b0e92e35cf19400347acd7cbd https://api.semanticscholar.org/1f94e9a19484de7a92d025a371b24c196fe1b117 https://api.semanticscholar.org/b34f2bcce6069e9c4173ca1fa3bd32d6fdc8b196 https://api.semanticscholar.org/f4a67539e254dcbf8d662f6ab7435ceb81a61990 https://api.semanticscholar.org/2ece257d957ccf0cb722f695354cecb97d44788c https://api.semanticscholar.org/6e173c90553ea9e77c105069f4aa18ed85b736c3 https://www.semanticscholar.org/paper/ef9ddbc35676ce8ffc2a8067044473727839dbac https://api.semanticscholar.org/8b4a5d5d09bb6fe56940869a312c3261938020c8 https://www.semanticscholar.org/paper/94eb48c1878efbe2ccff121bd600dd0fd8a75650 https://www.semanticscholar.org/paper/10f62af29c3fc5e2572baddca559ffbfd6be8787 https://api.semanticscholar.org/6bd4e38477543226b402cc3c1949c6cf08df0552