We propose an autoencoding sequence-based transceiver for communication over dispersive channels with intensity modulation and direct detection ( IM/DD ) , designed as a bidirectional deep recurrent neural network ( BRNN ) . The receiver uses a sliding window technique to allow for efficient data stream estimation . This study aims to automatically fuse voice and text data to provide tailored situational awareness information to paramedics . To train and test speech recognition models , we built a bidirectional deep recurrent neural network ( long short-term memory ( LSTM ) ) . Then we used convolutional neural networks on top of custom-trained word vectors for sentence-level classification tasks . This article proposes a new feature extraction method that uses the word embedding method from natural language processing to generate bidirectional real dense vectors to reflect the contextual relationships between the pixels . A bidirectional independent recurrent neural network ( BiIndRNN ) is combined with a convolutional neural network ( CNN ) to improve the sliced recurrent neural network ( SRNN ) algorithm model , which is then constructed in parallel with graph convolutional networks ( GCNs ) under an attention mechanism to fully exploit the deep features of images and to capture the semantic information of the context . This model is collectively named an improved SRNN and attention-treated GCN-based parallel ( SAGP ) model . The recently-proposed deep clustering algorithm introduced significant advances in monaural speaker-independent multi-speaker speech separation . Deep clustering operates on magnitude spectro-grams using bidirectional recurrent networks and K-means clustering , both of which require offline operation , i.e. , algorithm latency is longer than utterance length . This paper evaluates architectures for reduced latency deep clustering by combining : ( 1 ) block processing to efficiently propagate the memory encoded by the recurrent network , and ( 2 ) teacher-student learning , where low-latency models learn from an offline teacher . We develop a phase-sensitive objective function based on the signal-to-noise ratio ( SNR ) of the reconstructed signal , and show that in experiments it yields uniformly better results in terms of signal-to-distortion ratio ( SDR ) . We also investigate improvements to the modeling of dynamics , using bidirectional recurrent networks , as well as by incorporating speech recognition outputs in the form of alignment vectors concatenated with the spectral input features . Both methods yield further improvements , pointing to tighter integration of recognition with separation as a promising future direction . Bidirectional recurrent neural nets have demonstrated state-ofthe-art performance for parametric speech synthesis . In this paper , we introduce a top-down application of recurrent neural net models to unit-selection synthesis . We introduce segmental recurrent neural networks ( SRNNs ) which define , given an input sequence , a joint probability distribution over segmentations of the input and labelings of the segments . Representations of the input segments ( i.e. , contiguous subsequences of the input ) are computed by encoding their constituent tokens using bidirectional recurrent neural nets , and these " segment embeddings " are used to define compatibility scores with output labels . These local compatibility scores are integrated using a global semi-Markov conditional random field . We do this using three modalities ( 1 ) a screenshot of an image , ( 2 ) text descriptions of the functionality of GUI components sourced through Amazon ’s Mechanical Turk , and ( 3 ) parsed information from the screen hierarchy ’s XML dump . The first two modalities are aligned using a convolutional neural network , which detects objects in the screenshot and extracts salient features , paired with a bidirectional recurrent neural network which serves as a language model . Both of these models maps their respective modalities to a semantic space , and then aligns the two representations in that space . The proposed model includes an encoder and a decoder . For each input text image , the encoder part firstly combines deep convolutional layers with bidirectional Recurrent Neural Network to generate an ordered , high-level feature sequence , which could avoid the complicated text segmentation pre-processing . Then in the decoder , a recurrent network with attention mechanism is developed to generate text line output , enabling the model to selectively exploit image features from the encoder correspondingly . In this paper we introduce a new connectionist approach to on-line handwriting recognition and address in particular the problem of recognizing handwritten whiteboard notes . The approach uses a bidirectional recurrent neural network with the long short-term memory architecture . We use a recently introduced objective function , known as Connectionist Temporal Classification ( CTC ) , that directly trains the network to label unsegmented sequence data . One of the main challenges that make this problem even harder is the uncertainty exists in the actions of pedestrians in urban traffic environments , as well as the difficulty in inferring their end goals . In this paper , we are proposing a data-driven framework based on inverse reinforcement learning ( IRL ) and the bidirectional recurrent neural network architecture ( B-LSTM ) for long-term prediction of VRUs ’ intention . We evaluated our framework on real-life datasets for agent behavior modeling in traffic environments , and it has achieved an overall average displacement error of only 2.93 and 4.12 pixels over 2.0 and 3.0 s ahead prediction horizons , respectively . A new approach called Sigfind for the prediction of signal peptides in human protein sequences is introduced . The method is based on the bidirectional recurrent neural network architecture . The modifications to this architecture and a better learning algorithm result in a very accurate identification of signal peptides ( 99.5 % correct in fivefold crossvalidation ) . Second , at the DNN output layer , parameters for pitch-scaled spectrum and aperiodicity measures are estimated for constructing the excitation signal used in our baseline synthesis vocoder . Third , the bidirectional recurrent neural network architecture with long short term memory ( BLSTM ) units is adopted and trained with multi-task learning for DNN-based speech synthesis . Experimental results demonstrate that the quality of synthesized speech has been improved by adopting the new input vector and output parameters . In this study , we compare the performance of one-stage and two-stage architectures for DDI extraction . Our architectures are based on a bidirectional recurrent neural network layer composed of Gated Recurrent Units . The two-stage system obtained a 67.45 % micro-average F1 score on the test set . The neural machine translation system is implemented as an encoder - decoder network with recurrent neural networks . The encoder is a bidirectional neural network with gated recurrent units [ reference ] that reads an input sequence x = ( x 1 , ... , x m ) and calculates a forward sequence of hidden states ( − → h 1 , ... , − → h m ) , and a backward sequence The hidden states − → h j and ← The neural machine translation system is implemented as an encoder - decoder network with recurrent neural networks . The encoder is a bidirectional neural network with gated recurrent units [ reference ] that reads an input sequence x = ( x 1 , ... , x m ) and calculates a forward sequence of hidden states ( − → h 1 , ... , − → h m ) , and a backward sequence The hidden states − → h j and ← These representations are passed through a softmax layer , producing a probability distribution over the possible labels for every token in context : where is the weight matrix between the hidden vector and the output layer . We investigate six alternative neural network architectures for the task of error detection : convolutional , bidirectional recurrent , bidirectional LSTM , and multi - layer variants of each of them . In the convolutional neural network ( CNN , Figure [ reference ] a ) for token labeling , the hidden vector is calculated based on a fixed - size context window . It is designed to detect error patterns from a fixed window of 7 words , which is large enough to not require the use of more advanced composition functions . In contrast , the performance of the bidirectional recurrent network ( Bi - RNN ) is somewhat lower , especially on the test set . In Elman - type recurrent networks , the context signal from distant words decreases fairly rapidly due to the sigmoid activation function and diminishing gradients . section : Sentence - based Character Model The feature that distinguishes our model most from previous work is that we apply a bidirectional recurrent layer ( LSTM ) on all characters of a sentence to induce fully context sensitive initial word encodings . That is , we do not restrict the context of this layer to the words themselves ( as in Figure 1b ) . Kim2016 implemented a language model where encoding is performed by a convolutional network and LSTM over characters , whereas predictions are given on the word - level . Cao2016 proposed a method for learning both word embeddings and morphological segmentation with a bidirectional recurrent network over characters . There is also research on performing parsing and text classification with character - level neural models . Encoding Layer encodes the representations by incorporating the context information or enriching the representation with desirable features for future use . For instance , a model can adopt bidirectional recurrent neural network to model the temporal interaction on both direction , recursive neural network ( also known as TreeRNN ) to model the compositionality and the recursive structure of language , or self - attention to model the long - term dependency on sentence . Different components of encoder can be combined to obtain a better sentence matrix representation . The entire network is trained jointly such that the hidden representation captures the important information from the sequence for the prediction task . A bidirectional recurrent neural network ( bi - RNN ) is an extension of an RNN that reads the input sequence twice , from left to right and right to left , and the encodings are concatenated . The literature uses the term bi - RNN to refer to two related architectures , which we refer to here as “ context bi - RNN ” and “ sequence bi - RNN ” . Each feature vector is extracted from a small overlapping window of audio frames . The encoder is implemented as a deep bidirectional recurrent network ( BiRNN ) , to form a sequential representation of length . At the - th step an ARSG generates an output by focusing on the relevant elements of : where is the - th state of the recurrent neural network to which we refer as the generator , is a vector of the attention weights , also often called the alignment . The comparison of our models and this architecture is presented in § [ reference ] . paragraph : Multidimensional recurrent neural networks . There is another line of research that aims to extend RNNs to operate on multidimensional inputs . Therefore , the integration of multilingual data resources has important application value and practical significance . In this paper , we propose a novel model based on LDA and BiLSTM-CNN to solve the multilingual short text classification problem . In our method , we make full use of topic vectors and word vectors to extract text information from two aspects , in each language , to solve the sparse problem of short text features . Given the quality and quantity of data we have been given , we found that the addition of CNN layer provides very little to no additional improvement to our model 's performance and sometimes even worsen our F1-score . In the end , the deep learning architecture that gives us the highest macro F1-score is a simple BiLSTM-CNN . This paper proposes a research sharing-oriented approach for functional neuroimaging named entity recognition . The nine most representative entity categories were defined by analyzing the characteristics of task-based functional neuroimaging researches , and a multi-category named entity recognition method was designed based on BiLSTM-CNN . An experiment was performed on literatures obtained from the journal PLoS One . In order to solve the issues which are triggered by the lack of research on local adverse drug reactions in Xinjiang and the shortcomings of traditional models in dealing with irregular sentences , this paper proposes a method for adverse drug identification in Xinjiang . The method is combined with BiLSTM-CNN hybrid network which is based on attention mechanism . The method analyzes deeply on the network text context feature and the attention pooling mechanism . Since we leverage results of Subtask 2 directly to infer the result of Subtask 1 , the paper focus on the system solving Subtask 2 . By taking Subtask 2 as a sequence labeling task , our system relies on a recurrent neural network named BiLSTM-CNNCRF with rich linguistic features , such as POS tags , dependency parsing labels , chunking labels , NER labels , Brown clustering . Our system achieved the highest F1 score in both token level and phrase level . Experimental results show that the proposed BMCNN algorithm achieves state-of-the-art performance . In detail , BMCNN can restore both repetitive and irregular structures . Experimental results show that the proposed BMCNN algorithm achieves state-of-the-art performance . In detail , BMCNN can restore both repetitive and irregular structures . In particular , we introduce a novel dataset consisting of faculty listings from university webpages across the world in multiple languages and test different NER models in the task of extracting researcher names from these listings . We found that a neural network architecture that combines a bidirectional LSTM with a Conditional Random Fields output layer , LSTM-based character representations and a Hard Attention mechanism for HTML features outperforms other methods achieving 90.7 F1-score in the task . But , with the aid of clever strategies such as self-training , we can get a much simpler model , the second-order Hidden Markov Model , to achieve a 87.9 F1-score . We propose two tasks : ( 1 ) predicting which argument from an argument pair is more convincing and ( 2 ) ranking all arguments to the topic based on their convincingness . We experiment with feature-rich SVM and bidirectional LSTM and obtain 0.76 - 0.78 accuracy and 0.35 - 0.40 Spearman ’s correlation in a cross-topic evaluation . We release the newly created corpus UKPConvArg1 and the experimental software under open licenses . Recent advances on LU in H2 M conversations have shown accuracy improvements by adding encoded knowledge from different sources . Inspired by this , we explore several variants of a bidirectional LSTM architecture that relies on different knowledge sources , such as Web data , search engine click logs , expert feedback from H2 M models , as well as previous utterances in the conversation . We also propose ensemble techniques that aggregate these different knowledge sources into a single model . However , when dealing with out-of-vocabulary words , there is still a large error rate . We used a simple bidirectional LSTM architecture and a large-scale pretrained language model to generate high-quality contextualize character representations , which successfully reduced the weakness of the ambiguous meanings of each Chinese character that widely appears in Chinese characters , and hence effectively reduced OOV error rate . State-of-the-art performance is achieved on many datasets . The model integrates attention mechanism and target information analysis , which enables the model to concentrate on the important parts of the sentence and to make full use of the target information . The model uses bidirectional LSTM ( Bi-LSTM ) to build the memory of the sentence , and then CNN is applied to extracting attention from memory to get the attentive sentence representation . The model uses aspect embedding to analyze the target information of the representation and finally the model outputs the sentiment polarity through a softmax layer . Using the Fake News Challenge ( FNC-1 ) dataset , we have developed different models to detect fake news based on the relation between article headline and article body . Our models are assembled mainly from Convolutional Neural Network ( CNN ) , Long Short-Term Memory network ( LSTM ) and Bidirectional LSTM ( Bi-LSTM ) . In the contrary of other studies on the same dataset where they reported accuracy for a test data derived from the same training dataset , our experiments achieved 71.2 % accuracy for the official testing dataset . Smartphone can be used for the activity recognition of rehabilitation , aged care and so on . In this paper , we propose a bidirectional LSTM structure for human activity recognition using time series , collected from a smartphone ( with the accelerometer and gyroscope embedded on the phone ) worn on the waist of human , the best recognition accuracy of which can reach 93.79 % . The BRA has two main characteristics : ( 1 ) the motion manifold is extracted by a bidirectional long short-term memory recurrent neural network ( B-LSTM-RNN ) and ( 2 ) apart from statistical information of motion data , kinematic information including smoothness and bone length constrain , are also simultaneously exploited with noisy-clean motion pairs . Using a bidirectional LSTM unit , which is more suitable for time series and can infer information from the data in both time directions , our autoencoder extracts a manifold that can exploit the spatial and temporal relationships between previous and subsequent motion data . As a result , the refined data that are projected by the decoder from the motion manifold have much lower reproduction error . The Bilateral Multi - Perspective Matching ( BiMPM ) model obtains state - of - the - art performance on the SNLI dataset , achieving a maximum accuracy of 86.9 % , and going up to 88.8 % in an ensemble setup . An initial embedding layer vectorises words in P and H using pretrained GLoVe embeddings , and passing them to a context representation layer , which uses bidirectional LSTMs ( BiLSTMs ) to encode context vectors for each time - step . The core part of the model is the subsequent matching layer , where each contextual embedding or time - step of P is matched against all the embeddings of H , and vice versa . In order to use this architecture for error detection , we create a bidirectional LSTM , making use of the advanced features of LSTM and incorporating context on both sides of the target token . In addition , we experiment with a deep bidirectional LSTM , which includes two consecutive layers of bidirectional LSTMs , modeling even more complex features and performing multiple passes through the sentence . For comparison with non - neural models , we also report results using CRFs , which are a popular choice for sequence labeling tasks . The forward and backward activations from the previous LSTM layer are concatenated and fed into the next LSTM layer . The contrast model is a single layer bidirectional LSTM trained on 128 - dim features obtained by performing PCA on 512 - dimensional bottleneck features . The features are obtained from a 6 - layer DNN cross entropy trained on blocks of 11 consecutive FMLLR frames and 100 - dimensional i - vectors . subsection : Sentence Encoder Our sentence encoder is simply composed of multiple stacked bidirectional LSTM ( biLSTM ) layers with shortcut connections followed by a max pooling layer . Let bilstm represent the th biLSTM layer , which is defined as : where is the output of the th biLSTM at time t over input sequence . In this paper , we propose a novel approach to learn coreference-aware word representations for the NER task at the document level . In particular , we enrich the well-known neural architecture “ CNN-BiLSTM-CRF ” with a coreference layer on top of the BiLSTM layer to incorporate coreferential relations . Furthermore , we introduce the coreference regularization to ensure the coreferential entities to share similar representations and consistent predictions within the same coreference cluster . The experimental results show that the prediction accuracy of the bidirectional RNN is obviously higher than that of the unidirectional RNN . In a small-scale training set , the effect of BiLSTM/BiGRU is close , while the training speed of BiGRU is faster , which can be used in the wheel detection system . An optimized Bidirectional Long Short-Term Memory model based on deep attention mechanism is constructed . Then , using the power transformer defect text as the analysis object , the classification effect of DA-BiLSTM was tested . Finally , the classification effect is compared with several typical machine learning classification models . The experimental results of the simulation of the example show that , the classification effect of the DA-BiLSTM has a better advantage than several typical machine learning model . As to algorithm , this paper utilizes the BERT-BiLSTM-CRF model to automatically identify the subject knowledge points , and then evaluates the relationship between the knowledge points by calculating their semantic similarity , PMI and Normalized Google Distance between them . The experimental results show that BERT-BiLSTM-CRF outperforms the baselines significantly , and the three kinds of relationship evaluation models have achieved good results . Finally , computer science and physics science are taken as examples to construct the subject knowledge graphs successfully , which show the effectiveness of our method . In addition , we propose the SAMF-BiLSTM-D model based on SAMF-BiLSTM model for document-level text classification tasks . The method obtains the representation of all sentences in the document through SAMF-BiLSTM training , then integrates BiLSTM to learn the representation of all sentences , and further obtains the sentiment feature information of the entire document . Finally , we evaluate experiment results under five datasets . Recurrent neural networks have proven to be very effective in learning distributed representations and can be trained efficiently on natural language inference tasks . We build on top of one such model and propose a hierarchy of BiLSTM and max pooling layers that implements an iterative refinement strategy and yields state of the art results on the SciTail dataset as well as strong results for SNLI and MultiNLI . We can show that the sentence embeddings learned in this way can be utilized in a wide variety of transfer learning tasks , outperforming InferSent on 7 out of 10 and SkipThought on 8 out of 9 SentEval sentence embedding evaluation tasks . Following a deep learning paradigm , the whole architecture is trained in an end-to-end fashion in order to optimize two objectives : the detection of arousal onset and offset , and the classification of the type of arousal . The novelty of the approach is three-fold : it is the first use of a hybrid ST-BiLSTM network with biomedical signals ; it captures frequency information lower ( 0.1 Hz ) than the detection sampling rate ( 0.5 Hz ) ; and it requires no explicit mechanism to overcome class imbalance in the data . In the follow-up phase of the 2018 Physionet/CinC Challenge the proposed architecture achieved a state-of-the-art area under the precision-recall curve ( AUPRC ) of 0.50 on the hidden test data , tied for the second-highest official result overall . Due to current feature based malicious URLs detection models need manual feature engineering , and deep learning based models have their limit on processing long sequences , which reduces the detection performance . We proposed an attentional based BiLSTM model AB-BiLSTM for the Malicious URLs detection in this paper . Firstly , the URLs were preprocessed and converted into word vectors by using pre-trained Word2Vec , then BiLSTM combined with an attention mechanism was trained to extract URL sequences features and classify them . We propose two PP argumenthood prediction tasks branching from these two motivations : ( 1 ) binary argument/adjunct classification of PPs in VerbNet , and ( 2 ) gradient argumenthood prediction using human judgments as gold standard , and report results from prediction models that use pretrained word embeddings and other linguistically informed features . Our best results on each task are ( 1 ) $ acc.=0.955 $ , $ F_1=0.954 $ ( ELMo+BiLSTM ) and ( 2 ) Pearson 's $ r=0.624 $ ( word2vec+MLP ) . Furthermore , we demonstrate the utility of argumenthood prediction in improving sentence representations via performance gains on SRL when a sentence encoder is pretrained with our tasks . We propose two PP argumenthood prediction tasks branching from these two motivations : ( 1 ) binary argument-adjunct classification of PPs in VerbNet , and ( 2 ) gradient argumenthood prediction using human judgments as gold standard , and report results from prediction models that use pretrained word embeddings and other linguistically informed features . Our best results on each task are ( 1 ) $ acc.=0.955 $ , $ F_1=0.954 $ ( ELMo+BiLSTM ) and ( 2 ) Pearson 's $ r=0.624 $ ( word2vec+MLP ) . Furthermore , we demonstrate the utility of argumenthood prediction in improving sentence representations via performance gains on SRL when a sentence encoder is pretrained with our tasks . This paper explores the use of multi-task learning ( MTL ) for incorporating external knowledge in neural models . Specifically , we show how MTL can enable a BiLSTM sentiment classifier to incorporate information from sentiment lexicons . Our MTL set-up is shown to improve model performance ( compared to a single-task set-up ) on both English and Norwegian sentence-level sentiment datasets . The BiLSTM excels at representing elements in a sequence ( i.e. , words ) together with their contexts , capturing the element and an ‘ ‘ infinite ’ ’ window around it . We represent each word by its BiLSTM encoding , and use a concatenation of a minimal set of such BiLSTM encodings as our feature function , which is then passed to a non - linear scoring function ( multi - layer perceptron ) . Crucially , the BiLSTM is trained with the rest of the parser in order to learn a good feature representation for the parsing problem . The greedy transition - based parser follows standard techniques from the literature ( margin - based objective , dynamic oracle training , error exploration , MLP - based non - linear scoring function ) . We depart from the literature by replacing the hand - crafted feature function over carefully selected components of the configuration with a concatenation of BiLSTM representations of a few prominent items on the stack and the buffer , and training the BiLSTM encoder jointly with the rest of the network . section : Graph - based Parser Because the Row LSTM has a triangular receptive field ( Figure [ reference ] ) , it is unable to capture the entire available context . subsection : Diagonal BiLSTM The Diagonal BiLSTM is designed to both parallelize the computation and to capture the entire available context for any image size . To test whether this might be the case , we also ran an experiment with a 2 - layer BiLSTM ( + liq ) . This setup exhibits similar computational complexity and number of parameters to BiLSTM + + . We found that the second layer did not improve performance , suggesting that pooling over word / lemma occurrences in a given context between layers , is a powerful , yet simple technique . Our results show that embedding refinement using both the system ’s text inputs , as well as supplementary text from external background knowledge can yield large improvements . In particular , we have shown that relatively simple task architectures ( e.g. , based on simple BiLSTM readers ) can become competitive with state of the art , task - specific architectures when augmented with our reading architecture . Our analysis demonstrates that our model learns to exploit provided background knowledge in a semantically appropriate way . CONSTITUTION : A host-sided sub connection unit(21a ) and a display-side sub connection unit(22a ) include a bidirectional converter and a DC(Direct Current ) balancing codec . The bidirectional converts differential sub signals into single sub signal . The bidirectional converter converts single sub signal into differential sub signals . We analyze the spatio-temporal characteristics of GIS , introduce the cellular automata , propose a formal definition of spatio-temporal ontology and a method based on spatio-temporal semantic granularity , and discuss an integrated GIS architecture based on ontology and semantics . We implemented the eartly-stage creation and browse of the railway passenger traffic spatio-temporal ontology and the sharing and reuse of ontology among the GIS sub-systems by tegrating bidirectional in the experimental railway GIS system project . DC/AC microgrid is independent control grid form of integrate distributed energy systems with utility powers systems ; will powerfully understand the value and profit of the distributed energy resources . In this paper , the AC/DC micro-grid mostly contains of six parts : a solar source ( P V source ) , Wind source , a Boost converter , BESS , a Buck-Boost converter ( bidirectional ) , and a bi-directional DC/AC which is associated to the utility grid . The DC/AC microgid have two operation mode ( islanded or grid disconnect and grid-connected ) . Bi - GRU Encoder : We apply bidirectional GRU for encoding query and passage into vector representations . We set the number of hidden units to be 256 and 384 for the CNN and Daily Mail datasets , respectively . A similar approach was taken by Lee2016 for character - level machine translation . Their model ’s encoder uses a convolutional layer followed by max - pooling to reduce sequence length , a four - layer highway network , and a bidirectional GRU . The parallelism of the convolutional , pooling , and highway layers allows training speed comparable to subword - level models without hard - coded text segmentation . To solve the problems that there is no sufficient annotated data in low-resource languages and it is hard to mine the deep semantic correspondence between languages via existing bilingual word embedding learning methods , this paper presents an effective text processing method based on transfer learning and bilingual word embedding model CWDR-BiGRU ( Cross-context window of dynamic ratio bidirectional Gated Recurrent Unit ) which contains an enhanced Skip-gram called cross-context window of dynamic ratio and encoder-decoder . The method can process low-resource language text effectively only using sentence-aligned corpus of bilingual resource languages and annotated data of high-resource language . The experimental results of semantic reasoning and word embedding visualization show that CWDR-BiGRU can effectively train bilingual word embeddings . In the task of Chinese-Tibetan cross-lingual document classification , the accuracy of transfer learning method based on CWDR-BiGRU is higher than the conventional method by 13.5 % , and higher than the existing Bilingual Autoencoder , BilBOWA , BiCCV and BiSkip by 7.4 % , 5.8 % , 3.1 % and 1.6 % respectively , indicating CWDR-BiGRU which has reduced the difficulty of acquiring corpora for bilingual word embeddings can accurately excavate the deep alignment relationship and semantic properties . This paper presents our submission for the SemEval shared task 6 , sub-task A on the identification of offensive language . Our proposed model , C-BiGRU , combines a Convolutional Neural Network ( CNN ) with a bidirectional Recurrent Neural Network ( RNN ) . We utilize word2vec to capture the semantic similarities between words . As the number of layers increases , deep learning can obtain more complex features , but it will cause the problem of gradient disappearance , and errors can not be effectively back-propagated , and the residual networks can solve it . This paper proposes a ResGCNN network combining residual network , multilayer BiGRU , and TextCNN . The excellent classification performance is obtained on multiple English and Chinese data sets . By training the word embedding vectors for the medical-health field , conducting entity recognition and relationship annotation , and using deep learning technology , we construct a relation extraction model for extracting the relationships among diseases , symptoms and tests . Our relationship extraction model of the bidirectional gate recurrent unit ( BiGRU ) network based on character-level and sentence-level attention mechanisms achieved the best results on question-answer data in the online health community . Our research results can not only help physician diagnoses but also help patients perform health management , which has important industrial application value . Furthermore , to effectively handle the imbalanced classification problem , this paper proposes a new loss function called the synthetic stimulation loss , which uses a modulating factor to allow the model to focus on hard-to-classify samples . The proposed two methods are integrated into a bidirectional gated recurrent unit ( BiGRU ) . As a single model is not strong in noise immunity , this paper applies the mutual learning method to our model and forces the networks to teach each other . The method enhances the semantic representation of words by BERT(Bidirectional Encoder Representations from Transformers ) pre-trained language model . BERT can generates the semantic vectors dynamically according to the context of the words , and then inputs the word vectors into BiGRU-CRF for training . The whole model can be trained during training .
We propose an autoencoding sequence-based transceiver for communication over dispersive channels with intensity modulation and direct detection ( IM/DD ) , designed as a bidirectional deep recurrent neural network ( BRNN ) . The receiver uses a sliding window technique to allow for efficient data stream estimation .
This study aims to automatically fuse voice and text data to provide tailored situational awareness information to paramedics . To train and test speech recognition models , we built a bidirectional deep recurrent neural network ( long short-term memory ( LSTM ) ) . Then we used convolutional neural networks on top of custom-trained word vectors for sentence-level classification tasks .
This article proposes a new feature extraction method that uses the word embedding method from natural language processing to generate bidirectional real dense vectors to reflect the contextual relationships between the pixels . A bidirectional independent recurrent neural network ( BiIndRNN ) is combined with a convolutional neural network ( CNN ) to improve the sliced recurrent neural network ( SRNN ) algorithm model , which is then constructed in parallel with graph convolutional networks ( GCNs ) under an attention mechanism to fully exploit the deep features of images and to capture the semantic information of the context . This model is collectively named an improved SRNN and attention-treated GCN-based parallel ( SAGP ) model .
The recently-proposed deep clustering algorithm introduced significant advances in monaural speaker-independent multi-speaker speech separation . Deep clustering operates on magnitude spectro-grams using bidirectional recurrent networks and K-means clustering , both of which require offline operation , i.e. , algorithm latency is longer than utterance length . This paper evaluates architectures for reduced latency deep clustering by combining : ( 1 ) block processing to efficiently propagate the memory encoded by the recurrent network , and ( 2 ) teacher-student learning , where low-latency models learn from an offline teacher .
We develop a phase-sensitive objective function based on the signal-to-noise ratio ( SNR ) of the reconstructed signal , and show that in experiments it yields uniformly better results in terms of signal-to-distortion ratio ( SDR ) . We also investigate improvements to the modeling of dynamics , using bidirectional recurrent networks , as well as by incorporating speech recognition outputs in the form of alignment vectors concatenated with the spectral input features . Both methods yield further improvements , pointing to tighter integration of recognition with separation as a promising future direction .
Bidirectional recurrent neural nets have demonstrated state-ofthe-art performance for parametric speech synthesis . In this paper , we introduce a top-down application of recurrent neural net models to unit-selection synthesis .
We introduce segmental recurrent neural networks ( SRNNs ) which define , given an input sequence , a joint probability distribution over segmentations of the input and labelings of the segments . Representations of the input segments ( i.e. , contiguous subsequences of the input ) are computed by encoding their constituent tokens using bidirectional recurrent neural nets , and these " segment embeddings " are used to define compatibility scores with output labels . These local compatibility scores are integrated using a global semi-Markov conditional random field .
We do this using three modalities ( 1 ) a screenshot of an image , ( 2 ) text descriptions of the functionality of GUI components sourced through Amazon ’s Mechanical Turk , and ( 3 ) parsed information from the screen hierarchy ’s XML dump . The first two modalities are aligned using a convolutional neural network , which detects objects in the screenshot and extracts salient features , paired with a bidirectional recurrent neural network which serves as a language model . Both of these models maps their respective modalities to a semantic space , and then aligns the two representations in that space .
The proposed model includes an encoder and a decoder . For each input text image , the encoder part firstly combines deep convolutional layers with bidirectional Recurrent Neural Network to generate an ordered , high-level feature sequence , which could avoid the complicated text segmentation pre-processing . Then in the decoder , a recurrent network with attention mechanism is developed to generate text line output , enabling the model to selectively exploit image features from the encoder correspondingly .
In this paper we introduce a new connectionist approach to on-line handwriting recognition and address in particular the problem of recognizing handwritten whiteboard notes . The approach uses a bidirectional recurrent neural network with the long short-term memory architecture . We use a recently introduced objective function , known as Connectionist Temporal Classification ( CTC ) , that directly trains the network to label unsegmented sequence data .
One of the main challenges that make this problem even harder is the uncertainty exists in the actions of pedestrians in urban traffic environments , as well as the difficulty in inferring their end goals . In this paper , we are proposing a data-driven framework based on inverse reinforcement learning ( IRL ) and the bidirectional recurrent neural network architecture ( B-LSTM ) for long-term prediction of VRUs ’ intention . We evaluated our framework on real-life datasets for agent behavior modeling in traffic environments , and it has achieved an overall average displacement error of only 2.93 and 4.12 pixels over 2.0 and 3.0 s ahead prediction horizons , respectively .
A new approach called Sigfind for the prediction of signal peptides in human protein sequences is introduced . The method is based on the bidirectional recurrent neural network architecture . The modifications to this architecture and a better learning algorithm result in a very accurate identification of signal peptides ( 99.5 % correct in fivefold crossvalidation ) .
Second , at the DNN output layer , parameters for pitch-scaled spectrum and aperiodicity measures are estimated for constructing the excitation signal used in our baseline synthesis vocoder . Third , the bidirectional recurrent neural network architecture with long short term memory ( BLSTM ) units is adopted and trained with multi-task learning for DNN-based speech synthesis . Experimental results demonstrate that the quality of synthesized speech has been improved by adopting the new input vector and output parameters .
In this study , we compare the performance of one-stage and two-stage architectures for DDI extraction . Our architectures are based on a bidirectional recurrent neural network layer composed of Gated Recurrent Units . The two-stage system obtained a 67.45 % micro-average F1 score on the test set .
The neural machine translation system is implemented as an encoder - decoder network with recurrent neural networks . The encoder is a bidirectional neural network with gated recurrent units [ reference ] that reads an input sequence x = ( x 1 , ... , x m ) and calculates a forward sequence of hidden states ( − → h 1 , ... , − → h m ) , and a backward sequence The hidden states − → h j and ←
The neural machine translation system is implemented as an encoder - decoder network with recurrent neural networks . The encoder is a bidirectional neural network with gated recurrent units [ reference ] that reads an input sequence x = ( x 1 , ... , x m ) and calculates a forward sequence of hidden states ( − → h 1 , ... , − → h m ) , and a backward sequence The hidden states − → h j and ←
These representations are passed through a softmax layer , producing a probability distribution over the possible labels for every token in context : where is the weight matrix between the hidden vector and the output layer . We investigate six alternative neural network architectures for the task of error detection : convolutional , bidirectional recurrent , bidirectional LSTM , and multi - layer variants of each of them . In the convolutional neural network ( CNN , Figure [ reference ] a ) for token labeling , the hidden vector is calculated based on a fixed - size context window .
It is designed to detect error patterns from a fixed window of 7 words , which is large enough to not require the use of more advanced composition functions . In contrast , the performance of the bidirectional recurrent network ( Bi - RNN ) is somewhat lower , especially on the test set . In Elman - type recurrent networks , the context signal from distant words decreases fairly rapidly due to the sigmoid activation function and diminishing gradients .
section : Sentence - based Character Model The feature that distinguishes our model most from previous work is that we apply a bidirectional recurrent layer ( LSTM ) on all characters of a sentence to induce fully context sensitive initial word encodings . That is , we do not restrict the context of this layer to the words themselves ( as in Figure 1b ) .
Kim2016 implemented a language model where encoding is performed by a convolutional network and LSTM over characters , whereas predictions are given on the word - level . Cao2016 proposed a method for learning both word embeddings and morphological segmentation with a bidirectional recurrent network over characters . There is also research on performing parsing and text classification with character - level neural models .
Encoding Layer encodes the representations by incorporating the context information or enriching the representation with desirable features for future use . For instance , a model can adopt bidirectional recurrent neural network to model the temporal interaction on both direction , recursive neural network ( also known as TreeRNN ) to model the compositionality and the recursive structure of language , or self - attention to model the long - term dependency on sentence . Different components of encoder can be combined to obtain a better sentence matrix representation .
The entire network is trained jointly such that the hidden representation captures the important information from the sequence for the prediction task . A bidirectional recurrent neural network ( bi - RNN ) is an extension of an RNN that reads the input sequence twice , from left to right and right to left , and the encodings are concatenated . The literature uses the term bi - RNN to refer to two related architectures , which we refer to here as “ context bi - RNN ” and “ sequence bi - RNN ” .
Each feature vector is extracted from a small overlapping window of audio frames . The encoder is implemented as a deep bidirectional recurrent network ( BiRNN ) , to form a sequential representation of length . At the - th step an ARSG generates an output by focusing on the relevant elements of : where is the - th state of the recurrent neural network to which we refer as the generator , is a vector of the attention weights , also often called the alignment .
The comparison of our models and this architecture is presented in § [ reference ] . paragraph : Multidimensional recurrent neural networks . There is another line of research that aims to extend RNNs to operate on multidimensional inputs .
Therefore , the integration of multilingual data resources has important application value and practical significance . In this paper , we propose a novel model based on LDA and BiLSTM-CNN to solve the multilingual short text classification problem . In our method , we make full use of topic vectors and word vectors to extract text information from two aspects , in each language , to solve the sparse problem of short text features .
Given the quality and quantity of data we have been given , we found that the addition of CNN layer provides very little to no additional improvement to our model 's performance and sometimes even worsen our F1-score . In the end , the deep learning architecture that gives us the highest macro F1-score is a simple BiLSTM-CNN .
This paper proposes a research sharing-oriented approach for functional neuroimaging named entity recognition . The nine most representative entity categories were defined by analyzing the characteristics of task-based functional neuroimaging researches , and a multi-category named entity recognition method was designed based on BiLSTM-CNN . An experiment was performed on literatures obtained from the journal PLoS One .
In order to solve the issues which are triggered by the lack of research on local adverse drug reactions in Xinjiang and the shortcomings of traditional models in dealing with irregular sentences , this paper proposes a method for adverse drug identification in Xinjiang . The method is combined with BiLSTM-CNN hybrid network which is based on attention mechanism . The method analyzes deeply on the network text context feature and the attention pooling mechanism .
Since we leverage results of Subtask 2 directly to infer the result of Subtask 1 , the paper focus on the system solving Subtask 2 . By taking Subtask 2 as a sequence labeling task , our system relies on a recurrent neural network named BiLSTM-CNNCRF with rich linguistic features , such as POS tags , dependency parsing labels , chunking labels , NER labels , Brown clustering . Our system achieved the highest F1 score in both token level and phrase level .
Experimental results show that the proposed BMCNN algorithm achieves state-of-the-art performance . In detail , BMCNN can restore both repetitive and irregular structures .
Experimental results show that the proposed BMCNN algorithm achieves state-of-the-art performance . In detail , BMCNN can restore both repetitive and irregular structures .
In particular , we introduce a novel dataset consisting of faculty listings from university webpages across the world in multiple languages and test different NER models in the task of extracting researcher names from these listings . We found that a neural network architecture that combines a bidirectional LSTM with a Conditional Random Fields output layer , LSTM-based character representations and a Hard Attention mechanism for HTML features outperforms other methods achieving 90.7 F1-score in the task . But , with the aid of clever strategies such as self-training , we can get a much simpler model , the second-order Hidden Markov Model , to achieve a 87.9 F1-score .
We propose two tasks : ( 1 ) predicting which argument from an argument pair is more convincing and ( 2 ) ranking all arguments to the topic based on their convincingness . We experiment with feature-rich SVM and bidirectional LSTM and obtain 0.76 - 0.78 accuracy and 0.35 - 0.40 Spearman ’s correlation in a cross-topic evaluation . We release the newly created corpus UKPConvArg1 and the experimental software under open licenses .
Recent advances on LU in H2 M conversations have shown accuracy improvements by adding encoded knowledge from different sources . Inspired by this , we explore several variants of a bidirectional LSTM architecture that relies on different knowledge sources , such as Web data , search engine click logs , expert feedback from H2 M models , as well as previous utterances in the conversation . We also propose ensemble techniques that aggregate these different knowledge sources into a single model .
However , when dealing with out-of-vocabulary words , there is still a large error rate . We used a simple bidirectional LSTM architecture and a large-scale pretrained language model to generate high-quality contextualize character representations , which successfully reduced the weakness of the ambiguous meanings of each Chinese character that widely appears in Chinese characters , and hence effectively reduced OOV error rate . State-of-the-art performance is achieved on many datasets .
The model integrates attention mechanism and target information analysis , which enables the model to concentrate on the important parts of the sentence and to make full use of the target information . The model uses bidirectional LSTM ( Bi-LSTM ) to build the memory of the sentence , and then CNN is applied to extracting attention from memory to get the attentive sentence representation . The model uses aspect embedding to analyze the target information of the representation and finally the model outputs the sentiment polarity through a softmax layer .
Using the Fake News Challenge ( FNC-1 ) dataset , we have developed different models to detect fake news based on the relation between article headline and article body . Our models are assembled mainly from Convolutional Neural Network ( CNN ) , Long Short-Term Memory network ( LSTM ) and Bidirectional LSTM ( Bi-LSTM ) . In the contrary of other studies on the same dataset where they reported accuracy for a test data derived from the same training dataset , our experiments achieved 71.2 % accuracy for the official testing dataset .
Smartphone can be used for the activity recognition of rehabilitation , aged care and so on . In this paper , we propose a bidirectional LSTM structure for human activity recognition using time series , collected from a smartphone ( with the accelerometer and gyroscope embedded on the phone ) worn on the waist of human , the best recognition accuracy of which can reach 93.79 % .
The BRA has two main characteristics : ( 1 ) the motion manifold is extracted by a bidirectional long short-term memory recurrent neural network ( B-LSTM-RNN ) and ( 2 ) apart from statistical information of motion data , kinematic information including smoothness and bone length constrain , are also simultaneously exploited with noisy-clean motion pairs . Using a bidirectional LSTM unit , which is more suitable for time series and can infer information from the data in both time directions , our autoencoder extracts a manifold that can exploit the spatial and temporal relationships between previous and subsequent motion data . As a result , the refined data that are projected by the decoder from the motion manifold have much lower reproduction error .
The Bilateral Multi - Perspective Matching ( BiMPM ) model obtains state - of - the - art performance on the SNLI dataset , achieving a maximum accuracy of 86.9 % , and going up to 88.8 % in an ensemble setup . An initial embedding layer vectorises words in P and H using pretrained GLoVe embeddings , and passing them to a context representation layer , which uses bidirectional LSTMs ( BiLSTMs ) to encode context vectors for each time - step . The core part of the model is the subsequent matching layer , where each contextual embedding or time - step of P is matched against all the embeddings of H , and vice versa .
In order to use this architecture for error detection , we create a bidirectional LSTM , making use of the advanced features of LSTM and incorporating context on both sides of the target token . In addition , we experiment with a deep bidirectional LSTM , which includes two consecutive layers of bidirectional LSTMs , modeling even more complex features and performing multiple passes through the sentence . For comparison with non - neural models , we also report results using CRFs , which are a popular choice for sequence labeling tasks .
The forward and backward activations from the previous LSTM layer are concatenated and fed into the next LSTM layer . The contrast model is a single layer bidirectional LSTM trained on 128 - dim features obtained by performing PCA on 512 - dimensional bottleneck features . The features are obtained from a 6 - layer DNN cross entropy trained on blocks of 11 consecutive FMLLR frames and 100 - dimensional i - vectors .
subsection : Sentence Encoder Our sentence encoder is simply composed of multiple stacked bidirectional LSTM ( biLSTM ) layers with shortcut connections followed by a max pooling layer . Let bilstm represent the th biLSTM layer , which is defined as : where is the output of the th biLSTM at time t over input sequence .
In this paper , we propose a novel approach to learn coreference-aware word representations for the NER task at the document level . In particular , we enrich the well-known neural architecture “ CNN-BiLSTM-CRF ” with a coreference layer on top of the BiLSTM layer to incorporate coreferential relations . Furthermore , we introduce the coreference regularization to ensure the coreferential entities to share similar representations and consistent predictions within the same coreference cluster .
The experimental results show that the prediction accuracy of the bidirectional RNN is obviously higher than that of the unidirectional RNN . In a small-scale training set , the effect of BiLSTM/BiGRU is close , while the training speed of BiGRU is faster , which can be used in the wheel detection system .
An optimized Bidirectional Long Short-Term Memory model based on deep attention mechanism is constructed . Then , using the power transformer defect text as the analysis object , the classification effect of DA-BiLSTM was tested . Finally , the classification effect is compared with several typical machine learning classification models . The experimental results of the simulation of the example show that , the classification effect of the DA-BiLSTM has a better advantage than several typical machine learning model .
As to algorithm , this paper utilizes the BERT-BiLSTM-CRF model to automatically identify the subject knowledge points , and then evaluates the relationship between the knowledge points by calculating their semantic similarity , PMI and Normalized Google Distance between them . The experimental results show that BERT-BiLSTM-CRF outperforms the baselines significantly , and the three kinds of relationship evaluation models have achieved good results . Finally , computer science and physics science are taken as examples to construct the subject knowledge graphs successfully , which show the effectiveness of our method .
In addition , we propose the SAMF-BiLSTM-D model based on SAMF-BiLSTM model for document-level text classification tasks . The method obtains the representation of all sentences in the document through SAMF-BiLSTM training , then integrates BiLSTM to learn the representation of all sentences , and further obtains the sentiment feature information of the entire document . Finally , we evaluate experiment results under five datasets .
Recurrent neural networks have proven to be very effective in learning distributed representations and can be trained efficiently on natural language inference tasks . We build on top of one such model and propose a hierarchy of BiLSTM and max pooling layers that implements an iterative refinement strategy and yields state of the art results on the SciTail dataset as well as strong results for SNLI and MultiNLI . We can show that the sentence embeddings learned in this way can be utilized in a wide variety of transfer learning tasks , outperforming InferSent on 7 out of 10 and SkipThought on 8 out of 9 SentEval sentence embedding evaluation tasks .
Following a deep learning paradigm , the whole architecture is trained in an end-to-end fashion in order to optimize two objectives : the detection of arousal onset and offset , and the classification of the type of arousal . The novelty of the approach is three-fold : it is the first use of a hybrid ST-BiLSTM network with biomedical signals ; it captures frequency information lower ( 0.1 Hz ) than the detection sampling rate ( 0.5 Hz ) ; and it requires no explicit mechanism to overcome class imbalance in the data . In the follow-up phase of the 2018 Physionet/CinC Challenge the proposed architecture achieved a state-of-the-art area under the precision-recall curve ( AUPRC ) of 0.50 on the hidden test data , tied for the second-highest official result overall .
Due to current feature based malicious URLs detection models need manual feature engineering , and deep learning based models have their limit on processing long sequences , which reduces the detection performance . We proposed an attentional based BiLSTM model AB-BiLSTM for the Malicious URLs detection in this paper . Firstly , the URLs were preprocessed and converted into word vectors by using pre-trained Word2Vec , then BiLSTM combined with an attention mechanism was trained to extract URL sequences features and classify them .
We propose two PP argumenthood prediction tasks branching from these two motivations : ( 1 ) binary argument/adjunct classification of PPs in VerbNet , and ( 2 ) gradient argumenthood prediction using human judgments as gold standard , and report results from prediction models that use pretrained word embeddings and other linguistically informed features . Our best results on each task are ( 1 ) $ acc.=0.955 $ , $ F_1=0.954 $ ( ELMo+BiLSTM ) and ( 2 ) Pearson 's $ r=0.624 $ ( word2vec+MLP ) . Furthermore , we demonstrate the utility of argumenthood prediction in improving sentence representations via performance gains on SRL when a sentence encoder is pretrained with our tasks .
We propose two PP argumenthood prediction tasks branching from these two motivations : ( 1 ) binary argument-adjunct classification of PPs in VerbNet , and ( 2 ) gradient argumenthood prediction using human judgments as gold standard , and report results from prediction models that use pretrained word embeddings and other linguistically informed features . Our best results on each task are ( 1 ) $ acc.=0.955 $ , $ F_1=0.954 $ ( ELMo+BiLSTM ) and ( 2 ) Pearson 's $ r=0.624 $ ( word2vec+MLP ) . Furthermore , we demonstrate the utility of argumenthood prediction in improving sentence representations via performance gains on SRL when a sentence encoder is pretrained with our tasks .
This paper explores the use of multi-task learning ( MTL ) for incorporating external knowledge in neural models . Specifically , we show how MTL can enable a BiLSTM sentiment classifier to incorporate information from sentiment lexicons . Our MTL set-up is shown to improve model performance ( compared to a single-task set-up ) on both English and Norwegian sentence-level sentiment datasets .
The BiLSTM excels at representing elements in a sequence ( i.e. , words ) together with their contexts , capturing the element and an ‘ ‘ infinite ’ ’ window around it . We represent each word by its BiLSTM encoding , and use a concatenation of a minimal set of such BiLSTM encodings as our feature function , which is then passed to a non - linear scoring function ( multi - layer perceptron ) . Crucially , the BiLSTM is trained with the rest of the parser in order to learn a good feature representation for the parsing problem .
The greedy transition - based parser follows standard techniques from the literature ( margin - based objective , dynamic oracle training , error exploration , MLP - based non - linear scoring function ) . We depart from the literature by replacing the hand - crafted feature function over carefully selected components of the configuration with a concatenation of BiLSTM representations of a few prominent items on the stack and the buffer , and training the BiLSTM encoder jointly with the rest of the network . section : Graph - based Parser
Because the Row LSTM has a triangular receptive field ( Figure [ reference ] ) , it is unable to capture the entire available context . subsection : Diagonal BiLSTM The Diagonal BiLSTM is designed to both parallelize the computation and to capture the entire available context for any image size .
To test whether this might be the case , we also ran an experiment with a 2 - layer BiLSTM ( + liq ) . This setup exhibits similar computational complexity and number of parameters to BiLSTM + + . We found that the second layer did not improve performance , suggesting that pooling over word / lemma occurrences in a given context between layers , is a powerful , yet simple technique .
Our results show that embedding refinement using both the system ’s text inputs , as well as supplementary text from external background knowledge can yield large improvements . In particular , we have shown that relatively simple task architectures ( e.g. , based on simple BiLSTM readers ) can become competitive with state of the art , task - specific architectures when augmented with our reading architecture . Our analysis demonstrates that our model learns to exploit provided background knowledge in a semantically appropriate way .
CONSTITUTION : A host-sided sub connection unit(21a ) and a display-side sub connection unit(22a ) include a bidirectional converter and a DC(Direct Current ) balancing codec . The bidirectional converts differential sub signals into single sub signal . The bidirectional converter converts single sub signal into differential sub signals .
We analyze the spatio-temporal characteristics of GIS , introduce the cellular automata , propose a formal definition of spatio-temporal ontology and a method based on spatio-temporal semantic granularity , and discuss an integrated GIS architecture based on ontology and semantics . We implemented the eartly-stage creation and browse of the railway passenger traffic spatio-temporal ontology and the sharing and reuse of ontology among the GIS sub-systems by tegrating bidirectional in the experimental railway GIS system project .
DC/AC microgrid is independent control grid form of integrate distributed energy systems with utility powers systems ; will powerfully understand the value and profit of the distributed energy resources . In this paper , the AC/DC micro-grid mostly contains of six parts : a solar source ( P V source ) , Wind source , a Boost converter , BESS , a Buck-Boost converter ( bidirectional ) , and a bi-directional DC/AC which is associated to the utility grid . The DC/AC microgid have two operation mode ( islanded or grid disconnect and grid-connected ) .
Bi - GRU Encoder : We apply bidirectional GRU for encoding query and passage into vector representations . We set the number of hidden units to be 256 and 384 for the CNN and Daily Mail datasets , respectively .
A similar approach was taken by Lee2016 for character - level machine translation . Their model ’s encoder uses a convolutional layer followed by max - pooling to reduce sequence length , a four - layer highway network , and a bidirectional GRU . The parallelism of the convolutional , pooling , and highway layers allows training speed comparable to subword - level models without hard - coded text segmentation .
To solve the problems that there is no sufficient annotated data in low-resource languages and it is hard to mine the deep semantic correspondence between languages via existing bilingual word embedding learning methods , this paper presents an effective text processing method based on transfer learning and bilingual word embedding model CWDR-BiGRU ( Cross-context window of dynamic ratio bidirectional Gated Recurrent Unit ) which contains an enhanced Skip-gram called cross-context window of dynamic ratio and encoder-decoder . The method can process low-resource language text effectively only using sentence-aligned corpus of bilingual resource languages and annotated data of high-resource language . The experimental results of semantic reasoning and word embedding visualization show that CWDR-BiGRU can effectively train bilingual word embeddings . In the task of Chinese-Tibetan cross-lingual document classification , the accuracy of transfer learning method based on CWDR-BiGRU is higher than the conventional method by 13.5 % , and higher than the existing Bilingual Autoencoder , BilBOWA , BiCCV and BiSkip by 7.4 % , 5.8 % , 3.1 % and 1.6 % respectively , indicating CWDR-BiGRU which has reduced the difficulty of acquiring corpora for bilingual word embeddings can accurately excavate the deep alignment relationship and semantic properties .
This paper presents our submission for the SemEval shared task 6 , sub-task A on the identification of offensive language . Our proposed model , C-BiGRU , combines a Convolutional Neural Network ( CNN ) with a bidirectional Recurrent Neural Network ( RNN ) . We utilize word2vec to capture the semantic similarities between words .
As the number of layers increases , deep learning can obtain more complex features , but it will cause the problem of gradient disappearance , and errors can not be effectively back-propagated , and the residual networks can solve it . This paper proposes a ResGCNN network combining residual network , multilayer BiGRU , and TextCNN . The excellent classification performance is obtained on multiple English and Chinese data sets .
By training the word embedding vectors for the medical-health field , conducting entity recognition and relationship annotation , and using deep learning technology , we construct a relation extraction model for extracting the relationships among diseases , symptoms and tests . Our relationship extraction model of the bidirectional gate recurrent unit ( BiGRU ) network based on character-level and sentence-level attention mechanisms achieved the best results on question-answer data in the online health community . Our research results can not only help physician diagnoses but also help patients perform health management , which has important industrial application value .
Furthermore , to effectively handle the imbalanced classification problem , this paper proposes a new loss function called the synthetic stimulation loss , which uses a modulating factor to allow the model to focus on hard-to-classify samples . The proposed two methods are integrated into a bidirectional gated recurrent unit ( BiGRU ) . As a single model is not strong in noise immunity , this paper applies the mutual learning method to our model and forces the networks to teach each other .
The method enhances the semantic representation of words by BERT(Bidirectional Encoder Representations from Transformers ) pre-trained language model . BERT can generates the semantic vectors dynamically according to the context of the words , and then inputs the word vectors into BiGRU-CRF for training . The whole model can be trained during training .
End-to-End Optimized Transmission over Dispersive Intensity-Modulated Channels Using Bidirectional Recurrent Neural Networks Transforming unstructured voice and text data into insight for paramedic emergency service using recurrent and convolutional neural networks Pixel-Level Remote Sensing Image Recognition Based on Bidirectional Word Vectors Teacher-student Deep Clustering for Low-delay Single Channel Speech Separation Phase-sensitive and recognition-boosted speech separation using deep recurrent neural networks Unit Selection with Hierarchical Cascaded Long Short Term Memory Bidirectional Recurrent Neural Nets Segmental Recurrent Neural Networks Clarity: An Exploration of Semantic Information Encoded in Mobile Application GUIs End-to-End Chinese Image Text Recognition with Attention Model A novel approach to on-line handwriting recognition based on bidirectional long short-term memory networks Contextual Recurrent Predictive Model for Long-Term Intent Prediction of Vulnerable Road Users Finding Signal Peptides in Human Protein Sequences Using Recurrent Neural Networks Improving Deep Neural Network Based Speech Synthesis through Contextual Feature Parametrization and Multi-Task Learning Comparando enfoques deep learning en una fase y en dos fases para extraer interacciones farmacológicas de texto Neural Machine Translation of Rare Words with Subword Units Improving Neural Machine Translation Models with Monolingual Data Compositional Sequence Labeling Models for Error Detection in Learner Writing Morphosyntactic Tagging with a Meta-BiLSTM Model over Context Sensitive Token Encodings Attending to Characters in Neural Sequence Labeling Models Natural Language Inference over Interaction Space Multilingual Part-of-Speech Tagging with Bidirectional Long Short-Term Memory Models and Auxiliary Loss Attention-Based Models for Speech Recognition Cell-aware Stacked LSTMs for Modeling Sentences Multilingual Short Text Classification Based on LDA and BiLSTM-CNN Neural Network. Transforma at SemEval-2019 Task 6: Offensive Language Analysis using Deep Learning Architecture Research Sharing-Oriented Functional Neuroimaging Named Entity Recognition Identification of Local Adverse Drug Reactions in Xinjiang Based on Attention Mechanism and BiLSTM-CNN Hybrid Network DM_NLP at SemEval-2018 Task 8: neural sequence labeling with linguistic features Block-Matching Convolutional Neural Network (BMCNN): Improving CNN-Based Denoising by Block-Matched Inputs Block-Matching Convolutional Neural Network for Image Denoising Named entity recognition on the Web Which argument is more convincing? Analyzing and predicting convincingness of Web arguments using bidirectional LSTM Slot Tagging for Task Oriented Spoken Language Understanding in Human-to-Human Conversation Scenarios Robust Chinese Word Segmentation with Contextualized Word Representations A Bi-Directional LSTM-CNN Model with Attention for Aspect-Level Text Classification A Closer Look at Fake News Detection: A Deep Learning Perspective Human Activity Recognition with Smartphone Inertial Sensors Using Bidir-LSTM Networks Bidirectional recurrent autoencoder for 3D skeleton motion data refinement Grounded Textual Entailment Compositional Sequence Labeling Models for Error Detection in Learner Writing The IBM 2016 English Conversational Telephone Speech Recognition System Shortcut-Stacked Sentence Encoders for Multi-Domain Inference Coreference Aware Representation Learning for Neural Named Entity Recognition Wheel Center Detection From Point Cloud Based on Recurrent Neural Networks Defect Text Analysis Method of Electric Power Equipment Based on Double-Layer Bidirectional LSTM Model Automatic Construction of Subject Knowledge Graph based on Educational Big Data Bidirectional LSTM with self-attention mechanism and multi-channel features for sentiment classification Sentence embeddings in NLI with iterative refinement encoders Hybrid scattering-LSTM networks for automated detection of sleep arousals. A Bi-Directional LSTM Model with Attention for Malicious URL Detection Predicting Argumenthood of English Preposition Phrases Predicting the Argumenthood of English Prepositional Phrases Lexicon information in neural sentiment analysis: a multi-task learning approach Simple and Accurate Dependency Parsing Using Bidirectional LSTM Feature Representations Pixel Recurrent Neural Networks Dynamic Integration of Background Knowledge in Neural NLU Systems Display-Port connector Research on Spatio-temporal Ontology and Its Application in Geographic Information System Design and analysis of a DC/AC microgrid with centralized battery energy storage system ReasoNet: Learning to Stop Reading in Machine Comprehension Quasi-Recurrent Neural Networks A Word Embeddings Training Method Based on Modified Skip-Gram and Align nlpUP at SemEval-2019 Task 6: A Deep Neural Language Model for Offensive Language Detection Text Sentiment Analysis Based on ResGCNN Disease-Pertinent Knowledge Extraction in Online Health Communities Using GRU Based on a Double Attention Mechanism Relation Classification via Keyword-Attentive Sentence Mechanism and Synthetic Stimulation Loss Research on Chinese Naming Recognition Model Based on BERT Embedding
5e972375c85ca0ff6bd61841a78e78361540bac5 b9fa7565f3bc3c8f34ebf658b06d2e70b23dba76 0f14693402bf417577324c29350bb40682e3d168 7a380cdae912cfdf4504a3fedfbbb216dbf16c4e 3004a3e4d8969dc3c36c9274b0f76ecc874f2e6a b2a7687709dc217ddba60045137d452f62ac7c85 6b904d6e84c98c6ce22ce6923224b205a2a24ee1 7f9c1c88e44d8b44e9d2c2313d717d1270654b5a a0640779689b502c53b2a5c1b9d008360e564b74 43fa5235e49fa6f16d047c999234d1b93df360b0 b019d949b857e30698871e0ba1976af5d9e201ee ed7069ad1857e641ca5f135d2dd8b6f878abb886 d7050be7b8fe523368619b8f82798e1be94ca0ff f444421e3e241cba659cb78167c5e57624c81ec1 2cdcf918321e98a7b04881af2b2d0fccf212cf00 ff1a9fe9ce606e647ff610648b357a478a2968c7 dae72982bb383724736acd19038a4ac19cb006ea 88cac50d68497a596ca7059b9a08b9fabcb2baa1 fa9cabb75f1a54470b004bba1311dae09a552a96 7d68a952185e9840fc057e3fa72d3dceff893032 96af12f384a998ca05db74d3f36487ca1fa7eee4 2a271f0897286a3f4e89fbcdea5a3239e351f7e1 25ae911c13da7ef9def56ee30170920ebd48a668 972155e272eb40ea604c24303a30dab4e5031a6f 5b2079a501c3c39987a7ca41039f1a05184ee101 ed4d015589f0eb36222c0f4d6524b095ef5d542e 5db07006338a2123375debfd174c53c30f43b10a 6e0344c477c8eec2b8bbdbedfcc1d362237465e0 c32229283fdf1797a738930ea620f6714984ff46 2c86f451f6b0c3218e08a12dc82e09dc4a818508 f5e185719d669912282a5fe9078e8ecf1bc6cd52 c687de9a47df40376c31bbcc28ea9d81797202e9 14f33949171b28746fb32405b6b65749bdb84e3f d097d2c9671a4f9e3c1df09d9c8b77dc37de5ef3 8775df853e1d6ab440685c00194deb2a2d1d2e4d 53c3d659c2aeaf26adfc9b9aa5bc92d60bf9772e 5cc8ec7439b49a0a9f65dc068136d4d474270eed 611f838579b0fa05ef6ac4557bd3b7e8f5956ad6 4b9bdc6b9a53ea860a8664330f65c49fb40b70a1 0efb3bddce9053f71f802ed8631dfa0ea355af3a 0a4e965623edf4a5e33767d0f43e66eb0d619513 38809583f01fc0771472f1f638bd30cbdca48b11 ee16c5f6c37e0ae2d3169ed255079f53b464420e 101a1242d147b785a8881f66bf89839cbe24099b b1a586582bb954eb0a5def814c7bb06e69043ccc 99cc518243d5dbf716bf11c0221b22b3d573d864 71b1c711985b9752c3247f972bd1cc1e49f3b249 b5c15eabbc749e7d7c9500f50a6d622d140d85e2 c05692dd5c69b142c0f5e1c796b120794cbfe3cd
["Physics","Medicine","Mathematics","Computer Science"] ["Computer Science","Engineering"] ["Mathematics","Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Engineering","Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Mathematics","Computer Science"] ["Computer Science"] ["Medicine"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Engineering","Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"]
2019 2020 2020 2019 2015 2017 2015 2018 2017 2007 2020 2002 2018 2020 2016 2016 2016 2018 2016 2018 2016 2015 2019 2019 2019 2019 2020 2018 2018 2017 2019 2016 2019 2019 2018 2019 2018 2019 2018 2016 2016 2017 2019 2018 2019 2020 2020 2019 2019 2019 2019 2018 2019 2016 2016 2018 2010 2005 2017 2016 2017 2018 2019 2019 2020 2019 2019
ICASSP ICASSP INTERSPEECH ICONIP (3) WABI ACL ACL ACL ACL COLING ICLR ACL NIPS ACML WISA BIBM SemEval@NAACL-HLT APSIPA ACL (1) CoNLL ICAAI COLING ACL INTERSPEECH RepEval@EMNLP IJCAI ICARM ICBDE NODALIDA Transactions of the Association for Computational Linguistics ICML IECON CoCo@NIPS ICLR ICCCS (6) SemEval@NAACL-HLT
https://api.semanticscholar.org/5e972375c85ca0ff6bd61841a78e78361540bac5 https://api.semanticscholar.org/b9fa7565f3bc3c8f34ebf658b06d2e70b23dba76 https://api.semanticscholar.org/0f14693402bf417577324c29350bb40682e3d168 https://api.semanticscholar.org/7a380cdae912cfdf4504a3fedfbbb216dbf16c4e https://api.semanticscholar.org/3004a3e4d8969dc3c36c9274b0f76ecc874f2e6a https://api.semanticscholar.org/b2a7687709dc217ddba60045137d452f62ac7c85 https://api.semanticscholar.org/6b904d6e84c98c6ce22ce6923224b205a2a24ee1 https://api.semanticscholar.org/7f9c1c88e44d8b44e9d2c2313d717d1270654b5a https://api.semanticscholar.org/a0640779689b502c53b2a5c1b9d008360e564b74 https://api.semanticscholar.org/43fa5235e49fa6f16d047c999234d1b93df360b0 https://api.semanticscholar.org/b019d949b857e30698871e0ba1976af5d9e201ee https://api.semanticscholar.org/ed7069ad1857e641ca5f135d2dd8b6f878abb886 https://api.semanticscholar.org/d7050be7b8fe523368619b8f82798e1be94ca0ff https://api.semanticscholar.org/f444421e3e241cba659cb78167c5e57624c81ec1 https://www.semanticscholar.org/paper/1af68821518f03568f913ab03fc02080247a27ff https://www.semanticscholar.org/paper/f3b96ef2dc1fc5e14982f1b963db8db6a54183bb https://www.semanticscholar.org/paper/2ebc224d52761d5b76704c7b8f51369247a73d5f https://www.semanticscholar.org/paper/cc5582606aef717855bb37c7aa35639da755cb13 https://www.semanticscholar.org/paper/f6223a7549a6f35de2e9dfa5a27a9f93de304f47 https://www.semanticscholar.org/paper/1778e32c18bd611169e64c1805a51abff341ca53 https://www.semanticscholar.org/paper/acd87e4f672f0b92ea4164414c213560c23bee52 https://www.semanticscholar.org/paper/b624504240fa52ab76167acfe3156150ca01cf3b https://www.semanticscholar.org/paper/249b3b7421d3cdb932eecfe4b67203e0e46806b2 https://api.semanticscholar.org/2cdcf918321e98a7b04881af2b2d0fccf212cf00 https://api.semanticscholar.org/ff1a9fe9ce606e647ff610648b357a478a2968c7 https://api.semanticscholar.org/dae72982bb383724736acd19038a4ac19cb006ea https://api.semanticscholar.org/88cac50d68497a596ca7059b9a08b9fabcb2baa1 https://api.semanticscholar.org/fa9cabb75f1a54470b004bba1311dae09a552a96 https://api.semanticscholar.org/7d68a952185e9840fc057e3fa72d3dceff893032 https://api.semanticscholar.org/96af12f384a998ca05db74d3f36487ca1fa7eee4 https://api.semanticscholar.org/2a271f0897286a3f4e89fbcdea5a3239e351f7e1 https://api.semanticscholar.org/25ae911c13da7ef9def56ee30170920ebd48a668 https://api.semanticscholar.org/972155e272eb40ea604c24303a30dab4e5031a6f https://api.semanticscholar.org/5b2079a501c3c39987a7ca41039f1a05184ee101 https://api.semanticscholar.org/ed4d015589f0eb36222c0f4d6524b095ef5d542e https://api.semanticscholar.org/5db07006338a2123375debfd174c53c30f43b10a https://api.semanticscholar.org/6e0344c477c8eec2b8bbdbedfcc1d362237465e0 https://api.semanticscholar.org/c32229283fdf1797a738930ea620f6714984ff46 https://www.semanticscholar.org/paper/1235dd37312cb20aced0e97d953f6379d8a0c7d4 https://www.semanticscholar.org/paper/2ebc224d52761d5b76704c7b8f51369247a73d5f https://www.semanticscholar.org/paper/05d2700846c0323f79c1344aca5333994c7c03a5 https://www.semanticscholar.org/paper/26fe009b958e8728382d9d764bd7153632f0b869 https://api.semanticscholar.org/2c86f451f6b0c3218e08a12dc82e09dc4a818508 https://api.semanticscholar.org/f5e185719d669912282a5fe9078e8ecf1bc6cd52 https://api.semanticscholar.org/c687de9a47df40376c31bbcc28ea9d81797202e9 https://api.semanticscholar.org/14f33949171b28746fb32405b6b65749bdb84e3f https://api.semanticscholar.org/d097d2c9671a4f9e3c1df09d9c8b77dc37de5ef3 https://api.semanticscholar.org/8775df853e1d6ab440685c00194deb2a2d1d2e4d https://api.semanticscholar.org/53c3d659c2aeaf26adfc9b9aa5bc92d60bf9772e https://api.semanticscholar.org/5cc8ec7439b49a0a9f65dc068136d4d474270eed https://api.semanticscholar.org/611f838579b0fa05ef6ac4557bd3b7e8f5956ad6 https://api.semanticscholar.org/4b9bdc6b9a53ea860a8664330f65c49fb40b70a1 https://api.semanticscholar.org/0efb3bddce9053f71f802ed8631dfa0ea355af3a https://www.semanticscholar.org/paper/eec3a236ecd185712ce65fb336141f8656eea13d https://www.semanticscholar.org/paper/41f1d50c85d3180476c4c7b3eea121278b0d8474 https://www.semanticscholar.org/paper/2f92b10acf7c405e55c74c1043dabd9ded1b1800 https://api.semanticscholar.org/0a4e965623edf4a5e33767d0f43e66eb0d619513 https://api.semanticscholar.org/38809583f01fc0771472f1f638bd30cbdca48b11 https://api.semanticscholar.org/ee16c5f6c37e0ae2d3169ed255079f53b464420e https://www.semanticscholar.org/paper/c636a2dd242908fe2e598a1077c0c57bfdea8633 https://www.semanticscholar.org/paper/2d876ed1dd2c58058d7197b734a8e4d349b8f231 https://api.semanticscholar.org/101a1242d147b785a8881f66bf89839cbe24099b https://api.semanticscholar.org/b1a586582bb954eb0a5def814c7bb06e69043ccc https://api.semanticscholar.org/99cc518243d5dbf716bf11c0221b22b3d573d864 https://api.semanticscholar.org/71b1c711985b9752c3247f972bd1cc1e49f3b249 https://api.semanticscholar.org/b5c15eabbc749e7d7c9500f50a6d622d140d85e2 https://api.semanticscholar.org/c05692dd5c69b142c0f5e1c796b120794cbfe3cd