We introduce a novel method for representation learning that uses an artificial supervision signal based on counting visual primitives . This supervision signal is obtained from an equivariance relation , which does not require any manual annotation . However , it is hard to know which method is optimal for a given dataset or which hyperparameters to use for a chosen method . To this end , we provide a framework to characterize the loss of a causal inference method against the true ACE , by framing causal inference as a representation learning problem . We show that many popular methods , including back-door methods can be considered as weighting or representation learning algorithms , and provide general error bounds for their causal estimates . Motivated by theories of language and communication that explain why communities with large numbers of speakers have , on average , simpler languages with more regularity , we cast the representation learning problem in terms of learning to communicate . Our starting point sees the traditional autoencoder setup as a single encoder with a fixed decoder partner that must learn to communicate . Nonetheless , gold POS tags still yield better parsing results as compared to the baseline / AT POS taggers , supporting the claim that POS tagging needs further improvement for downstream tasks . subsection : Effects on Representation Learning English ( WSJ ) GANs provide an attractive alternative to maximum likelihood techniques . One can additionally argue that their learning process and the lack of a heuristic cost function ( such as pixel - wise independent mean - square error ) are attractive to representation learning . GANs have been known to be unstable to train , often resulting in generators that produce nonsensical outputs . Similar to SDNE , our models rely on the autoencoder to learn non - linear node embeddings from local graph neighborhoods . However , our models have several important distinctions : 1 ) we leverage extensive parameter sharing between the encoder and decoder parts to enhance representation learning ; 2 ) our LoNGAE model can optionally concatenate side node features to the adjacency matrix for improved link prediction performance ; and 3 ) the LoNGAE model can be trained end - to - end in a single stage for multi - task learning of link prediction and semi - supervised node classification . Sentence embedding is an effective feature representation for most deep learning-based NLP tasks . One prevailing line of methods is using recursive latent tree-structured networks to embed sentences with task-specific structures . To address these challenges , we proposed DeepEnroll , a cross-modal inference learning model to jointly encode enrollment criteria ( text ) and patients records ( tabular data ) into a shared latent space for matching inference . DeepEnroll applies a pre-trained Bidirectional Encoder Representations from Transformers(BERT ) model to encode clinical trial information into sentence embedding . And uses a hierarchical embedding model to represent patient longitudinal EHR . This paper discusses two patterns related to sentential embedding . First , it deals with the effect of verb movement and adjunction on the optionality of complementizers ; in this context it offers an explanation of Sten Vikner 's generalisation that optional complementizers are only found in languages without obligatory verb movement . The comparison of verbs that can take CP complements with verbs that can take NP complements , when these verbs appeared with prepositional phrase complements , revealed activations in bilateral anterior MTG and precuneus , showing that embedding also affects processing at the lexical level . The results thus show that sentential embedding , even without syntactic movement , loads onto several cortical resources including the left IFG , indicating that the generation of syntactic layers is cognitively costly , and that embedding affects processing at both the sentence and the lexical levels . This paper proposes how to mine concerns and reviews that are relevant to TV dramas from the results of collecting Web pages on those TV dramas . The proposed framework consists of the natural language processing based techniques of analyzing sentence embeddings of those collected Web pages . More specifically , we apply the technique of BERT to the task of judging whether the theme of a collected Web page is relevant to the given drama . Such models provide low-dimensional vectorized representations of text which are informed by large corpora but can be applied to small-data NLP tasks . Example architectures include , for static word embeddings , word2vec/skip-grams ( Mikolov et al. 2013 ) , GloVe ( Pennington et al. 2014 ) and fasttext ( Bojanowski et al. 2017 ) , for sentence and paragraph embeddings , doc2vec ( Le & Mikolov 2014 ) and sent2vec ( Pagliardini et al. 2018 ) , and for context-sensitive word and sequence embeddings , ULM-Fit ( Howard & Ruder 2018 ) and BERT ( Devlin et al. 2018 ) , the latter adapting the transformer architecture of Vaswani et al. ( 2017 ) . These models have found use within the biomedical domain , particularly for processing scientific literature , where the NCBI ’s PubMed and MedLine databases provide a large , freely available data source for model training . Sentiment embedding based on the word embedding can capture both semantics and the emotion information . We have introduced two kinds of improving embedding methods ( MEC and Emo2Vec ) for the sentiment words embedding . For emotion structure based on the psychology of emotion , knowledge graph can represent the cognitive relations between different emotion types . When an attention mechanism is used to compute a representation of a single sequence , it is commonly referred to as self - attention or intra - attention . Together with Recurrent Neural Networks ( RNNs ) or convolutions , self - attention has proven to be useful for tasks such as machine reading [ reference ] and learning sentence representations [ reference ] . However , [ reference ] showed that not only self - attention can improve a method based on RNNs or convolutions , but also that it is sufficient for constructing a powerful model obtaining state - of - the - art performance on the machine translation task . Inspired by this recent work , we introduce an attention - based architecture to perform node classification of graph - structured data . Many - task systems are less commonly developed . Collobert2008AUA propose a many - task system sharing word embeddings between the tasks , hashimoto2016joint train a many - task model where the tasks are arranged hierarchically according to their linguistic level , and subramanian2018learning train a shared - encoder many - task model for the purpose of learning better sentence representations for use in downstream tasks , not for improving results on the original tasks . section : Conclusion Then a neural network classifier is applied to decide relationship between the two sentences . Different neural networks have been utilized for sentence encoding , such as LSTM DBLP : conf / emnlp / BowmanAPM15 , GRU DBLP : journals / corr / VendrovKFU15 , CNN DBLP : conf / acl / MouMLX0YJ16 , BiLSTM and its variants DBLP : journals / corr / Finally , we show that it is possible to exploit the strengths of this model without the need for an external parser by integrating a fast parser into the model ( as in the full SPINN ) , and that the lack of external parse information yields little loss in accuracy . Because this paper aims to introduce a general purpose model for sentence encoding , we do not pursue the use of soft attention , despite its demonstrated effectiveness on the SNLI task . However , we expect that it should be possible to productively combine our model with soft attention to reach state - of - the - art performance . To focus specifically on the strengths of these models at producing informative sentence representations , we use sentence embedding as an intermediate step in the NLI classification task : each model must produce a vector representation of each of the two sentences without using any context from the other sentence , and the two resulting vectors are then passed to a neural network classifier which predicts the label for the pair . This choice allows us to focus on existing models for sentence embedding , and it allows us to evaluate the ability of those models to learn useful representations of meaning ( which may be independently useful for subsequent tasks ) , at the cost of excluding from consideration possible strong neural models for NLI that directly compare the two inputs at the word or phrase level . label= [ text width=40 mm , align = center ] Distributed representations of phrases and sentences are also the focus of Socher et al . . Their methods typically require parsing and is shown to work for sentence - level representations . And it is not obvious how to extend their methods beyond single sentences . It captures natural language inference , also known in previous incarnations as Recognizing Textual Entailment ( RTE ) , and constitutes one of the largest high - quality labeled resources explicitly constructed in order to require understanding sentence semantics . We hypothesize that the semantic nature of NLI makes it a good candidate for learning universal sentence embeddings in a supervised way . That is , we aim to demonstrate that sentence encoders trained on natural language inference are able to learn sentence representations that capture universally useful features . Recently , a number of competitive methods have tackled unsupervised representation learning by maximising the mutual information between the representations produced from augmentations . The resulting representations are then invariant to stochastic augmentation strategies , and can be used for downstream tasks such as clustering or classification . Hereby , we differentiate between approaches based on Convolutional Neural Networks ( CNNs ) and Recurrent Neural Networks ( RNNs ) . We then introduce and evaluate three state-of-the-art deep learning systems for unsupervised representation learning from raw audio : ( 1 ) pre-trained image classification CNNs , ( 2 ) a deep convolutional generative adversarial network ( DCGAN ) , and ( 3 ) a recurrent sequence-to-sequence autoencoder ( S2SAE ) . For each of these algorithms , the representations are obtained from the spectrograms of the input audio data . Our method , which we call Deep InfoMax ( DIM ) , outperforms a number of popular unsupervised learning methods and competes with fully-supervised learning on several classification tasks . DIM opens new avenues for unsupervised learning of representations and is an important step towards flexible formulations of representation-learning objectives for specific end-goals . We do so through the introduction of a differentiable approximation bridge ( DAB ) neural network which provides smooth approximations to the gradient of the non-differentiable function . We present strong empirical results ( performing over 600 experiments ) in three different domains : unsupervised ( image ) representation learning , image classification , and sequence sorting to demonstrate that our proposed method improves state of the art performance . We demonstrate that utilizing non-differentiable functions in unsupervised ( image ) representation learning improves reconstruction quality and posterior linear separability by 10 % . We do so by training with a differentiable approximation bridge ( DAB ) neural network which approximates the non-differentiable forward function and provides gradient updates during backpropagation . We present strong empirical results ( performing over 600 experiments ) in four different domains : unsupervised ( image ) representation learning , variational ( image ) density estimation , image classification , and sequence sorting to demonstrate that our proposed method improves state of the art performance . We demonstrate that training with DAB aided discrete non-differentiable functions improves image reconstruction quality and posterior linear separability by 10 % against the Gumbel-Softmax relaxed estimator [ 37 , 26 ] as well as providing a 9 % improvement in the test variational lower bound in comparison to the state of the art RELAX [ 16 ] discrete estimator . We present Momentum Contrast ( MoCo ) for unsupervised visual representation learning . From a perspective on contrastive learning as dictionary look-up , we build a dynamic dictionary with a queue and a moving-averaged encoder . We propose two flavors of this approach , whereby the positive samples for those triplets whose anchors are unlabeled are obtained either by applying a transformation to the anchor , or by selecting the nearest sample in the training set . We compare our approach to supervised and unsupervised representation learning as well as the ratio between the amount of labeled and unlabeled data . We evaluate all the above approaches on an audio tagging task using the DCASE 2018 Task 4 dataset , and we show the impact of this ratio on the tagging performance . We propose ProtoNCE loss , a generalized version of the InfoNCE loss for contrastive learning by encouraging representations to be closer to their assigned prototypes . PCL achieves state-of-the-art results on multiple unsupervised representation learning benchmarks , with > 10 % accuracy improvement in low-resource transfer tasks . We build a joint multimodal model of text and images for automatically assigning illustrative images to journalistic articles . We approach the task as an unsupervised representation learning problem of finding a common representation that abstracts from individual modalities , inspired by multimodal Deep Boltzmann Machine of Srivastava and Salakhutdinov . We use state-of-theart image content classification features obtained from the Convolutional Neural Network of Krizhevsky et al. as input “ images ” and entire documents instead of keywords as input texts . It rather relies on an automatically generated surrogate task . The task is created by taking the idea of data augmentation – which is commonly used in supervised learning – to the extreme . Starting with trivial surrogate classes consisting of one random image patch each , we augment the data by applying a random set of transformations to each patch . subsection : Feature Visualization The visualization experiments ’ aim is to qualitatively investigate whether popular models such as the restricted Boltzman machine ( RBM ) can be combined with Patchy - san for unsupervised feature learning . For every input graph , we have generated receptive fields for all nodes and used these as input to an RBM . One of the early deep learning models which has been used for image denoising is the Stacked Denoising Auto - encoders ( SdA ) . It is an extension of the stacked auto - encoder and was originally designed for unsupervised feature learning . Denoising auto - encoders can be stacked to form a deep network by feeding output of the previous layer to the current layer as input . Graph Kernels [ reference ] have been proposed as a way to use relational data as part of the classification process , but are quite slow unless approximated [ reference ] . Our approach is complementary ; instead of encoding the structure as part of a kernel function , we learn a representation which allows them to be used directly as features for any classification method . section : Unsupervised Feature Learning Distributed representations have been proposed to model structural relationship between concepts Thus , to be useful , an unsupervised learning algorithm must in effect correctly guess the likely set of downstream classification tasks without being directly exposed to them . A significant fraction of unsupervised learning research is driven by generative modelling . It is motivated by the belief that the ability to synthesize , or " create " the observed data entails some form of understanding , and it is hoped that a good generative model will automatically learn a disentangled representation , even though it is easy to construct perfect generative models with arbitrarily bad representations . section : Related Work There exists a large body of work on unsupervised representation learning . Early methods were based on stacked ( often denoising ) autoencoders or restricted Boltzmann machines [ reference ][ reference ][ reference ][ reference ] . document : Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks In recent years , supervised learning with convolutional networks ( CNNs ) has seen huge adoption in computer vision applications . Unsupervised representation learning is a fairly well studied problem in general computer vision research , as well as in the context of images . A classic approach to unsupervised representation learning is to do clustering on the data ( for example using K - means ) , and leverage the clusters for improved classification scores . In the context of images , one can do hierarchical clustering of image patches coates2012learning to learn powerful image representations . This suggests that adversarial examples expose fundamental blind spots in our training algorithms . The cause of these adversarial examples was a mystery , and speculative explanations have suggested it is due to extreme nonlinearity of deep neural networks , perhaps combined with insufficient model averaging and insufficient regularization of the purely supervised learning problem . We show that these speculative hypotheses are unnecessary . section : Supervised training data for reading comprehension The reading comprehension task naturally lends itself to a formulation as a supervised learning problem . Specifically we seek to estimate the conditional probability , where is a context document , a query relating to that document , and the answer to that query . DistBelief has been used to achieve state - of - the - art results in several domains and has been shown to be much faster than single GPU training . Existing work on distributed deep learning has focused exclusively on supervised and unsupervised learning . In this paper we develop a new architecture for the reinforcement learning paradigm . These models are simple , expressive and can be trained efficiently with the highly scalable stochastic gradient back - propagation . Our neural variational framework is suitable for both unsupervised and supervised learning tasks , and can be generalised to incorporate any type of neural networks . section : Neural Variational Inference Framework Commonly used algorithms for unsupervised feature learning include Restricted Boltzmann Machines ( RBM ) , auto - encoders , sparse coding and hybrids such as PSD . Many other methods such as ICA , Reconstruction ICA ( RICA ) , Sparse Filtering and methods related to vector quantization such as Orthogonal Matching Pursuit ( OMP - k ) have also been used in the literature to extract unsupervised feature representations . These algorithms could be divided into two categories : explicitly modeling or not the input distribution . subsection : Variational Auto - Encoders Numerous techniques exist for unsupervised learning in deep networks , e.g. sparse auto - encoders and sparse coding , denoising auto - encoders , deconvolutional networks , restricted Boltzmann machines , deep Boltzmann machines , generative adversarial networks and variational auto - encoders . In this paper we focus on the class of models in the variational auto - encoding framework . Recent successes in word embedding and document embedding have motivated researchers to explore similar representations for networks and to use such representations for tasks such as edge prediction , node label prediction , and community detection . Such network embedding methods are largely focused on finding distributed representations for unsigned networks and are unable to discover embeddings that respect polarities inherent in edges . Hierarchical ARTM model allows us to obtain relations between texts in several abstraction levels which we used as vector representation . To avoid misspelling sensitivity , our model includes document embedding . In the experimental part , we show that our model can improve results of topic modeling on social media datasets . Recently BERT has been adopted in state-of-the-art text summarization models for document encoding . However , such BERT-based extractive models use the sentence as the minimal selection unit , which often results in redundant or uninformative phrases in the generated summaries . Does the TEI Have a Word to Say ? . In this chapter , the authors examine the issue of digital formats for document encoding , archiving and publishing , through the specific example of “ born-digital ” scholarly journal articles . This small area of electronic publishing represents a microcosm of the state of the art , and provides a good basis for this discussion . However , such corpus is not always available , especially for specialized heavy industries such as oil , mining , or steel . To address the problem , this paper proposes a semi-supervised learning framework to create document corpus and embedding starting from an industry taxonomy , along with a very limited set of relevant positive and negative documents . Our solution organizes candidate documents into a graph and adopts different explore and exploit strategies to iteratively create the corpus and its embedding . In this paper , we investigate Arabic document classification using Word and document Embeddings as representational basis rather than relying on text preprocessing and bag-of-words representation . We demonstrate that document Embeddings outperform text preprocessing techniques either by learning them using Doc2Vec or averaging word vectors using a simple method for document Embedding construction . Moreover , the results show that the classification accuracy is less sensitive to word and document vectors learning parameters . Using our machine learning approach , we are able to train a logistic regression model to predict adolescents ’ engagement in substance abuse during distinct life phases with accuracy ranging from 76.5 % to 88.1 % . We show the e ↵ ects of grade level and text message aggregation strategy on the e cacy of document embedding generation with doc2vec . Additional examination of the vectorizations for specific terms extracted from the text message data adds quantitative depth to this analysis . Using a hypermedia system for systems engineering.- Document linking and embedding : A new hypermedia structuring paradigm.- Hooking up 3-space : Three-dimensional models as fully-fledged hypermedia documents.- Distributed multimedia QoS parameters from presentation modelling by coloured petri nets.- Image processing in multimedia.- Designing on-line animated help for multimedia applications.- The development of a virtual world for learning newtonian mechanics.- Other faces of virtual reality.- Techniques for reducing virtual reality latency with architectural support and consideration of human factors.- Modular composing high-performance real-time rendering software.- Fusion of visual tools in virtual spaces.- From individual work and desktop-based collaboration to ubiquitous meeting environments.- Multimedia conferencing architecture and user interfaces for collaborative environment.- Multimedia communications and groupware.- Conceptual support by the new media for co-operative learning in the next century.- Hypermedia environments for learning to program.- Educational multimedia : How to allow for cultural factors.- HELENA : A shell for creating educational hypermedia applications.- Multimedia lecture room : A new tool for education.- A personal information support system for performing creative tasks.- A pattern of Islands : Exploring public information space in a private vehicle.- The concept of a " personal book".- Adaptive hypermedia : In this paper , we aim to address the above issue and propose a new model which successfully integrates a word embedding model , word2vec , into an NMF framework so as to leverage the semantic relationships between words . Empirical results , on several real-world datasets , demonstrate the benefits of our model in terms of text document clustering as well as document/word embedding . The model discussed in the international standardization documents are presented and interpreted . It is shown how editing , formatting and dynamic embedding of documents in the office environment , or in office procedures , can be regarded as abstract processes working on the basis of this model . This demonstrates the usefulness of the architecture model for future implementations of office sys tems . In addition , most existing document embedding algorithms are unsupervised and their learned representations may not be optimal for classification when labeling information is available . In this paper , we study the problem of linked document embedding for classification and propose a linked document embedding framework LDE , which combines link and label information with content information to learn document representations for classification . Experimental results on real-world datasets demonstrate the effectiveness of the proposed framework . - gram vector ( bow3 ) as input to the computation unit ; in particular , we generated bow3 vectors by multiplying the NB - weights with binary vectors , motivated by the good performance of NB - LM . This third layer is a bow - convolution layer with one region of variable size that takes one - hot vectors with - gram vocabulary as input to learn document embedding . The seq2 - bow - CNN for Elec in the table is the same except that the regions sizes of seq - convolution layers are 3 and 4 . One difference is that the attention weights are computed with a bilinear term instead of simple dot - product , that is . The document embedding is computed using a weighted sum as in the Attentive Reader , . In the end , where is a new embedding function . One could also try other extensions to capture the connection between target and context words . For example , we also tried an attention - based LSTM model , which is inspired by the recent success of attention - based neural network in machine translation and document encoding . We implement the soft - attention mechanism to enhance TD - LSTM .
We introduce a novel method for representation learning that uses an artificial supervision signal based on counting visual primitives . This supervision signal is obtained from an equivariance relation , which does not require any manual annotation .
However , it is hard to know which method is optimal for a given dataset or which hyperparameters to use for a chosen method . To this end , we provide a framework to characterize the loss of a causal inference method against the true ACE , by framing causal inference as a representation learning problem . We show that many popular methods , including back-door methods can be considered as weighting or representation learning algorithms , and provide general error bounds for their causal estimates .
Motivated by theories of language and communication that explain why communities with large numbers of speakers have , on average , simpler languages with more regularity , we cast the representation learning problem in terms of learning to communicate . Our starting point sees the traditional autoencoder setup as a single encoder with a fixed decoder partner that must learn to communicate .
Nonetheless , gold POS tags still yield better parsing results as compared to the baseline / AT POS taggers , supporting the claim that POS tagging needs further improvement for downstream tasks . subsection : Effects on Representation Learning English ( WSJ )
GANs provide an attractive alternative to maximum likelihood techniques . One can additionally argue that their learning process and the lack of a heuristic cost function ( such as pixel - wise independent mean - square error ) are attractive to representation learning . GANs have been known to be unstable to train , often resulting in generators that produce nonsensical outputs .
Similar to SDNE , our models rely on the autoencoder to learn non - linear node embeddings from local graph neighborhoods . However , our models have several important distinctions : 1 ) we leverage extensive parameter sharing between the encoder and decoder parts to enhance representation learning ; 2 ) our LoNGAE model can optionally concatenate side node features to the adjacency matrix for improved link prediction performance ; and 3 ) the LoNGAE model can be trained end - to - end in a single stage for multi - task learning of link prediction and semi - supervised node classification .
Sentence embedding is an effective feature representation for most deep learning-based NLP tasks . One prevailing line of methods is using recursive latent tree-structured networks to embed sentences with task-specific structures .
To address these challenges , we proposed DeepEnroll , a cross-modal inference learning model to jointly encode enrollment criteria ( text ) and patients records ( tabular data ) into a shared latent space for matching inference . DeepEnroll applies a pre-trained Bidirectional Encoder Representations from Transformers(BERT ) model to encode clinical trial information into sentence embedding . And uses a hierarchical embedding model to represent patient longitudinal EHR .
This paper discusses two patterns related to sentential embedding . First , it deals with the effect of verb movement and adjunction on the optionality of complementizers ; in this context it offers an explanation of Sten Vikner 's generalisation that optional complementizers are only found in languages without obligatory verb movement .
The comparison of verbs that can take CP complements with verbs that can take NP complements , when these verbs appeared with prepositional phrase complements , revealed activations in bilateral anterior MTG and precuneus , showing that embedding also affects processing at the lexical level . The results thus show that sentential embedding , even without syntactic movement , loads onto several cortical resources including the left IFG , indicating that the generation of syntactic layers is cognitively costly , and that embedding affects processing at both the sentence and the lexical levels .
This paper proposes how to mine concerns and reviews that are relevant to TV dramas from the results of collecting Web pages on those TV dramas . The proposed framework consists of the natural language processing based techniques of analyzing sentence embeddings of those collected Web pages . More specifically , we apply the technique of BERT to the task of judging whether the theme of a collected Web page is relevant to the given drama .
Such models provide low-dimensional vectorized representations of text which are informed by large corpora but can be applied to small-data NLP tasks . Example architectures include , for static word embeddings , word2vec/skip-grams ( Mikolov et al. 2013 ) , GloVe ( Pennington et al. 2014 ) and fasttext ( Bojanowski et al. 2017 ) , for sentence and paragraph embeddings , doc2vec ( Le & Mikolov 2014 ) and sent2vec ( Pagliardini et al. 2018 ) , and for context-sensitive word and sequence embeddings , ULM-Fit ( Howard & Ruder 2018 ) and BERT ( Devlin et al. 2018 ) , the latter adapting the transformer architecture of Vaswani et al. ( 2017 ) . These models have found use within the biomedical domain , particularly for processing scientific literature , where the NCBI ’s PubMed and MedLine databases provide a large , freely available data source for model training .
Sentiment embedding based on the word embedding can capture both semantics and the emotion information . We have introduced two kinds of improving embedding methods ( MEC and Emo2Vec ) for the sentiment words embedding . For emotion structure based on the psychology of emotion , knowledge graph can represent the cognitive relations between different emotion types .
When an attention mechanism is used to compute a representation of a single sequence , it is commonly referred to as self - attention or intra - attention . Together with Recurrent Neural Networks ( RNNs ) or convolutions , self - attention has proven to be useful for tasks such as machine reading [ reference ] and learning sentence representations [ reference ] . However , [ reference ] showed that not only self - attention can improve a method based on RNNs or convolutions , but also that it is sufficient for constructing a powerful model obtaining state - of - the - art performance on the machine translation task . Inspired by this recent work , we introduce an attention - based architecture to perform node classification of graph - structured data .
Many - task systems are less commonly developed . Collobert2008AUA propose a many - task system sharing word embeddings between the tasks , hashimoto2016joint train a many - task model where the tasks are arranged hierarchically according to their linguistic level , and subramanian2018learning train a shared - encoder many - task model for the purpose of learning better sentence representations for use in downstream tasks , not for improving results on the original tasks . section : Conclusion
Then a neural network classifier is applied to decide relationship between the two sentences . Different neural networks have been utilized for sentence encoding , such as LSTM DBLP : conf / emnlp / BowmanAPM15 , GRU DBLP : journals / corr / VendrovKFU15 , CNN DBLP : conf / acl / MouMLX0YJ16 , BiLSTM and its variants DBLP : journals / corr /
Finally , we show that it is possible to exploit the strengths of this model without the need for an external parser by integrating a fast parser into the model ( as in the full SPINN ) , and that the lack of external parse information yields little loss in accuracy . Because this paper aims to introduce a general purpose model for sentence encoding , we do not pursue the use of soft attention , despite its demonstrated effectiveness on the SNLI task . However , we expect that it should be possible to productively combine our model with soft attention to reach state - of - the - art performance .
To focus specifically on the strengths of these models at producing informative sentence representations , we use sentence embedding as an intermediate step in the NLI classification task : each model must produce a vector representation of each of the two sentences without using any context from the other sentence , and the two resulting vectors are then passed to a neural network classifier which predicts the label for the pair . This choice allows us to focus on existing models for sentence embedding , and it allows us to evaluate the ability of those models to learn useful representations of meaning ( which may be independently useful for subsequent tasks ) , at the cost of excluding from consideration possible strong neural models for NLI that directly compare the two inputs at the word or phrase level . label= [ text width=40 mm , align = center ]
Distributed representations of phrases and sentences are also the focus of Socher et al . . Their methods typically require parsing and is shown to work for sentence - level representations . And it is not obvious how to extend their methods beyond single sentences .
It captures natural language inference , also known in previous incarnations as Recognizing Textual Entailment ( RTE ) , and constitutes one of the largest high - quality labeled resources explicitly constructed in order to require understanding sentence semantics . We hypothesize that the semantic nature of NLI makes it a good candidate for learning universal sentence embeddings in a supervised way . That is , we aim to demonstrate that sentence encoders trained on natural language inference are able to learn sentence representations that capture universally useful features .
Recently , a number of competitive methods have tackled unsupervised representation learning by maximising the mutual information between the representations produced from augmentations . The resulting representations are then invariant to stochastic augmentation strategies , and can be used for downstream tasks such as clustering or classification .
Hereby , we differentiate between approaches based on Convolutional Neural Networks ( CNNs ) and Recurrent Neural Networks ( RNNs ) . We then introduce and evaluate three state-of-the-art deep learning systems for unsupervised representation learning from raw audio : ( 1 ) pre-trained image classification CNNs , ( 2 ) a deep convolutional generative adversarial network ( DCGAN ) , and ( 3 ) a recurrent sequence-to-sequence autoencoder ( S2SAE ) . For each of these algorithms , the representations are obtained from the spectrograms of the input audio data .
Our method , which we call Deep InfoMax ( DIM ) , outperforms a number of popular unsupervised learning methods and competes with fully-supervised learning on several classification tasks . DIM opens new avenues for unsupervised learning of representations and is an important step towards flexible formulations of representation-learning objectives for specific end-goals .
We do so through the introduction of a differentiable approximation bridge ( DAB ) neural network which provides smooth approximations to the gradient of the non-differentiable function . We present strong empirical results ( performing over 600 experiments ) in three different domains : unsupervised ( image ) representation learning , image classification , and sequence sorting to demonstrate that our proposed method improves state of the art performance . We demonstrate that utilizing non-differentiable functions in unsupervised ( image ) representation learning improves reconstruction quality and posterior linear separability by 10 % .
We do so by training with a differentiable approximation bridge ( DAB ) neural network which approximates the non-differentiable forward function and provides gradient updates during backpropagation . We present strong empirical results ( performing over 600 experiments ) in four different domains : unsupervised ( image ) representation learning , variational ( image ) density estimation , image classification , and sequence sorting to demonstrate that our proposed method improves state of the art performance . We demonstrate that training with DAB aided discrete non-differentiable functions improves image reconstruction quality and posterior linear separability by 10 % against the Gumbel-Softmax relaxed estimator [ 37 , 26 ] as well as providing a 9 % improvement in the test variational lower bound in comparison to the state of the art RELAX [ 16 ] discrete estimator .
We present Momentum Contrast ( MoCo ) for unsupervised visual representation learning . From a perspective on contrastive learning as dictionary look-up , we build a dynamic dictionary with a queue and a moving-averaged encoder .
We propose two flavors of this approach , whereby the positive samples for those triplets whose anchors are unlabeled are obtained either by applying a transformation to the anchor , or by selecting the nearest sample in the training set . We compare our approach to supervised and unsupervised representation learning as well as the ratio between the amount of labeled and unlabeled data . We evaluate all the above approaches on an audio tagging task using the DCASE 2018 Task 4 dataset , and we show the impact of this ratio on the tagging performance .
We propose ProtoNCE loss , a generalized version of the InfoNCE loss for contrastive learning by encouraging representations to be closer to their assigned prototypes . PCL achieves state-of-the-art results on multiple unsupervised representation learning benchmarks , with > 10 % accuracy improvement in low-resource transfer tasks .
We build a joint multimodal model of text and images for automatically assigning illustrative images to journalistic articles . We approach the task as an unsupervised representation learning problem of finding a common representation that abstracts from individual modalities , inspired by multimodal Deep Boltzmann Machine of Srivastava and Salakhutdinov . We use state-of-theart image content classification features obtained from the Convolutional Neural Network of Krizhevsky et al. as input “ images ” and entire documents instead of keywords as input texts .
It rather relies on an automatically generated surrogate task . The task is created by taking the idea of data augmentation – which is commonly used in supervised learning – to the extreme . Starting with trivial surrogate classes consisting of one random image patch each , we augment the data by applying a random set of transformations to each patch .
subsection : Feature Visualization The visualization experiments ’ aim is to qualitatively investigate whether popular models such as the restricted Boltzman machine ( RBM ) can be combined with Patchy - san for unsupervised feature learning . For every input graph , we have generated receptive fields for all nodes and used these as input to an RBM .
One of the early deep learning models which has been used for image denoising is the Stacked Denoising Auto - encoders ( SdA ) . It is an extension of the stacked auto - encoder and was originally designed for unsupervised feature learning . Denoising auto - encoders can be stacked to form a deep network by feeding output of the previous layer to the current layer as input .
Graph Kernels [ reference ] have been proposed as a way to use relational data as part of the classification process , but are quite slow unless approximated [ reference ] . Our approach is complementary ; instead of encoding the structure as part of a kernel function , we learn a representation which allows them to be used directly as features for any classification method . section : Unsupervised Feature Learning Distributed representations have been proposed to model structural relationship between concepts
Thus , to be useful , an unsupervised learning algorithm must in effect correctly guess the likely set of downstream classification tasks without being directly exposed to them . A significant fraction of unsupervised learning research is driven by generative modelling . It is motivated by the belief that the ability to synthesize , or " create " the observed data entails some form of understanding , and it is hoped that a good generative model will automatically learn a disentangled representation , even though it is easy to construct perfect generative models with arbitrarily bad representations .
section : Related Work There exists a large body of work on unsupervised representation learning . Early methods were based on stacked ( often denoising ) autoencoders or restricted Boltzmann machines [ reference ][ reference ][ reference ][ reference ] .
document : Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks In recent years , supervised learning with convolutional networks ( CNNs ) has seen huge adoption in computer vision applications .
Unsupervised representation learning is a fairly well studied problem in general computer vision research , as well as in the context of images . A classic approach to unsupervised representation learning is to do clustering on the data ( for example using K - means ) , and leverage the clusters for improved classification scores . In the context of images , one can do hierarchical clustering of image patches coates2012learning to learn powerful image representations .
This suggests that adversarial examples expose fundamental blind spots in our training algorithms . The cause of these adversarial examples was a mystery , and speculative explanations have suggested it is due to extreme nonlinearity of deep neural networks , perhaps combined with insufficient model averaging and insufficient regularization of the purely supervised learning problem . We show that these speculative hypotheses are unnecessary .
section : Supervised training data for reading comprehension The reading comprehension task naturally lends itself to a formulation as a supervised learning problem . Specifically we seek to estimate the conditional probability , where is a context document , a query relating to that document , and the answer to that query .
DistBelief has been used to achieve state - of - the - art results in several domains and has been shown to be much faster than single GPU training . Existing work on distributed deep learning has focused exclusively on supervised and unsupervised learning . In this paper we develop a new architecture for the reinforcement learning paradigm .
These models are simple , expressive and can be trained efficiently with the highly scalable stochastic gradient back - propagation . Our neural variational framework is suitable for both unsupervised and supervised learning tasks , and can be generalised to incorporate any type of neural networks . section : Neural Variational Inference Framework
Commonly used algorithms for unsupervised feature learning include Restricted Boltzmann Machines ( RBM ) , auto - encoders , sparse coding and hybrids such as PSD . Many other methods such as ICA , Reconstruction ICA ( RICA ) , Sparse Filtering and methods related to vector quantization such as Orthogonal Matching Pursuit ( OMP - k ) have also been used in the literature to extract unsupervised feature representations . These algorithms could be divided into two categories : explicitly modeling or not the input distribution .
subsection : Variational Auto - Encoders Numerous techniques exist for unsupervised learning in deep networks , e.g. sparse auto - encoders and sparse coding , denoising auto - encoders , deconvolutional networks , restricted Boltzmann machines , deep Boltzmann machines , generative adversarial networks and variational auto - encoders . In this paper we focus on the class of models in the variational auto - encoding framework .
Recent successes in word embedding and document embedding have motivated researchers to explore similar representations for networks and to use such representations for tasks such as edge prediction , node label prediction , and community detection . Such network embedding methods are largely focused on finding distributed representations for unsigned networks and are unable to discover embeddings that respect polarities inherent in edges .
Hierarchical ARTM model allows us to obtain relations between texts in several abstraction levels which we used as vector representation . To avoid misspelling sensitivity , our model includes document embedding . In the experimental part , we show that our model can improve results of topic modeling on social media datasets .
Recently BERT has been adopted in state-of-the-art text summarization models for document encoding . However , such BERT-based extractive models use the sentence as the minimal selection unit , which often results in redundant or uninformative phrases in the generated summaries .
Does the TEI Have a Word to Say ? . In this chapter , the authors examine the issue of digital formats for document encoding , archiving and publishing , through the specific example of “ born-digital ” scholarly journal articles . This small area of electronic publishing represents a microcosm of the state of the art , and provides a good basis for this discussion .
However , such corpus is not always available , especially for specialized heavy industries such as oil , mining , or steel . To address the problem , this paper proposes a semi-supervised learning framework to create document corpus and embedding starting from an industry taxonomy , along with a very limited set of relevant positive and negative documents . Our solution organizes candidate documents into a graph and adopts different explore and exploit strategies to iteratively create the corpus and its embedding .
In this paper , we investigate Arabic document classification using Word and document Embeddings as representational basis rather than relying on text preprocessing and bag-of-words representation . We demonstrate that document Embeddings outperform text preprocessing techniques either by learning them using Doc2Vec or averaging word vectors using a simple method for document Embedding construction . Moreover , the results show that the classification accuracy is less sensitive to word and document vectors learning parameters .
Using our machine learning approach , we are able to train a logistic regression model to predict adolescents ’ engagement in substance abuse during distinct life phases with accuracy ranging from 76.5 % to 88.1 % . We show the e ↵ ects of grade level and text message aggregation strategy on the e cacy of document embedding generation with doc2vec . Additional examination of the vectorizations for specific terms extracted from the text message data adds quantitative depth to this analysis .
Using a hypermedia system for systems engineering.- Document linking and embedding : A new hypermedia structuring paradigm.- Hooking up 3-space : Three-dimensional models as fully-fledged hypermedia documents.- Distributed multimedia QoS parameters from presentation modelling by coloured petri nets.- Image processing in multimedia.- Designing on-line animated help for multimedia applications.- The development of a virtual world for learning newtonian mechanics.- Other faces of virtual reality.- Techniques for reducing virtual reality latency with architectural support and consideration of human factors.- Modular composing high-performance real-time rendering software.- Fusion of visual tools in virtual spaces.- From individual work and desktop-based collaboration to ubiquitous meeting environments.- Multimedia conferencing architecture and user interfaces for collaborative environment.- Multimedia communications and groupware.- Conceptual support by the new media for co-operative learning in the next century.- Hypermedia environments for learning to program.- Educational multimedia : How to allow for cultural factors.- HELENA : A shell for creating educational hypermedia applications.- Multimedia lecture room : A new tool for education.- A personal information support system for performing creative tasks.- A pattern of Islands : Exploring public information space in a private vehicle.- The concept of a " personal book".- Adaptive hypermedia :
In this paper , we aim to address the above issue and propose a new model which successfully integrates a word embedding model , word2vec , into an NMF framework so as to leverage the semantic relationships between words . Empirical results , on several real-world datasets , demonstrate the benefits of our model in terms of text document clustering as well as document/word embedding .
The model discussed in the international standardization documents are presented and interpreted . It is shown how editing , formatting and dynamic embedding of documents in the office environment , or in office procedures , can be regarded as abstract processes working on the basis of this model . This demonstrates the usefulness of the architecture model for future implementations of office sys tems .
In addition , most existing document embedding algorithms are unsupervised and their learned representations may not be optimal for classification when labeling information is available . In this paper , we study the problem of linked document embedding for classification and propose a linked document embedding framework LDE , which combines link and label information with content information to learn document representations for classification . Experimental results on real-world datasets demonstrate the effectiveness of the proposed framework .
- gram vector ( bow3 ) as input to the computation unit ; in particular , we generated bow3 vectors by multiplying the NB - weights with binary vectors , motivated by the good performance of NB - LM . This third layer is a bow - convolution layer with one region of variable size that takes one - hot vectors with - gram vocabulary as input to learn document embedding . The seq2 - bow - CNN for Elec in the table is the same except that the regions sizes of seq - convolution layers are 3 and 4 .
One difference is that the attention weights are computed with a bilinear term instead of simple dot - product , that is . The document embedding is computed using a weighted sum as in the Attentive Reader , . In the end , where is a new embedding function .
One could also try other extensions to capture the connection between target and context words . For example , we also tried an attention - based LSTM model , which is inspired by the recent success of attention - based neural network in machine translation and document encoding . We implement the soft - attention mechanism to enhance TD - LSTM .
Representation Learning by Learning to Count Quantifying Error in the Presence of Confounders for Causal Inference Shaping representations through communication: community size effect in artificial learning systems Robust Multilingual Part-of-Speech Tagging via Adversarial Training Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks Learning to Make Predictions on Graphs with Autoencoders Learning to Embed Sentences Using Attentive Recursive Trees DeepEnroll: Patient-Trial Matching with Deep Embedding and Entailment Prediction Selection and Head Chains An fMRI study of syntactic layers: Sentential and lexical aspects of embedding BERT based Web Mining of Concerns and Reviews for TV Drama Audience Representing document-level semantics of biomedical literature using pre-trained embedding models: Novel assessments Knowledge Representation for Emotion Intelligence Graph Attention Networks Semi-Supervised Sequence Modeling with Cross-View Training Neural Natural Language Inference Models Enhanced with External Knowledge A Fast Unified Model for Parsing and Sentence Understanding A large annotated corpus for learning natural language inference Distributed Representations of Sentences and Documents Supervised Learning of Universal Sentence Representations from Natural Language Inference Data DHOG: Deep Hierarchical Object Grouping Deep Unsupervised Representation Learning for Audio-Based Medical Applications Learning deep representations by mutual information estimation and maximization Differentiable Approximation Bridges For Training Networks Containing Non-Differentiable Functions Improving Discrete Latent Representations With Differentiable Approximation Bridges Momentum Contrast for Unsupervised Visual Representation Learning Semi-supervised Triplet Loss Based Learning of Ambient Audio Embeddings Prototypical Contrastive Learning of Unsupervised Representations Matching Images to Texts Discriminative Unsupervised Feature Learning with Convolutional Neural Networks Learning Convolutional Neural Networks for Graphs Image Restoration Using Convolutional Auto-encoders with Symmetric Skip Connections DeepWalk: online learning of social representations InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks Explaining and Harnessing Adversarial Examples Teaching Machines to Read and Comprehend Massively Parallel Methods for Deep Reinforcement Learning Neural Variational Inference for Text Processing No more meta-parameter tuning in unsupervised sparse feature learning Towards Conceptual Compression SIGNet: Scalable Embeddings for Signed Networks Combined document embedding and hierarchical topic model for social media texts analysis Discourse-Aware Neural Extractive Model for Text Summarization Encoding Models for Scholarly Literature: Does the TEI Have a Word to Say? Industry Specific Word Embedding and its Application in Log Classification Arabic Text Classification Based on Word and Document Embeddings A Machine Learning Approach to Predicting Alcohol Consumption in Adolescents From Historical Text Messaging Data Multimedia, Hypermedia, and Virtual Reality Models, Systems, and Applications Non-negative Matrix Factorization Meets Word Embedding International standard for an office document architecture model Linked Document Embedding for Classification Effective Use of Word Order for Text Categorization with Convolutional Neural Networks Text Understanding with the Attention Sum Reader Network Effective LSTMs for Target-Dependent Sentiment Classification
6a25e474075e26ddd18a24cd80deeb1d2ab33b42 6f42556a6ef5928db35c7588433b9eb601172bb3 ef27f2a5ac58247d2e374276042a42863c8810c0 cdb1418ff8f5ff7a132d77a7be69090caa0bd7bb cc893c34ac761f70f5eeea3ea6542574dad3c8dc 968da948bd2dafe9853b4ecac4f279467f941642 2747d20d9ef671acf9f41c8a46b19a2f52fbd5f8 2c807b1c6f61a4625c145861a36860fc2b7fe563 cac598f2c99c04c475ce41ff380afbddcd32480d dff28ad196f0b7e2457c24a81f356badafee4b11 0c3a3f680b4d273556e4fba5c8280e09d8fdd056 35d0766ea4d9760ef9ce900d4f7db5905171658e eae7d5b15423a148e6bb32d24bbabedfacd0e2df 7a405c5c3658037798782fb8f92296e57ce99b25 8860e434d9bcddb5be07b38be2cca5c66b83f201 ec46830a4b275fd01d4de82bffcabe6da086128f 54a406a8bddf33c828668e26b8d2c9b5c1e1d6ef c5dd36d11da107d9898085e9296fb3b66597b7ab 0a55819e380418a22270ea8cf55642e798e344da 21bcb27995ae1007f4dabe5973c5fa6df7706f3e 785c133994d4d6c915bead5bfef108cab813f772 c8ff62ce1e64a0be55515bb5fa23193d22f7a39f 967bdac1931ab114815f53901681c1a980eee9d8 5e2ad4c165fdd9c2ba3823ea05078eba3142f80a 1d0aa95bfe0008fd3ffe76ac4f0c707fb0d37f4d 945bc1bbfa2bc8b4296c56bd040b4a03222b03b3 166691c4ccf18ab9166100f2bf4a91e1490df3b2 e6a02689604ff8606bc7ee82689b4e6fac16b999 bcbba7e62b4557a1e1a77da8e8362fbb8ed5a44f 5f4abbaa11dbd49ef3a0f5b17b5c80c3769e5b1d
["Computer Science"] ["Mathematics","Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Medicine","Mathematics","Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science","Mathematics"] ["Computer Science"] ["Computer Science","Mathematics"] ["Mathematics","Computer Science"] ["Mathematics","Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science","Mathematics"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"]
2017 2019 2019 2018 2016 2018 2018 2020 2003 2009 2019 2019 2019 2018 2018 2018 2016 2015 2014 2017 2020 2020 2019 2019 2019 2019 2019 2020 2014 2014 2016 2016 2014 2016 2016 2015 2015 2015 2016 2014 2016 2018 2018 2019 2011 2019 2016 2019 1996 2017 1985 2016 2015 2016 2016
ICCV NAACL-HLT ICLR 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA) WI (Companion) KONVENS ICDE ICLR EMNLP ACL ACL EMNLP ICML EMNLP ICLR ICASSP NIPS ICML ArXiv KDD '14 NIPS ICLR ICLR NIPS ArXiv ICML ArXiv NIPS PAKDD (2) CIKM AISI SIGIR CIKM HLT-NAACL ACL COLING
https://api.semanticscholar.org/6a25e474075e26ddd18a24cd80deeb1d2ab33b42 https://api.semanticscholar.org/6f42556a6ef5928db35c7588433b9eb601172bb3 https://api.semanticscholar.org/ef27f2a5ac58247d2e374276042a42863c8810c0 https://www.semanticscholar.org/paper/1751668492bac56f0ae2b6410417515ab3215945 https://www.semanticscholar.org/paper/8388f1be26329fa45e5807e968a641ce170ea078 https://www.semanticscholar.org/paper/2e4c06dd00c4c09ad5ac6be883cc66c19d88ea79 https://api.semanticscholar.org/cdb1418ff8f5ff7a132d77a7be69090caa0bd7bb https://api.semanticscholar.org/cc893c34ac761f70f5eeea3ea6542574dad3c8dc https://api.semanticscholar.org/968da948bd2dafe9853b4ecac4f279467f941642 https://api.semanticscholar.org/2747d20d9ef671acf9f41c8a46b19a2f52fbd5f8 https://api.semanticscholar.org/2c807b1c6f61a4625c145861a36860fc2b7fe563 https://api.semanticscholar.org/cac598f2c99c04c475ce41ff380afbddcd32480d https://api.semanticscholar.org/dff28ad196f0b7e2457c24a81f356badafee4b11 https://www.semanticscholar.org/paper/33998aff64ce51df8dee45989cdca4b6b1329ec4 https://www.semanticscholar.org/paper/0c47cad9729c38d9db1f75491b1ee4bd883a5d4e https://www.semanticscholar.org/paper/3b1d8eb163ffff598c2faa0d9d7cf933857a359f https://www.semanticscholar.org/paper/36c097a225a95735271960e2b63a2cb9e98bff83 https://www.semanticscholar.org/paper/f04df4e20a18358ea2f689b4c129781628ef7fc1 https://www.semanticscholar.org/paper/f527bcfb09f32e6a4a8afc0b37504941c1ba2cee https://www.semanticscholar.org/paper/ee7b883e35d754ae4f71c21bb71f9f03e4ffbb2c https://api.semanticscholar.org/0c3a3f680b4d273556e4fba5c8280e09d8fdd056 https://api.semanticscholar.org/35d0766ea4d9760ef9ce900d4f7db5905171658e https://api.semanticscholar.org/eae7d5b15423a148e6bb32d24bbabedfacd0e2df https://api.semanticscholar.org/7a405c5c3658037798782fb8f92296e57ce99b25 https://api.semanticscholar.org/8860e434d9bcddb5be07b38be2cca5c66b83f201 https://api.semanticscholar.org/ec46830a4b275fd01d4de82bffcabe6da086128f https://api.semanticscholar.org/54a406a8bddf33c828668e26b8d2c9b5c1e1d6ef https://api.semanticscholar.org/c5dd36d11da107d9898085e9296fb3b66597b7ab https://api.semanticscholar.org/0a55819e380418a22270ea8cf55642e798e344da https://www.semanticscholar.org/paper/a6fd96a900d4130940b488863b71fd09ad41ccb9 https://www.semanticscholar.org/paper/7c6de5a9e02a779e24504619050c6118f4eac181 https://www.semanticscholar.org/paper/3c6599052623542f6e21a2cc45a6fde4fb2dc374 https://www.semanticscholar.org/paper/fff114cbba4f3ba900f33da574283e3de7f26c83 https://www.semanticscholar.org/paper/35da0a2001eea88486a5de677ab97868c93d0824 https://www.semanticscholar.org/paper/8388f1be26329fa45e5807e968a641ce170ea078 https://www.semanticscholar.org/paper/bee044c8e8903fb67523c1f8c105ab4718600cdb https://www.semanticscholar.org/paper/d1505c6123c102e53eb19dff312cb25cea840b72 https://www.semanticscholar.org/paper/d5ed07113ddcd038062525a5a54550c012ac9a74 https://www.semanticscholar.org/paper/73e8633886dc380a46fc02f2e1ec5bf68dba0734 https://www.semanticscholar.org/paper/097851f362667a599208f4a06fa289dac2e91fbe https://www.semanticscholar.org/paper/bda89e0d181eda7e49ea831225eda86d075e111c https://api.semanticscholar.org/21bcb27995ae1007f4dabe5973c5fa6df7706f3e https://api.semanticscholar.org/785c133994d4d6c915bead5bfef108cab813f772 https://api.semanticscholar.org/c8ff62ce1e64a0be55515bb5fa23193d22f7a39f https://api.semanticscholar.org/967bdac1931ab114815f53901681c1a980eee9d8 https://api.semanticscholar.org/5e2ad4c165fdd9c2ba3823ea05078eba3142f80a https://api.semanticscholar.org/1d0aa95bfe0008fd3ffe76ac4f0c707fb0d37f4d https://api.semanticscholar.org/945bc1bbfa2bc8b4296c56bd040b4a03222b03b3 https://api.semanticscholar.org/166691c4ccf18ab9166100f2bf4a91e1490df3b2 https://api.semanticscholar.org/e6a02689604ff8606bc7ee82689b4e6fac16b999 https://api.semanticscholar.org/bcbba7e62b4557a1e1a77da8e8362fbb8ed5a44f https://api.semanticscholar.org/5f4abbaa11dbd49ef3a0f5b17b5c80c3769e5b1d https://www.semanticscholar.org/paper/fbf417c83ae5b895fc645346e4efbf3a0aabeac9 https://www.semanticscholar.org/paper/f2e50e2ee4021f199877c8920f1f984481c723aa https://www.semanticscholar.org/paper/0b0dc14b8a8dcccbfc62a38355fff2f6a361e9d2