Semantic dependency parsing, which aims to find rich bi-lexical relationships, allows words to have multiple dependency heads, resulting in graph-structured representations. We propose an approach to semi-supervised learning of semantic dependency parsers based on the CRF autoencoder framework. Our encoder is a discriminative neural semantic dependency parser that predicts the latent parse graph of the input sentence. Our decoder is a generative neural model that reconstructs the input sentence conditioned on the latent parse graph. Our model is arc-factored and therefore parsing and learning are both tractable. Experiments show our model achieves significant and consistent improvement over the supervised baseline. Semantic dependency parsing (SDP) is a task aiming at discovering sentence-internal linguistic information. The focus of SDP is the identification of predicate-argument relationships for all content words inside a sentence (Oepen et al., 2014; . Compared with syntactic dependencies, semantic dependencies are more general, allowing a word to be either unattached or the argument of multiple predicates. The set of semantic dependencies within a sentence form a directed acyclic graph (DAG), distinguishing SDP from syntactic dependency parsing tasks, where dependencies are usually tree-structured. Extraction of such high-level structured semantic information potentially benefits downstream NLP tasks (Reddy et al., 2017; Schuster et al., 2017) . Several supervised SDP models are proposed in the recent years by modifying syntactic dependency parsers. Their parsing mechanisms are either transition-based (Ribeyre et al., 2014; Kanerva et al., 2015; Wang et al., 2018) or graph-based (Martins & Almeida, 2014; Dozat & Manning, 2018; Wang et al., 2019) . One limitation of supervised SDP is that labeled SDP data resources are limited in scale and diversity. Due to the rich relationships in SDP, the annotation of semantic dependency graphs is expensive and difficult, calling for professional linguists to design rules and highly skilled annotators to annotate sentences. This limitation becomes more severe with the rise of deep learning, because neural approaches are more data-hungry and susceptible to over-fitting when lacking training data. To alleviate this limitation, we investigate semi-supervised SDP capable of learning from both labeled and unlabeled data. While a lot of work has been done on supervised SDP, the research of unsupervised and semisupervised SDP is still lacking. Since parsing results of semantic dependencies are DAGs without the tree-shape restriction, most existing successful unsupervised (Klein & Manning, 2004; I. Spitkovsky et al., 2010; Jiang et al., 2016; Cai et al., 2017) and semi-supervised (Koo et al., 2008; Druck et al., 2009; Suzuki et al., 2009; Corro & Titov, 2019) learning models for syntactic dependency parsing cannot be applied to SDP directly. There also exist several unsupervised (Poon & Domingos, 2009; Titov & Klementiev, 2011) and semi-supervised (Das & Smith, 2011; Kočiskỳ et al., 2016; Yin et al., 2018) methods for semantic parsing, but these models are designed for semantic representations different from dependency graphs, making their adaptation to SDP difficult. In this work, we propose an end-to-end neural semi-supervised model leveraging both labeled and unlabeled data to learn a dependency graph parser. Our model employs the framework of Condi-tional Random Field Autoencoder (Ammar et al., 2014) , modeling the conditional reconstruction probability given the input sentence with its dependency graph as the latent variable. Our encoder is the supervised model of Dozat & Manning (2018) , formulating an SDP task as labeling each arc in a directed graph with a simple neural network. Analogous to a CRF model (Sutton et al., 2012) , our encoder is capable of computing the probability of a dependency graph conditioned on the input sentence. The decoder is a generative model based on recurrent neural network language model (Mikolov et al., 2010) , which formulates the probability of generating the input sentence, but we take into account the information given by the dependency parse graphs when generating the input. Our model is arc-factored, i.e., the encoding, decoding and reconstructing probabilities can all be factorized into the product of arc-specific quantities, making both learning and parsing tractable. A unified learning objective is defined that takes advantage of both labeled and unlabeled data. Compared with previous semi-supervised approaches based on Variational Autoencoder (Kingma & Welling, 2013) , our learning process does not involve sampling, promising better stability. We evaluate our model on SemEval 2015 Task 18 Dataset (English) (Oepen et al., 2015) and find that our model consistently outperforms the state-of-the-art supervised baseline. We also conduct detailed analysis showing the benefits of different amounts of unlabeled data. In this work, we proposed a semi-supervised learning model for semantic dependency parsing using CRF Autoencoders. Our model is composed of a discriminative neural encoder producing a dependency graph conditioned on an input sentence, and a generative neural decoder for input reconstruction based on the dependency graph. The model works in an arc-factored fashion, promising end-to-end learning and efficient parsing. We evaluated our model under both full-supervision settings and semi-supervision settings. Our model outperforms the baseline on multiple target representations. By adding unlabeled data, our model exhibits further performance improvements.
We propose an approach to semi-supervised learning of semantic dependency parsers based on the CRF autoencoder framework. This paper focuses on semi-supervised semantic dependency parsing using the CRF-autoencoder to train the model in a semi-supervised style, indicating effectiveness on low resource labeled data tasks.