This paper introduces a new, graphbased view of the data of the FrameNet project, which we hope will make it easier to understand the mixture of semantic and syntactic information contained in FrameNet annotation. We show how English FrameNet and other Frame Semantic resources can be represented as sets of interconnected graphs of frames, frame elements, semantic types, and annotated instances of them in text. We display examples of the new graphical representation based on the annotations, which combine Frame Semantics and Construction Grammar, thus capturing most of the syntax and semantics of each sentence. We consider how graph theory could help researchers to make better use of FrameNet data for tasks such as automatic Frame Semantic role labeling, paraphrasing, and translation. Finally, we describe the development of FrameNet-like lexical resources for other languages in the current Multilingual FrameNet project. which seeks to discover cross-lingual alignments, both in the lexicon (for frames and lexical units within frames) and across parallel or comparable texts. We conclude with an example showing graphically the semantic and syntactic similarities and differences between parallel sentences in English and Japanese. We will release software for displaying such graphs from the current data releases.
Overview In this paper, we provide a new graph-based display of FrameNet annotation, which we hope will make the complex data model of FrameNet more accessible to a variety of users. We begin with a brief introduction to the Frame Semantics and the FrameNet project and their underlying graph structures. Section 3 illustrates how annotation maps words in sentences to nodes in FrameNet, showing the struture of a sentence in the new graph representation. Sect. 4 discusses how the graph representation could help NLP developers, particularly w.r.t. automatic semantic role labeling. In Sect. 5, we introduce the Multilingual FrameNet project, and what comparisons of frame structures across languages might reveal by way of another example sentence in the new format, then discuss our conclusions and acknowledge support for our work. Frame Semantics and English FrameNet The FrameNet Project Baker, 2010, Ruppenhofer et al., 2016] at the International Computer Science Institute (ICSI) is an ongoing project to produce a lexicon of English that is both human-and machine-readable, based on the theory of Frame Semantics developed by Charles Fillmore and colleagues [Fillmore, 1997] and supported by annotating corpus examples of the lexical items. Although FrameNet (FN) is a lexical resource, it is organized not around words, but rather the roughly 1,200 semantic frames [Fillmore, 1976] : characterizations of events, relations, states and entities which are the conceptual basis for understanding the word senses, called lexical units (LUs). Frames are distinguished by the set of roles involved, known as frame elements (FEs). Defining individual lexical units relative to semantic frames provides a crucial level of generalization for their meaning and use. Much of the information in FN is derived from the more than 200,000 manually annotated corpus sentences; annotators not only mark the target word which evokes the frame, but also those phrases which are syntactically related to the target word and express its frame elements. FN covers roughly 13,500 LUs, and provides very rich syntagmatic information about the combinatorial possibilities of each LU. Each frame averages about 10 frame elements, and the same frame can be evoked by words (or multiword expressions) of any part of speech. FrameNet frames are connected by eight types of relations, including full inheritance (IS-A relation) in which all core FEs are inherited, weaker forms of inheritance (called Using and Perspective on), and relations between statives, inchoatives, and causatives. Most frames are linked in a single large lattice (analyzed in Valverde-Albacete [2008] ). The full graph is difficult to render, but can be browsed at https://framenet.icsi.berkeley. edu/fndrupal/FrameGrapher FrameNet also has a small hierarchy of semantic types which can be marked on Frames, FEs and LUs; a portion is shown in Fig. 1 . Many of the semantic types in FrameNet are similar to nodes in widely used ontologies, but they are limited to those which are linguistically important; for example, most agent FEs (not only those called "Agent", but all those descended from the AGENT FE in the high-level frame Intentionally act) have the semantic type SENTIENT (Non-sentient actants receive the FE CAUSE). 1 Some semantic 1 Matching FE semantic types to fillers is complicated by phenomena such as metonymy (The White house announced today . . . ) and personification (She still runs good, but eventually she'll need new tires. ), not fully addressed in FN. types add information which cross-cuts the frame hierarchy; e.g., POSITIVE JUDGEMENT and NEG-ATIVE JUDGEMENT are used to separate those LUs in the frames Judgement, Judgement communication and Judgement direct address that have positive affect from those with negative affect. Frame Semantic and Construction Grammar representation of sentence meaning The development of Frame Semantics has gone hand in hand with the development of Construction Grammar, by Fillmore and a wide range of colleagues (Michaelis [2010] , Feldman et al. [2010] ). FrameNet annotators not only mark which spans of the corpus sentences instantiate which Frame Elements, but also the phrase type (PT) of the constituent that covers that span 2 and the grammatical function (GF, a.k.a. grammatical relation) between that constituent and the target instance of the lexical unit as a coextensive set of spans on three annotation "layers". Additional information is added on other "layers" indicating the presence of copulas and other support verbs, the antecedents of relative clauses, etc. This syntactic information, based on Construction Grammar, can be combined with the FE labels to form a joint syntactico-semantic representation of much of the meaning of a sentence. In graph terms, the annotation process creates a mapping between the string of characters in the sentence and (1) nodes representing frame elements in the frame hierarchy and (2) nodes representing parts of constructions in the Construction Grammar hierarchy. We illustrate this with an example sentence extracted from a TED talk entitled "Do schools kill creativity?" by Ken Robinson 3 : The thing they were good at in school wasn't valued, or was actually stigmatized. The graph representation derived from FrameNet annotation is shown in Fig. 2 . 4 In this figure, the nodes of the graph are syntactico-semantic entities (solid borders) or semantic entities (dotted borders) and the words of the sentence are the terminal nodes of the graph (in boxes). Each edge specifies the relationship between nodes, solid black for syntactico-semantic Though not shown in this graph, each frame instance is also linked to the frame hierarchy graph (Sec. 2). The edges descending from the frames semantically represent the relations described by Frame Elements in the same hierarchy. The dotted lines pointing to dotted nodes are links into the semantic type hierarchy (Sec. 2.1). The syntactic features of the non-terminal nodes are summarized by Phrase Type (PT) labels (S, N, NP, V, VP, PP, etc. with their conventional meanings) and part-ofspeech (not shown). Other features on the edges are syntactico-semantic categories: T (target, the word(s) that evokes the frame), RelC (relative clause), Ant. (antecedent of relative clause), Head (syntactic and semantic head), Sem H (semantic head), and Supp (support, a syntactic head). Applications of FrameNet data as a graph The ability to separate syntactic and semantic dependency is potentially of use in many tasks involving FrameNet data, including automatic semantic role labeling (ASRL), inferencing, language generation, and cross-linguistic comparison. Because of the clear representation of syntactic and semantic dependency in the graph (displayed in Fig. 2 by vertical position, arrow direction, and non-local edges), many tasks should be able to use the graph even without special processing for the subtypes of edges, e.g. for relative clauses as seen under NP[3]. To find out the overall meaning of this sentence, one can start from the "S" node and follow the edges marked "Head" or "Sem H" to the two instances of the Judgement frame. From there, the application can drill further down as needed, into the frame hierarchy, the semantic type hierarchy, or the fillers of the frame roles. One task in particular that could use the full power of such graphs is automatic semantic role labeling (ASRL). The high cost of expert semantic annotation has spurred interest in building ASRL systems. Much of this has been based on the Prop-Bank [Palmer et al., 2005] style of annotation, but work on Frame Semantic role labelers has continued, with increasing success (Das et al. [2014] , Roth and Lapata [2015] Figure 3 : Frame Semantic Annotation of Equivalent Japanese Sentence generally reflect the effort those researchers have made to understand the FrameNet data in depth, including dependencies between semantic roles within a frame, propagation of semantic types across frames, and dependencies between syntax and semantics in a specific sentence. When Frame Element annotation is treated simply as independent tags for machine learning (even if syntactic information is imported from other sources), the learning algorithms are starved of the information needed to make smarter generalizations about the large proportion of the syntactic information about each lexical unit that is predictable from other lexical units in the frame, other related frames, or structures of the language as a whole, such as passivization and relative clause structure. The current distribution format of the FrameNet data does not make this clear. Since FrameNet data is basically discrete and categorial, treating it as an interlocking set of graphs should enable better use of all the information, explicit and implicit, in FrameNet. Multilingual FrameNet The development of the FrameNet resource at ICSI has inspired the creation of a number of Frame Semantics-based projects for other languages: efforts on Spanish, German, Japanese, Chinese, Swedish, Brazilian Portuguese, and French have all received substantial funding, primarily from their national or provincial governments. The basic research question is: to what extent are the semantic frames universal and to what extent are they language-specific? Even if equivalent frames exist in two languages, how much of the frame structure will be preserved in translation? If a different frame is used, is it a near neighbor via frame relations in one or both of the languages? These questions have also been discussed by, e.g. Boas [2009],Čulo [2013] , andČulo and de Melo [2012] . The sentence in Fig. 2 is part of an experiment in annotation of parallel texts; TED talks were chosen because translations are freely available in all of these languages. The TED talk translations are done by volunteers, so they may not be of professional quality, but this is a common situation on the web today, which NLP research has to deal with. In general the TED talk translations tend to be fairly "literal", so we would expect that the frames would be very similar across languages. However, frame differences occur even here. E.g, in the graph of the Japanese translation of this sentence (shown in Fig. 3) , the first conjunct has the Judgement frame like the English, but the second instance of Judgement in English is translated by the frame Labeling in Japanese. Here the agent of the labeling is the school, pre-sumably metonymic for either the faculty, the students, or both. 5 Thus, the graph representation of the FrameNet data helps to make clear which parts of the sentences to compare across languages. We hope that ultimately such comparisons will lead to graph-based MT systems that can transfer meaning at a deeper level. One of the goals of the Multilingual FrameNet project is to quantify the patterns of frame occurrence across varied languages. The new annotation of parallel texts has just begun, so we the number of instances of frames is still small, but we can report some suggestive results based on comparing the annotation of verbs of motion in two texts. One is the TED talk, where we have annotation for English and Brazilian Portuguese; the other is a chapter of the Sherlock Holmes story "The Hound of the Baskervilles", translated by professional translators, where we compare annotation in English and Spanish. (We some annotation previously on these texts in English, Spanish, Japanese and German, but not Portuguese.) Name Lang Same Partial Diff. Tot. TED EN-PT  38  4  22  64  Hound EN-ES  33  3  23  59   Table 1 : Frame similarity and difference across parallel texts Table 1 gives the counts for instances of verbs of motion in two texts, showing cases where the aligned verbs are the same or different across languages. We had hypothesized that the professional, literary translations of the "Hound" text would have more cross-linguistic differences, while that the volunteer translations of the TED talks would be more often frame-preserving. The counts shown here conform to that expectation, but the differences are not conclusive. Conclusion FrameNet data is extremely rich, but not usually presented in a form that is easy for use in NLP. There are clear advantages to viewing the FrameNet annotation data as a graph that separates out entities (nodes) from relations (edges) and clarifies which information is semantic, syntactic, or both. The semantic information can be cleanly integrated with FrameNet's already elaborate graph of frames and semantic types, while generalizations over syntactic information should enable improved use of FrameNet annotation in ASRL training and cross-linguistic comparison. Acknowledgements
1 2 3 4 5 6
Overview Frame Semantics and English FrameNet Frame Semantic and Construction Applications of FrameNet data as a graph Multilingual FrameNet Conclusion
Uses of Graph methods with Frame Semantic Annotation and Parsing
The FrameNet lexical database as a set of graphs FN annotation Sentences Conclusions References
Visualize of complex relations, including cross-lingual relations
Query with graph expressions (e.g. using Neo4j DB)
Express constraints as graph unification ( Construction grammar)
Summarize valences (Kernel Dependency Graphs, cf.
S or VP Judgement
Ext Cognizer T Obj Dep Evaluee Reason
NP admire NP PPing
The FrameNet lexical database as a set of graphs FN annotation Sentences Conclusions References
Visualize of complex relations, including cross-lingual relations
Query with graph expressions (e.g. using Neo4j DB)
Express constraints as graph unification ( Construction grammar)
Summarize valences (Kernel Dependency Graphs, cf.
S or VP Judgement
Ext Cognizer T Obj Dep Evaluee Reason
NP admire NP PPing