Controlling Topic-Focus Articulation in Meaning-to-Text Generation using
  Graph Neural Networks
A bare meaning representation can be expressed in various ways using natural
language, depending on how the information is structured on the surface level.
We are interested in finding ways to control topic-focus articulation when
generating text from meaning. We focus on distinguishing active and passive
voice for sentences with transitive verbs. The idea is to add pragmatic
information such as topic to the meaning representation, thereby forcing either
active or passive voice when given to a natural language generation system. We
use graph neural models because there is no explicit information about word
order in a meaning represented by a graph. We try three different methods for
topic-focus articulation (TFA) employing graph neural models for a
meaning-to-text generation task. We propose a novel encoding strategy about
node aggregation in graph neural models, which instead of traditional encoding
by aggregating adjacent node information, learns node representations by using
depth-first search. The results show our approach can get competitive
performance with state-of-art graph models on general text generation, and lead
to significant improvements on the task of active-passive conversion compared
to traditional adjacency-based aggregation strategies. Different types of TFA
can have a huge impact on the performance of the graph models.
Chunliu Wang, Rik van Noord, Johan Bos
# Controlling Topic-Focus Articulation in Meaning-to-Text Generation

###### Abstract

A bare meaning representation can be expressed in various way using natural language, depending on how the information is structured on the surface level. We are interested in finding ways to control topic-focus articulation when generating text from meaning. We focus on distinguishing active and passive voice for sentences with transitive verbs. The idea is to add pragmatic information such as topic to the meaning representation, thereby forcing either active or passive voice when given to a natural language generation system. We use graph neural models because there is no explicit information about word order in a meaning represented by a graph. We try three different methods for topic-focus articulation employing graph neural models for a meaning-to-text generation task. We propose a novel encoding strategy about node aggregation in graph neural models, which instead of traditional encoding by aggregating adjacent node information, learns node representations by using depth-first search. The results show our approach can get competitive performance with state-of-art graph models on general text generation, and lead to significant improvements on the task of active-passive conversion compared to traditional adjacency-based aggregation strategies. Different types of TFA can have a huge impact on the performance of the graph models.

## 1 Introduction

Topic-Focus Articulation (TFA) refers to the way information is packaged within a sentence, in particular the division of the topic (the given information) and the focus (the new information). There are various linguistic devices that determine how topic and focus can be articulated in a sentence: dislocation, word order, active/passive voice, intonation, particles, and more [1, 1, 2, 1, 14]. TFA plays a crucial role in natural language generation when the input is an abstract representation of meaning and the output is a text. The same information can be realised in various ways, depending on the perspective one takes. Take for example the information provided by the situation where a wolf killed two sheep. We could ask several questions about this situation, viewing it from different points of view:

Q\({}_{1}\): What about the sheep?

A\({}_{1}\): _The two sheep were killed by a wolf._

Q\({}_{2}\): What about the wolf?

A\({}_{2}\): _The wolf killed two sheep._

A flexible natural language generation system, one that takes as input formal meaning representations, and outputs texts, ideally should be able to package the information related to the perspective being taken. In this paper we investigate ways of adding TFA to a formal meaning representation in order to get different texts where information is packaged in different ways. We focus on active-passive alternation in English, as exemplified by the examples above, as in English usually the subject of a sentence is the topic [1]. The challenge is to find a simple, intuitive way to add TFA to an abstract meaning representation that gives the desired effects when given to an meaning-to-text NLG component. For this purpose we employ graphical representations of meaning that abstract away from any surface order of the message they convey the meaning of. The NLG modules in our experiments are implemented by neural models. To the best of our knowledge we are the first that experiment with controlling TFA in the context of text generation from formal meaning representations.

The paper is organised as follows. In Section 2 we introduce the semantic formalism of our choice, Discourse Representation Theory [1], and in particular the TFA-neural
graph representation of meaning that we use in our experiments. Here we also review previous meaning-to-text approaches for Discourse Representation Structures and graph-based methods for graph-to-text generation task. In Section 3 we describe the process of acquiring graph-structured data for graph neural networks (GNNs), and then we focus on presenting the three different TFAs we added in meaning representation. The basic idea is to use the simple markers to mark active-voice data and passive-voice data respectively to help the neural models distinguish different types of input graph. In Section 4 we introduce the implementation settings, and the evaluation metrics. We present a comparison of local encoders and deep encoders based on three types of GNNs training on different graph representation with three types of TFAs.

## 2 Background

### Discourse Representation Structures

Discourse Representation Theory (DRT) is a well-studied semantic formalism covering many linguistic phenomena including scope of negation and quantifiers, interpretation of pronouns, presupposition, temporal and discourse relations (Kamp and Reyle, 1993). The meaning representation proposed by DRT is the Discourse Representation Structure (DRS), a recursive first-order logic representation comprising of discourse referents (the entities introduced in the discourse) and relations between them. We work with a particular variant of DRS, namely the one proposed in the Parallel Meaning Bank (Abzianidze et al., 2017).

There are five types of semantic information that can be found in the PMB-style DRSs: entities, constants, roles, comparison operators, and discourse relations (including negation). The entities are represented by concepts denoted by WordNet synsets (Fellbaum, 1998) (for nouns, verbs, adjectives and adverbs), indicating the lemma, part-of-speech and sense number (e.g., book.n.02 encodes the entity with the second sense of the noun "book"). The constants are used to represent names, numbers, dates, and deixis. The roles (_Theme_, _Agent_, and so on), are represented by the thematic relations proposed in VerbNet (Kipper et al., 2008) and semantically connect two entities. Comparison operators relate and compare entities or constants. Discourse relations convey the rhetorical function between different discourse units.

As Figure 1 shows, there are various ways of representing a DRS. Besides the classic box notation, DRSs can also be displayed in variable-free sequential notation or as directed acyclic graphs, following a recent proposal by Bos (2021). For use in a neural graph model, the DRS data can be converted into a set of triples, using the variable-free sequential box notation. Doing so, the roles, comparison operators and discourse relations are regarded as edge labels, while entities, constants and discourse units ("boxes") form the nodes in the Discourse Representation Graph (DRG).

### Generating Text from DRSs

The purpose of DRS-to-text generation is to produce a text from an input DRS data. Basile (2015) proposed a pipeline consisting of three components: a surface order module, a lexicalization module that served for constructing an alignment between the abstract structure and the text, and a surface real

Figure 1: (a) Box format DRS for the sentence of _Two sheep were not killed by the wolf_, (b) DRS in sequential box notation, (c) corresponding DRG.
ization module to construct the final output. Wang et al. (2021) uses a standard LSTM model on clause format DRSs to generate text and shows that char-level encoder achieve significant performance, and Liu et al. (2021) encode DRS as a tree and uses a sibling treeLSTM model to produce text, which focus on the solution for condition ordering and variable naming in DRS representation.

In this paper, however, as our motivation is to convert freely active and passive voice on DRS data, we consider the graphical representation as input and treat the meaning-to-text generation task as Graph2Seq learning task. A DRG is neutral with respect to word order in the surface realisation of text, and therefore serves our purpose of topic-focus articulation better than a sequential representation of meaning in which order is explicitly encoded.1

Footnote 1: A completely different architecture, based on sequential meaning representations, would also be possible, where TFA would take place by operating on the sequential meaning, before given to a Seq2Seq module.

### Neural Networks for Graph

How to encode the input graph is the key issue for the graph-to-text generation task. A GNN layer computes every node representation by aggregating its neighbor's representation, the design of how to aggregate is what mostly distinguishes the various types of GNNs. Graph Convolutional Networks (GCN) learn representations of nodes by summing over the representations of immediate neighborhood of each node (Kipf and Welling, 2017), which has been applied to various text generation tasks (Damonte and Cohen, 2019; Guo et al., 2019; Song et al., 2020) and has achieved remarkable performance. Some variants use mean or max pooling as aggregation, weighting all neighbors with equal importance, with the same core computation as GCN. Velickovic et al. (2018) consider it is unreasonable to assign equal importance to all adjacent nodes of a node and proposed Graph Attention Networks (GAT), which updates each node representation by incorporating the attention mechanism to calculate the importance of adjacent information. Li et al. (2016) proposed the Gated Graph Neural Networks (GGNN) for the reason that GCN model has difficulty to learn deep layers nodes information, which use a Gated Recurrent Unit (GRU) to facilitate information propagation between local layers. This model is used in AMR-to-text generation tasks (Ribeiro et al., 2019) and syntax-based machine translation (Beck et al., 2018).

All the above methods belong to local graph encoding strategies as they are based on _local node aggregation_. The opposite approach to it is global encoding strategies, which typically based on Transformer and leverage global node aggregation. We propose deep traversal graph encoders to replace the global graph encoders and local graph encoders. Different from the global graph encoder, which compute a node representation based on all nodes in the graph, deep traversal graph encoder focus on capture rich information from the depth-first search of nodes, while avoiding introducing noise from all nodes.

Figure 2: (a) Levi graph for a DRG. (b) Two possible realisations for (active and passive voice). (c) Local graph encoder learning information from adjacent nodes, implicitly capturing 2-hop information by using 2-layers model. (d) Deep traversal encoder capturing non-local nodes information by using depth-first search for a node.
## 3 Method

### Graph Preparation

A DRG is a directed labeled graph defined as \(G=(V,E)\), where \(V\) is a set of nodes and \(E\) is a set of edges. Each edge in \(E\) can be represented as \((v_{i},l,v_{j})\), where \(v_{i}\) and \(v_{e}\) are the indices of incoming nodes and outgoing nodes respectively, and \(l\) is the edge label. We use the extended Levi graph method Beck et al. (2018) that changes edge labels to additional nodes, and the graph becomes a directed unlabeled graph where each labeled edge \(e=(v_{i},l,v_{j})\) is transformed into two unlabeled edges \(e_{1}=(v_{i}\), \(l)\) and \(e_{2}=(l\), \(v_{j})\), as shown in Figure 2. This approach is also convenient for representing named entities with more than one token, representing the direction of edges according to the order between tokens of named entities without adding additional edge labels.

### Adding Topic-Focus Articulation

A DRG can be textually paraphrased in various ways. For instance, for sentences with a transitive verb, active as well as passive voice could be generated if nothing is known about the information structure. Here we introduce three ways of adding information to the graph about the topic of the sentence (and assume that the rest of the meaning is considered focus). The idea is that topic controls whether an English sentence with a transitive verb is generated in active or passive voice. We investigate three different methods of adding TFA to a DRG. Our aim is to explore which way of TFA can simply and effectively distinguish active and passive voice, and whether it is convenient for the model to learn this information. We propose three types of augmenting the DRG with TFA:

1. Concept \(\rightarrow\) TOPIC \(\rightarrow\) Concept (CTC)
2. Box \(\rightarrow\) TOPIC \(\rightarrow\) Concept (BTC)
3. Role \(\rightarrow\) Role (RTR)

CtcHere we specify the subject by giving _TOPIC_ marker as edge label and get a self-loop representation. As shown in Figure 3 (a), we give _TOPIC_ marker as edge label to the concept _wolf.n.01_, and then _wolf_ becomes the subject of text and reference text become active-voice text, vice versa, if _sheep_ is the subject, _TOPIC_ marker is added as edge label to the concept _sheep.n.01_ and the reference text is passive-voice. This method does not depend on other content in the text, but only focuses on which concept is the subject, and when adding this type TAF in graph and convert graph to Levi graph, the directed acyclic graph will become a directed cycle graph.

BtcHere we try to connect more information by giving _TOPIC_ as edge label between discourse unit node and subject concept node. As shown in Figure 3 (b), when we use _sheep_ as the subject of the reference text, we give _TOPIC_ marker as edge label to the concept _sheep.n.01_ and its located discourse unit node _Box_, whereas we add _TOPIC_ marker between the concept _wolf.n.01_ and its discourse node _Box_. This method considers the global discourse

Figure 3: Three types of TFA applied to a DRG for _A wolf killed two sheep_ (in blue) or _The wolf killed two sheep_ (in red).
information, and is more intuitive in terms of graph format.

_Rtr_When we do not know the concept information contained in the data, it is possible to add TFA by using VerbNet role information. As shown in Figure 3 (c), _Agent_ is always the VerbNet role connect concept to the verb node, and the text is active-voice if that concept is subject. _Patient_ is one of types of VerbNet roles that connect verb node and the concept of the action or event being imposed by the agent, there are also other types of VerbNet roles introduced in Section 4.1. This method do not consider any content of original DRSs data, but by adding the edge such as _Agent_\(\rightarrow\)_Patient_ or _Patient_\(\rightarrow\)_Agent_ in the graph to decide active-voice and passive-voice. This representation only based on the transformed extended Levi graph, and essentially changes the flow of information into and out of the graph nodes.

### Graph Neural Networks

Previous research shows it is difficult for local graph encoders to capture non-local nodes information and deeper-layers models perform worse because useful information in earlier layers may get lost [19, 10]. Inspired by the work of Ribeiro et al. (2020), they propose to integrate GAT and Transformer architectures into a unified global-local graph encoder to learn deep layers nodes information and adopt it on knowledge graphs to text generation task. In this paper, we propose a deep traversal encoder that aggregates node representations by using all nodes from each node's depth-first search.

Figure 2 (a) presents a DRG data, the corresponding text is shown in Figure 2 (b). For the node _kill.v.01_, it is difficult to access the information from _intruder.n.01_ 4-hop away, especially when we use two layers (Figure 2 (c)), even if we use four layers, the model is very hard to capture. While when using deep traversal for the node _kill.v.01_, the node _intruder.n.01_ has the same distance as node _intruder.n.01_(Figure 2 (d)), comparing with local graph encoder, deep traversal encoder essentially changes the graph structure, and compared to the global graph encoder, this method preserves more information of related nodes.

#### 3.3.1 Deep Traversal Encoder

Compare with the local graph encoder, a deep traversal encoder updates each node information based on depth-first search, which aggregates all the deep traversal nodes instead of adjacency nodes. In this paper, we compute a layer of the depth-first traversal convolution for a node \(i\in V\) based on GGNN, which shows significant performance on the meaning-to-text generation task in previous research [10].

Gated Graph Neural NetworksThe main difference between GCN and GGNN is anologous to the difference between convolutional and recurrent networks [1]. GGNN can be seen as a multi-layer GCN where layer-wise parameters are tied and gating mechanisms are added. With this, the model can propagate node information between long distance nodes in the graph [19, 1]. In particular, the \(l\)-th layer of a GGNN is calculated as:

\[h_{i}^{(l+1)}=GRU(h_{i}^{(l)},\sum_{j\in deep(N_{l})}\frac{W_{dir(j,i)}^{(l)}h_ {j}^{(l)})}{[deep(N_{i})]}) \tag{1}\]

where \(W_{dir(j,i)}^{(l)}\) represents the weight matrix of the \(l\)-th layer, they are direction-specific parameter, where \(dir(j,i)\in\{default,reverse,self\}\), which refer to the original edges, the reversed edges to the original edges, and the self-loop edges [17]. \(deep(N_{i})\) is the the set of nodes by depth-first search for node \(i\), the core component of the proposed approach, where the standard GNNs use \(N_{i}\), which is the set of immediate neighbors of node \(i\). \(\rho\) is the activation function, we use \(ReLU\) in the experiments. \(h_{j}\) is the embedding representation of node \(j\in V\) at layer \(l\), \(GRU\) is a gated recurrent unit, a combination function.

#### 3.3.2 Decoder

We adopt standard fully batched attention-based LSTM decoder [1], where the attention memory is the concatenation of the attention vectors among all input words. The initial state of the decoder is the representation of the output of the encoder. In order to alleviate the data sparsity, we add a copy mechanism [10, 11] on the top of decoder, which favors generating words such as dates, numbers, and named entities that appear in DRGs.
Experiments

### Implementation Details

DataWe use the data from release 4.0.0 of the Parallel Meaning Bank (Abzianidze et al., 2017), which contains 10,711 gold instances and 127,302 silver instances. Each instance contains a text and the corresponding DRS in various formats. We use the sequential box notation to produce the discourse representation graphs (DRGs). We randomly split the gold data to get 1,000 instances for development and 1,000 instances for test, then we use the remaining gold and silver data as our training data (Table 1).

Table 2 shows the distribution of active and passive voice data as shown in the PMB dataset. For convenience, we use VerbNet roles to distinguish the types of active-voice and passive-voice for sentences with transitive verbs in the PMB data. There are five types of active-voice and passive-voice DRGs representation divided by VerbNet roles. The nodes of the VerbNet roles (ingoing or outgoing) determine whether active or passive voice is used. For example, when the Agent role is an incoming node, this DRG corresponds to passive voice. For the challenge set of active and passive data, we use all the types of passive voice sentences in the gold data, and the same number of active voice sentences data corresponding different to types (after removing all interrogative sentences, that we disregard for our experiments). There are less passive (106) than active sentences (3140) in the PMB data. To get a balanced evaluation set, we draw randomly 106 from the active instances.

SettingAll the models we used implemented based on OpenNMT (Klein et al., 2017). For the vocabulary, we construct vocabularies from all words, the vocabulary sizes as shown in Table 1. For the hyperparameters, they are set based on performance on the development set. We use SGD optimizer with the initial learning rate set to 1 and decay 0.8. In addition, we set the dropout to 0.5 at the decoder layer to avoid overfitting with batch size 32. For the deep traversal encoder, we only use one layer for all the graph models. For the local graph encoder, all the graph models we use two layers, _ReLU_ activation and _tanh_ highway. In order to mitigate the effects of random seeds, we report the average for 3 training runs of each model.

Evaluation MetricsFor automatic evaluation, we use three standard metrics measuring word-overlap between system output and references. They are BLEU (Papineni et al., 2002), and METEOR (Lavie and Agarwal, 2007), which are used as standard in machine translation evaluation and very common in NLG. In order to better evaluate the results of different models, we employ the ROSE manual evaluation metric which proposed by the Wang et al. (2021), because this measure is simple to define and easy to reproduce. That was carried out by creating semantic challenge set and assigning three binary dimensions (either 0 or 1) to each generated text: (1) semantics, (2) grammaticality, and (3) phenomenon. The result is correct only when the ROSE-score is 1, and we use the ratio of the correct number to the total number of data as the final manual evaluation method accuracy.

### Automatic Metrics Results

In order to explore which TFA can better represent the active/passive voice and have a positive impact on the models. We use automatic metric evaluation methods on the normal test data in order to get the performance for general generation task. At the same time, we also evaluate the active and passive dataset for obtaining rough performance for this specific tasks. Table 3 is the results of automatic metric results on the normal test and active-passive dataset training on different types of representations by both local graph encoder and deep traversal encoder (see also Appendix A).

Our results show different types of TFA for the graph-structured data have remarkable different influence for the performance of models on specific tasks (active and passive dataset), but also on general text generation (normal test set). On the local graph encoder, GGNN has the best performance on both dataset when we use the edge of _RTR_ as TFA in the graph. However, when we use deep traversal encoder, _CTC_ become the best TFA, and the normal test performance of the model that using _RTR_ as TFA is degraded compared to the local graph encoder, but for active-passive dataset, the deep traversal encoders training with different TFA get the best results compare with the models based

\begin{table}
\begin{tabular}{l c c c c c} \hline \hline  & \multicolumn{2}{c}{**Document-level**} & \multicolumn{2}{c}{**Word-level**} \\ \hline
**Data** & **Train** & **dev** & **test** & **src** & **tgt** \\ \hline _gold_ & 8,711 & 1,000 & 1,000 & 7,856 & 7,699 \\ _gold + silver_ & 136,013 & 1,000 & 1,000 & 46,620 & 44,849 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Document statistics and vocabulary sizes.
on local graph encoder.

### Manual Metrics Results

For manual evaluation, in addition to evaluating whether the position of the subject and object is replaced, the transition between active and passive must take into account the grammatical inflection of the verb. Different types of grammatical inflection of the verb generated by the models will lead to different types of errors, which will affect the three dimensions of manual metrics evaluation at the same time. In this paper, _phenomena_ metric only focus on active and passive voice of verbs in sentences, regardless of semantics and grammaticality. Due to the limited amount of passive data, we create a challenge set using active and passive data in the gold training data. All the passive data and active data in DRG can be transformed by changing the TFA, based on this, we can get a challenge set that is different from the data in the training set. We report the results of accuracy for the active-voice and passive-voice challenge set in Table 4.

Our results show that for the local graph encoder, the representation by adding _RTR_ can get best performance, especially for the task of converting passive data to active data to generate active-voice text, as its _phenomenon_ score and _ROSE_ are the highest. When we change the active DRG data to passive DRG data to expect the models to generate passive text, if _CTC_ and _BTC_ are used as TFA, it is difficult for the models to generate passive voice. They tend to generate active voice text, and almost impossible to generate ideal text, although both semantics and grammaticality are fine. Compared with local graph encoder, deep traversal encoder can get better performance by using _CTC_ and _BTC_ as TFA in the DRG, especially for the _CTC_, the performance for both converting active to passive task and converting passive to active task better than the best results on local graph encoder.

## 5 Discussion

From the experimental results, for all models based on different TFA and encoders, the difficulty lies in whether the correct passive voice text can be generated. There are significant gaps of scores between _phenomenon_ and the other two dimensions in the task of generate passive voice from Figure 4. The models intend to generate text in training data with correct semantic and grammar but wrong phenomenon, that means the models cannot learn the TFA very well. We further subdivide and analyze the types of errors from the _semantic_ and _grammaticality_ two dimensions (see Appendix B).

When the edge label in a DRG between the subject and the object connected with the verb is different, which in the Levi graph is the type of the intermediate node for a VerbNet role. We find that the local encoder can achieve good results by using _RTR_ as TFA. This method is somewhat similar to the methods of adding sequential information in a graph used by Beck et al. (2018) and Guo et al. (2019) to improve the performance, but it is equiva

\begin{table}
\begin{tabular}{l|l|r|r|r|r} \hline \hline \multicolumn{1}{c}{} & \multicolumn{4}{c}{**gold data**} & \multicolumn{2}{c}{**silver data**} \\ \hline \hline
**Type** & **Example of passive** & **Active** & **Passive** & **Active** & **Passive** \\ \hline Patient \(\rightarrow\) Agent & Bill was killed by an intruder. & 786 & 43 & 5,285 & 381 \\ Theme \(\rightarrow\) Agent & The college was founded by Mr Smith. & 1,913 & 42 & 28,120 & 1,125 \\ Experiencer \(\rightarrow\) Agent & I got stung by a bee. & 41 & 5 & 311 & 40 \\ Result \(\rightarrow\) Agent & That was written by Taro Akagawa. & 211 & 13 & 1,241 & 65 \\ Source \(\rightarrow\) Agent & He was deserted by his friends. & 189 & 3 & 85 & 32 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Distribution of active/passive voice types in gold and silver data in PMB release 4.0.0.

\begin{table}
\begin{tabular}{l r r r} \hline \hline
**Model** & **Normal Test** & **A \& P Test** \\ \hline
**TFA** & **BLEU** & **METEOR** & **BLEU** & **METEOR** \\ \hline \multicolumn{3}{l}{**Local Graph Encoder**} \\ RTR & **55.4** & **44.0** & **74.4** & **54.1** \\ BTC & 53.8 & 43.3 & 66.9 & 50.4 \\ CTC & 55.0 & 43.6 & 71.8 & 52.6 \\ \hline \multicolumn{3}{l}{**Deep Traversal Encoder**} \\ RTR & 54.7 & 43.4 & 76.1 & 54.3 \\ BTC & 54.5 & 43.7 & 73.3 & 53.9 \\ CTC & **55.1** & **43.8** & **75.3** & **55.7** \\ \hline \hline \end{tabular}
\end{table}
Table 3: The normal test data results and the active & passive (A\&P) data results of GGNN models training on DRG representation with different types of TFA.
lent to adding a very small part, so it is not difficult to understand why it achieves better performance.

The deep traversal encoder achieves good results by using the self-loop type TFA (_CTC_). The deep traversal encoder essentially obtains all the outflow and inflow nodes information of a node at one time through depth-first search. When we add the _RTR_ to the graph, it essentially introduces noise to some nodes in the graph that originally have no direct flow of information, such as the node _Agent_ and _sheep.n.01_ in Figure 3 (e).

However, using the _CTC_ in a graph will not have this problem, and through the deep traversal encoder, other nodes in the graph also receive _TOPIC_ node information, which is equivalent to increasing the model's memory of _TOPIC_ node, so that the models can learn the difference of active and passive in graph structure data. Intuitively, adding _TOPIC_ markers between discourse unit nodes and concepts (_BTC_) can get good performance, but in practice it cannot achieve good performance on both local graph encoder and deep traversal encoder. We believe that this has a strong correlation with the discourse unit node _Box_, which is very similar to the artificial global node used by Guo et al. (2019) and Cai and Lam (2020), especially when there is only one discourse in the DRG, this node has less influence on the models, and the added edge label _TOPIC_ is also difficult to be learned by the models. This has yet to be found where a better way to go to further verification.

## 6 Conclusion

In this paper we propose the use of GNNs to control the generation of active and passive voice text from formal meaning representations. We use discourse representation structures to represent meaning of sentences, and present the process of how to convert DRS data into graph-structured data for graph models. On the graph level, we introduce three types of TFA to distinguish active and passive DRS data. On the model level, we propose the deep traversal encoder for capturing more information for this task. We apply one of the most popular graph models, GGNN, to the deep traversal encoder and the local graph model encoder, respectively, to compare the performance of the three TFA. Our experimental results show that using self-loop labeled TFA (CTC) can achieve optimal results on deep traversal encoder, while using edge-oriented TFA (TRT) can achieve optimal results on local graph encoder. Our main contributions can be summarized as follows:

1. To the best of our knowledge, we are the first to propose to control TFA in the context of text generation from formal meaning representations;
2. We present the process of how to convert formal meaning representations into graph-structured data for graph models;
3. We compare the performance of three graph neural networks, which have not been studied so far for meaning-to-text generation;
4. We introduce deep traversal encoder to replace local graph encoder for capturing more information;
5. We compare three types of TFA in the graph notation of active-voice and passive-voice data to control the models to generate active-passive voice documents.

\begin{table}
\begin{tabular}{l|l|c c c c|c c c c|c} \hline \hline  & & \multicolumn{4}{c|}{**Passive \(\rightarrow\) Active**} & \multicolumn{4}{c|}{**Active \(\rightarrow\) Passive**} & **ALL** \\ \hline
**Model** & **TFA** & **Sem.** & **Gram.** & **Phen.** & **ROSE** & **Sem.** & **Gram.** & **Phen.** & **ROSE** & **ROSE** \\ \hline \multirow{3}{*}{**Local Graph Encoder**} & RTR & 64.2 & 92.5 & 80.2 & **59.4** & 64.2 & 80.2 & 50.0 & **34.9** & **47.2** \\  & BTC & 61.3 & 85.8 & 60.4 & 45.3 & 50.0 & 88.7 & 16.0 & 11.3 & 28.3 \\  & CTC & 65.1 & 89.6 & 69.8 & 50.9 & 62.3 & 68.9 & 20.8 & 10.4 & 30.7 \\ \hline \multirow{3}{*}{**Deep Traversal Encoder**} & RTR & 70.8 & 84.9 & 68.9 & 62.3 & 53.8 & 86.8 & 38.7 & 24.5 & 43.4 \\  & BTC & 65.1 & 88.7 & 72.6 & 60.4 & 60.4 & 88.7 & 24.5 & 14.2 & 37.3 \\ \cline{1-1}  & CTC & 73.6 & 93.4 & 87.7 & **67.0** & 61.3 & 86.8 & 56.6 & **38.7** & **52.8** \\ \hline \hline \end{tabular}
\end{table}
Table 4: Manual metric performance of GGNN models on active and passive challenge set, training on different types of graph data with local graph encoder and deep traversal encoder.