The problem of accelerating drug discovery relies heavily on automatic tools to optimize precursor molecules to afford them with better biochemical properties. Our work in this paper substantially extends prior state-of-the-art on graph-to-graph translation methods for molecular optimization. In particular, we realize coherent multi-resolution representations by interweaving the encoding of substructure components with the atom-level encoding of the original molecular graph. Moreover, our graph decoder is fully autoregressive, and interleaves each step of adding a new substructure with the process of resolving its attachment to the emerging molecule. We evaluate our model on multiple molecular optimization tasks and show that our model significantly outperforms previous state-of-the-art baselines. Molecular optimization seeks to modify compounds in order to improve their biochemical properties. This task can be formulated as a graph-to-graph translation problem analogous to machine translation. Given a corpus of molecular pairs {(X, Y )}, where Y is a paraphrase of X with better chemical properties, the model is trained to translate an input molecular graph into its better form. The task is difficult since the space of potential candidates is vast, and molecular properties can be complex functions of structural features. Moreover, graph generation is computationally challenging due to complex dependencies involved in the joint distribution over nodes and edges. Similar to machine translation, success in this task is predicated on the inductive biases built into the encoder-decoder architecture, in particular the process of generating molecular graphs. Prior work (Jin et al., 2019) proposed a junction tree encoder-decoder that utilized valid chemical substructures (e.g., aromatic rings) as building blocks to generate graphs. Each molecule was represented as a junction tree over chemical substructures in addition to the original atom-level graph. While successful, the approach remains limited in several ways. The tree and graph encoding were carried out separately, and decoding proceeded in strictly successive steps: first generating the junction tree for the new molecule, and then attaching its substructures together. This means the predicted attachments do not impact the subsequent substructure choices (see Figure 1a) . Moreover, the attachment prediction process is non-autoregressive, thus it can predict inconsistent substructure attachments across different nodes in the junction tree (see Figure 1b ). We propose a multi-resolution, hierarchically coupled encoder-decoder for graph generation. Our auto-regressive decoder interleaves the prediction of substructure components with their attachments to the molecule being generated. In particular, a target graph is unraveled as a sequence of triplet predictions (where to expand the graph, new substructure type, its attachment). This enables us to model strong dependencies between successive attachments and substructure choices. The encoder is designed to represent molecules at different resolutions in order to match the proposed decoding process. Specifically, the encoding of each molecule proceeds across three levels, with each layer capturing essential information for its corresponding decoding step. The graph convolution of atoms at the lowest level supports the prediction of attachments and the convolution over substructures at the highest level supports the prediction of successive substructures. Compared to prior work, our decoding process is much more efficient because it decomposes each generation step into a hierarchy of smaller steps in order to avoid combinatorial explosion. We also extend the method to handle conditional translation where desired criteria are fed as input to the translation process. This enables our method to handle different combinations of criteria at test time. Since their tree and graph decoders are isolated, the model can generate invalid junction trees which cannot be assembled into any molecule. This problem can be solved when we interleave the tree and graph decoding steps, allowing the predicted attachments to guide the substructure prediction; b) Their non-autoregressive graph decoder often predicts inconsistent local substructure attachments during training. To this end, we propose an autoregressive decoder that interleaves the prediction of substructures with their attachments. We evaluate our new model on multiple molecular optimization tasks. Our baselines include previous state-of-the-art graph generation methods (You et al., 2018a; Liu et al., 2018; Jin et al., 2019) and an atom-based translation model we implemented for a more comprehensive comparison. Our model significantly outperforms these methods in discovering molecules with desired properties, yielding 3.3% and 8.1% improvement on QED and DRD2 optimization tasks. During decoding, our model runs 6.3 times faster than previous substructure-based generation methods. We further conduct ablation studies to validate the advantage of our hierarchical decoding and multi-resolution encoding. Finally, we show that conditional translation can succeed (generalize) even when trained on molecular pairs with only 1.6% of them having desired target property combination. In this paper, we developed a hierarchical graph-to-graph translation model that generates molecular graphs using chemical substructures as building blocks. In contrast to previous work, our model is fully autoregressive and learns coherent multi-resolution representations. The experimental results show that our method outperforms previous models under various settings. A ADDITIONAL FIGURES The message passing network MPN ψ (H, {x u }, {x uv }) over graph H is defined as: Algorithm 3 LSTM MPN with T message passing iterations wu } w∈N (u)\v for all edges (u, v) ∈ H simultaneously. end for Return node representations end function Attention Layer Our attention layer is a bilinear attention function with parameter θ = {A θ }: Figure 7: Illustration of AtomG2G decoding process. Atoms marked with red circles are frontier nodes in the queue Q. In each step, the model picks the first node v t from Q and predict whether there will be new atoms attached to v t . If so, it predicts the atom type of new node u t (atom prediction). Then the model predicts the bond type between u t and other nodes in Q sequentially for |Q| steps (bond prediction, |Q| = 2). Finally, it adds the new atom to the queue Q. AtomG2G Architecture AtomG2G is an atom-based translation method that is directly comparable to HierG2G. Here molecules are represented solely as molecular graphs rather than a hierarchical graph with substructures. Table 3 : Training set size and substructure vocabulary size for each dataset.
We propose a multi-resolution, hierarchically coupled encoder-decoder for graph-to-graph translation. A hierarchical graph-to-graph translation model to generate molecular graphs using chemical substructures as building blocks that is fully autoregressive and learns coherent multi-resolution representations, outperforming previous models. The authors present a hierarchical graph-to-graph translation method for generating novel organic molecules.