$j$ into a vector, and $\pi$ is a hyper-parameter to control smootheness. One can see that LSE with $\pi=1$ can serve as convex and differentiable approximation of max-pooling [@boyd2004convex]. While LSE with $\pi \rightarrow 0$ degenerates to sum-pooling.

RsNN Model {#sub:rnn_model}
----------

With the feature representations of object categories produced by CNN, the RsNN model is designed to generate the image parsing tree for predicting hierarchical structure and interaction relations. The inputs to scene configuration generation are a set $\Psi$ of nodes, where each node $\mathbf{v}_k \in \Psi$ denotes the feature representation of an object category. As illustrated in Fig. 3, the RsNN model takes two nodes $\mathbf{v}_k$ and $\mathbf{v}_l$ and their contextual information as the inputs. The output of RsNN includes three variables: (i) a single real value $h_{kl}$ to denote the confidence score of merging $\mathbf{v}_k$ and $\mathbf{v}_l$, (ii) a relation probability vector $y_{kl}$ for predicting relation label between the two nodes, and (iii) a feature vector $\mathbf{x}_{kl}$ as the combined representation. In each recursion step, the algorithm considers all pairs of nodes, and choose the pair (e.g., $\mathbf{v}_k$ and $\mathbf{v}_l$) with the highest score to merge. After the merging, we add $\mathbf{x}_{kl}$ and remove $\mathbf{v}_k$ and $\mathbf{v}_l$ 