style="font-variant:small-caps;">Semfit</span>]{} layer. This could cause problems to tasks like machine translation or text generation where the ability to predict the next word is essential. For contextual representation, there exists no such problem as we do not seek to induce $\bm{P}$.

As a way to predict the next word with the [<span style="font-variant:small-caps;">Semfit</span>]{} layer, proposed to search for the nearest neighbor of the context vector $\bm{c}$ in the target embedding space $\bm{W}$. This decoding algorithm is sub-optimal unless the distribution of $p(w|c)$ is sharp. Suppose our training resulted in the global optimum, finding the nearest neighbor of $\bm{c}$ is equivalent to finding the word vector with the largest inner product with $\bm{p(w|c)}\bm{W}$[^8]. For a specific word $w_0$, its inner product $$\begin{aligned}
   d_{w_0} & = & \bm{c_{optim}} \cdot \bm{w_0} \nonumber \\
   & \propto & \bm{p(w|c)}\bm{W} \cdot \bm{w_0} \nonumber \\
   & = & p(w_0|c) + \Sigma_{w'\neq w_0} p(w'|c) \bm{w}' \cdot \bm{w_0} \quad \label{equation:dot}\end{aligned}$$ $d_{w_0}$ is influenced not only by $w_0$’s conditional probability, but also by its distance to other words and their conditional probabilities. As a result, the most desired word does not always get chosen. Rewriting this in a matrix form, we are essentially calculating $\bm{p(w|c)}\bm{WW^T}$ during decoding. Unless $\bm{W}$ is from 