title: What's the point to have a UNK token for out of vocabulary words during decoding?
body: <p>First of all, I know this question is kind of off-topic, but I have already tried to ask elsewhere but got no response.</p>
  
  <p>Adding a <code>UNK</code> token to the vocabulary is a conventional way to handle oov words in tasks of NLP. It is totally understandable to have it for encoding, but what's the point to have it for decoding? I mean you would never expect your decoder to generate a <code>UNK</code> token during prediction, right?</p>