<p>Good suggestions, which will probably do for most applications. If you want to get fancy and state-of-the-art, then you can train a model to predict unknown word embeddings. Take a look at this recent EMNLP 2017 paper: <a href="https://arxiv.org/pdf/1707.06961.pdf" rel="nofollow noreferrer">https://arxiv.org/pdf/1707.06961.pdf</a></p>
<p>TL;DR given a set of known word embeddings, the idea is to train a character-level BiLSTM which attempts to predict the embeddings given solely the characters of the word. Then this net can generalize to predict embeddings for unknown words. Ideally the net captures some morphological information, e.g. the predicted embedding for <code>apples</code> will be close to <code>apple</code>, and the evaluations in the paper seem to support this hypothesis.</p>
<p>There's a GitHub repository with pretrained models here: <a href="https://github.com/yuvalpinter/mimick" rel="nofollow noreferrer">https://github.com/yuvalpinter/mimick</a></p>
How to initialize word-embeddings for Out of Vocabulary Word?
<p>I am trying to use CoNLL-2003 NER (English) Dataset and I am trying to utilize pretrained embeddings for it. I am using SENNA pretrained embeddings. Now I have around 20k words in my vocabulary and out of this I have embedding available for only 9.5k words.<br>
My current approach is to initialize an array of <code>20k X embedding_size</code> with zeros and initialize the 9.5k words whose embeddings is known to me and make all the embeddings learn-able.<br></p>

<p>My question is what is the best way to do this? Any reference to such research will be very helpful?</p>