<p>It is often not a good idea to adjust word2vec embeddings if you do not have sufficiently large corpus in your training. To clarify that, take an example where your corpus has <em>television</em> but not <em>TV</em>. Even though they might have word2vec embeddings, after training only <em>television</em> will be adjust and not <em>TV</em>. So you disrupt the information from word2vec.</p>

<p>To solve this problem you have 3 options:</p>

<ol>
<li>You let the LSTM in the upper layer figure out what the word might mean based on its context. For example, <em>I like choc.</em> the LSTM can figure out it is an object. This was demonstrated by <a href="https://arxiv.org/abs/1410.3916" rel="nofollow noreferrer">Memory Networks</a>.</li>
<li>Easy option, pre-process, canonicalise as much as you can before passing to the model. Spell checkers often capture these very well and are really fast.</li>
<li>You can use character encoding along side word2vec. This is employed in many of the question answering models such as <a href="https://arxiv.org/abs/1410.3916" rel="nofollow noreferrer">BiDAF</a> where the character representation is merged with word2vec so you have some information relating characters to words. In this case, <em>choc</em> might be similar to <em>chocolate</em>.</li>
</ol>
word2Vec and abbreviations
<p>I am working on text classification task where my dataset contains a lot of abbreviations and proper nouns. For instance: <strong>Milka choc. bar</strong>.<br>
My idea is to use bidirectional LSTM model with word2vec embedding.<br>
And here is my problem how to code words, that not appears in the dictionary? 
I partially solved this problem by merging pre-trained vectors with randomly initialized. Here is my implementation:</p>

<pre><code>import gensim
from gensim.models import Word2Vec
from gensim.utils import simple_preprocess

from gensim.models.keyedvectors import KeyedVectors

word_vectors = KeyedVectors.load_word2vec_format('ru.vec', binary=False, unicode_errors='ignore')

EMBEDDING_DIM=300
vocabulary_size=min(len(word_index)+1,num_words)
embedding_matrix = np.zeros((vocabulary_size, EMBEDDING_DIM))
for word, i in word_index.items():
    if i&gt;=num_words:
        continue
    try:
        embedding_vector = word_vectors[word]
        embedding_matrix[i] = embedding_vector
    except KeyError:
        embedding_matrix[i]=np.random.normal(0,np.sqrt(0.25),EMBEDDING_DIM)

def LSTMModel(X,words_nb, embed_dim, num_classes):
    _input = Input(shape=(X.shape[1],))
    X = embedding_layer = Embedding(words_nb,
                            embed_dim,
                            weights=[embedding_matrix],
                            trainable=True)(_input)
   X = The_rest_of__the_LSTM_model()(X)
</code></pre>

<p>Do you think, that allowing the model to adjust the embedding weights is a good idea? 
Could you please tell me, how can I encode words like <strong>choc</strong>? Obviously, this abbreviation stands for <strong>chocolate</strong>.   </p>