title: Character embeddings with Keras
body: <p>I am trying to implement the type of character level embeddings described in <a href="https://arxiv.org/pdf/1603.01360.pdf" rel="noreferrer">this paper</a> in Keras. The character embeddings are calculated using a bidirectional LSTM.</p>
  
  <p><a href="https://i.stack.imgur.com/PSvdx.png" rel="noreferrer"><img src="https://i.stack.imgur.com/PSvdx.png" alt="enter image description here"></a></p>
  
  <p>To recreate this, I've first created a matrix of containing, for each word, the indexes of the characters making up the word:</p>
  
  <pre><code>char2ind = {char: index for index, char in enumerate(chars)}
  max_word_len = max([len(word) for sentence in sentences for word in sentence])
  X_char = []
  for sentence in X:
      for word in sentence:
          word_chars = []
          for character in word:
              word_chars.append(char2ind[character])
  
          X_char.append(word_chars)
  X_char = sequence.pad_sequences(X_char, maxlen = max_word_len)
  </code></pre>
  
  <p>I then define a BiLSTM model with an embedding layer for the word-character matrix. I assume the input_dimension will have to be equal to the number of characters. I want a size of 64 for my character embeddings, so I set the hidden size of the BiLSTM to 32:</p>
  
  <pre><code>char_lstm = Sequential()
  char_lstm.add(Embedding(len(char2ind) + 1, 64))    
  char_lstm.add(Bidirectional(LSTM(hidden_size, return_sequences=True)))
  </code></pre>
  
  <p>And this is where I get confused. How can I retrieve the embeddings from the model? I'm guessing I would have to compile the model and fit it then retrieve the weights to get the embeddings, but what parameters should I use to fit it ?</p>
  
  <hr>
  
  <p>Additional details:</p>
  
  <p>This is for an NER task, so the dataset technically could be be anything in the word - label format, although I am specifically working with the WikiGold ConLL corpus available here: <a href="https://github.com/pritishuplavikar/Resume-NER/blob/master/wikigold.conll.txt" rel="noreferrer">https://github.com/pritishuplavikar/Resume-NER/blob/master/wikigold.conll.txt</a>
  The expected output from the network are the labels (I-MISC, O, I-PER...)</p>
  
  <p>I expect the dataset to be large enough to be training character embeddings directly from it. All words are coded with the index of their constituting characters, alphabet size is roughly 200 characters. The words are padded / cut to 20 characters. There are around 30 000 different words in the dataset.</p>
  
  <p>I hope to be able learn embeddings for each characters based on the info from the different words. Then, as in the paper, I would concatenate the character embeddings with the word's glove embedding before feeding into a Bi-LSTM network with a final CRF layer.</p>
  
  <p>I would also like to be able to save the embeddings so I can reuse them for other similar NLP tasks.</p>