title: Merging layers on Keras (dot product)
body: <p>I've been following Towards Data Science's tutorial about word2vec and skip-gram models, but I stumbled upon a problem that I cannot solve, despite searching about it for hours and trying a lot of unsuccessful solutions.</p>
  
  <p><a href="https://towardsdatascience.com/understanding-feature-engineering-part-4-deep-learning-methods-for-text-data-96c44370bbfa" rel="noreferrer">https://towardsdatascience.com/understanding-feature-engineering-part-4-deep-learning-methods-for-text-data-96c44370bbfa</a></p>
  
  <p>The step that it shows you how to build the skip-gram model architecture seems deprecated because of the use of the Merge layer from keras.layers.</p>
  
  <p>I've seem many discussions about it, and the majority of answers was the you need to use the Functional API of Keras to merge layers now. But the problem is, I'm a total beginner in Keras and have no idea how to translate my code from Sequential to Functional, here's the code that the author used (and I copied):</p>
  
  <pre><code>from keras.layers import Merge
  from keras.layers.core import Dense, Reshape
  from keras.layers.embeddings import Embedding
  from keras.models import Sequential
  
  # build skip-gram architecture
  word_model = Sequential()
  word_model.add(Embedding(vocab_size, embed_size,
                           embeddings_initializer="glorot_uniform",
                           input_length=1))
  word_model.add(Reshape((embed_size, )))
  
  context_model = Sequential()
  context_model.add(Embedding(vocab_size, embed_size,
                    embeddings_initializer="glorot_uniform",
                    input_length=1))
  context_model.add(Reshape((embed_size,)))
  
  model = Sequential()
  model.add(Merge([word_model, context_model], mode="dot"))
  model.add(Dense(1, kernel_initializer="glorot_uniform", activation="sigmoid"))
  model.compile(loss="mean_squared_error", optimizer="rmsprop")
  
  # view model summary
  print(model.summary())
  
  # visualize model structure
  from IPython.display import SVG
  from keras.utils.vis_utils import model_to_dot
  
  SVG(model_to_dot(model, show_shapes=True, show_layer_names=False, 
  rankdir='TB').create(prog='dot', format='svg'))
  </code></pre>
  
  <p>And when I run the block, the following error is shown:</p>
  
  <pre><code>ImportError                               Traceback (most recent call last)
  &lt;ipython-input-79-80d604373468&gt; in &lt;module&gt;()
  ----&gt; 1 from keras.layers import Merge
        2 from keras.layers.core import Dense, Reshape
        3 from keras.layers.embeddings import Embedding
        4 from keras.models import Sequential
        5 
  
  ImportError: cannot import name 'Merge'
  </code></pre>
  
  <p>What I'm asking here is some guidance on how to transform this Sequential into a Functional API structure.</p>