title: Bilinear Tensor Product in TensorFlow
body: <p>I'm working on re-implementing <a href="http://papers.nips.cc/paper/5028-reasoning-with-neural-tensor-networks-for-knowledge-base-completion.pdf" rel="nofollow noreferrer">this paper</a> and the key operation is a bilinear tensor product. I hardly know what that means, but the paper has a nice little graphic, which I understand. </p>
  
  <p><a href="https://i.stack.imgur.com/BC9gZ.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/BC9gZ.png" alt="enter image description here"></a></p>
  
  <p>The key operation is <strong>e_1 * W * e_2</strong>, and I want to know how to implement it in tensorflow, because the rest <em>should</em> be easy.</p>
  
  <p>Basically, given 3D tensor W, slice it into matrices, and for the j'th slice (a matrix), multiply it on each side by <strong>e_1</strong> and <strong>e_2</strong>, resulting in a scalar, which is the jth entry in the resulting vector (the output of this operation).</p>
  
  <p>So I want to perform a product of <strong>e_1</strong>, a d-dimensional vector, <strong>W</strong>, the d x d x k tensor, and <strong>e_2</strong>, another d-dimensional vector. Could this product be expressed concisely in TensorFlow as it is now, or would I have to define my own op somehow?</p>
  
  <p><strong>EARLIER EDITS</strong></p>
  
  <p>Why doesn't multiplying these tensors work, and is there some way to define it more explicitly so that it works? </p>
  
  <pre class="lang-py prettyprint-override"><code>&gt;&gt;&gt; import tensorflow as tf
  &gt;&gt;&gt; tf.InteractiveSession()
  &gt;&gt;&gt; a = tf.ones([3, 3, 3])
  &gt;&gt;&gt; a.eval()
  array([[[ 1.,  1.,  1.],
          [ 1.,  1.,  1.],
          [ 1.,  1.,  1.]],
  
         [[ 1.,  1.,  1.],
          [ 1.,  1.,  1.],
          [ 1.,  1.,  1.]],
  
         [[ 1.,  1.,  1.],
          [ 1.,  1.,  1.],
          [ 1.,  1.,  1.]]], dtype=float32)
  &gt;&gt;&gt; b = tf.ones([3, 1, 1])
  &gt;&gt;&gt; b.eval()
  array([[[ 1.]],
  
         [[ 1.]],
  
         [[ 1.]]], dtype=float32)
  &gt;&gt;&gt; 
  </code></pre>
  
  <p>The error message is </p>
  
  <pre class="lang-py prettyprint-override"><code>ValueError: Shapes TensorShape([Dimension(3), Dimension(3), Dimension(3)]) and TensorShape([Dimension(None), Dimension(None)]) must have the same rank
  </code></pre>
  
  <p><strong>CURRENTLY</strong></p>
  
  <p>Turns out that multiplying two 3D tensors doesn't work either with <code>tf.matmul</code>, so but <code>tf.batch_matmul</code> does. <code>tf.batch_matmul</code> will also do 3D tensors and matrices. Then I tried 3D and a vector: </p>
  
  <pre class="lang-py prettyprint-override"><code>ValueError: Dimensions Dimension(3) and Dimension(1) are not compatible
  </code></pre>