title: Scipy sparse CSR matrix to TensorFlow SparseTensor - Mini-Batch gradient descent
body: <p>I have a Scipy sparse CSR matrix created from sparse TF-IDF feature matrix in SVM-Light format. The number of features is huge and it is sparse so I have to use a SparseTensor or else it is too slow. </p>
  
  <p>For example, number of features is 5, and a sample file can look like this:</p>
  
  <pre><code>0 4:1
  1 1:3 3:4
  0 5:1
  0 2:1
  </code></pre>
  
  <p>After parsing, the training set looks like this:</p>
  
  <pre><code>trainX = &lt;scipy CSR matrix&gt;
  trainY = np.array( [0,1,00] )
  </code></pre>
  
  <p>I have two important questions:</p>
  
  <p>1) How I do convert this to a SparseTensor (sp_ids, sp_weights) efficiently so that I perform fast multiplication (W.X) using lookup: <a href="https://www.tensorflow.org/versions/master/api_docs/python/nn.html#embedding_lookup_sparse" rel="noreferrer">https://www.tensorflow.org/versions/master/api_docs/python/nn.html#embedding_lookup_sparse</a></p>
  
  <p>2) How do I randomize the dataset at each epoch and recalculate sp_ids, sp_weights to so that I can feed (feed_dict) for the mini-batch gradient descent.</p>
  
  <p>Example code on a simple model like logistic regression will be very appreciated. The graph will be like this:</p>
  
  <pre><code># GRAPH
  mul = tf.nn.embedding_lookup_sparse(W, X_sp_ids, X_sp_weights, combiner = "sum")  # W.X
  z = tf.add(mul, b) #  W.X + b
  
  
  cost_op = tf.reduce_sum(tf.nn.sigmoid_cross_entropy_with_logits(z, y_true))  # this already has built in sigmoid apply
  train_op = tf.train.GradientDescentOptimizer(0.05).minimize(cost_op)  # construct optimizer
  
  predict_op = tf.nn.sigmoid(z) # sig(W.X + b)
  </code></pre>