title: Multidimensional RNN on Tensorflow
body: <p>I'm trying to implement a 2D RNN in the context of human action classification (joints on one axis of the RNN and time on the other) and have been searching high and low for something in Tensorflow that could do the job. </p>
  
  <p>I heard of <code>GridLSTMCell</code> (<a href="https://www.tensorflow.org/api_docs/python/tf/contrib/rnn/GridLSTMCell" rel="nofollow noreferrer">internally</a> and <a href="https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/grid_rnn/python/ops/grid_rnn_cell.py" rel="nofollow noreferrer">externally</a> contributed) but couldn't get it to work with <code>dynamic_rnn</code> (accepts a 3-D tensor but I'd have to provide a 4-D tensor [batchsize, max_time, num_joints, n_features]). </p>
  
  <p>Additionally, <code>ndlstm</code> is also a (somewhat unknown) <a href="https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/ndlstm" rel="nofollow noreferrer">part</a> of the TF library which basically uses a normal 1-D LSTM and transposes the output to feed it into a second 1-D LSTM. This was also advocated <a href="https://github.com/tensorflow/tensorflow/issues/1453#issuecomment-194976468" rel="nofollow noreferrer">here</a> but am not too sure if it's correct / if it's the same idea as what I would need.</p>
  
  <p>Any help would be appreciated.</p>