title: Trouble with zero-padding inputs for Steered Convolution Layer
body: <p>I'm using Tensorflow's new graphics library to apply a steered convolution to a series of meshes.  In many cases, you will have a series of meshes that are not the same size and you must zero-pad the smaller ones.  According to the documentation, the "sizes" argument of the graph_conv.feature_steered_convolution_layer function takes in an int tensor consisting of the number of non-padded elements of each mesh.  For some reason, when this argument is set something other than "None", I get a warning telling me that the sparse array used in the "neighbors" argument is being converted to a dense matrix.  This causes my program to run absurdly slowly.  </p>
  
  <p>The issue seems to be tied to the way that it calculates gradients.  If the optimizer is commented out, the error does not come up.  </p>
  
  <p>I read about a similar problem (link below) where the solution to the problem was to use tf.dynamic_partition rather than tf.gather.  However, the tf.gather functions, in this case are located within the graph_convolution library.  I attempted to make some edits in my copy of the library, but to no avail.</p>
  
  <p><a href="https://stackoverflow.com/questions/45882401/how-to-deal-with-userwarning-converting-sparse-indexedslices-to-a-dense-tensor/45917500#45917500">How to deal with UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape</a></p>
  
  <pre class="lang-py prettyprint-override"><code>from __future__ import absolute_import
  from __future__ import division
  from __future__ import print_function
  
  from absl.testing import parameterized
  import numpy as np
  import tensorflow as tf
  
  from tensorflow_graphics.nn.layer import graph_convolution as graph_conv
  
  #Number of meshes
  N = 2
  #Number of spatial dimensions
  d = 2
  
  #################################
  #Data consists of the vertices of two meshes.  The first mesh has 5 vertices and the second has 4.
      #Shape of data is (numberOfMeshes,maxNumberofVertices,numberofSpatialDimensions)
  
  #An array containing the actual size of each non-padded mesh
  sz = np.array([5,4],dtype=np.int64)
  #The maximum number of vertices in a mesh
  datav = 5
  
  #Input placeholder for input data (vertices)
  V0 = tf.placeholder(dtype=tf.float64,name="V0",shape=(N,datav,d)) 
  #Input Placeholder for labels for classification (For now, I'm just using throw-away data as my labels)
  L = tf.placeholder(shape=(N,5,1),dtype=tf.float64)
  SZ = tf.placeholder(shape=(N),dtype=tf.int64)
  #Input placeholder for the sparse array representing the adjacency matrix shape:(numberOfMeshes,datav,datav)
      #The warning is not raised if "SZ" is changed to "None
  adj_sp = tf.sparse_placeholder(shape=(SZ.shape[0],datav,datav),dtype=tf.float64,name='SPP')
  
  
  
  #The steered graph convolution that is included in Tensorflow's new graphics package
  output = graph_conv.feature_steered_convolution_layer(data=V0,neighbors=adj_sp,sizes=SZ,translation_invariant=False,num_weight_matrices=1,num_output_channels=1)
  
  loss = tf.losses.softmax_cross_entropy(L,output, weights=1.0)
  optimizer = tf.train.AdamOptimizer(learning_rate=.001).minimize(loss) #Warning not raised if this is commented out
  </code></pre>
  
  <p>When the above code is run, I get the following warning:</p>
  
  <pre><code>C:\Python37\lib\site-packages\tensorflow\python\ops\gradients_impl.py:110: 
  UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown 
  shape. This may consume a large amount of memory.
    "Converting sparse IndexedSlices to a dense Tensor of unknown shape. "
  </code></pre>
  
  <p>I am starting to think that this might have more to do with the library itself than with this piece of code.  I have referenced this posed in GitHub in case it requires updates (or additional documentation) to the library.
  <a href="https://github.com/tensorflow/graphics/issues/13" rel="nofollow noreferrer">https://github.com/tensorflow/graphics/issues/13</a></p>