title: Keras Error: Please provide data which shares the same first dimension
body: <p>I try to use a customize layer in keras. It's a simple layer just a matmul with trainable params.</p>
  <pre><code>from tensorflow import keras
  import numpy as np
  from tensorflow.keras.models import Model
  from tensorflow.keras.layers import Input
  from tensorflow.keras.optimizers import RMSprop
  from keras.layers import Layer
  from tensorflow.keras import backend as K
  
  class MultiLayer(Layer):
      def __init__(self, output_dim, **kwargs):
          self.output_dim = output_dim
          super(MultiLayer, self).__init__(**kwargs)
      def build(self, input_shape):
          # Create a trainable weight variable for this layer
          if self.output_dim[0] != input_shape[1]:
              raise Exception(&quot;expect input shape with [{},?], but get input with shape {}&quot;.format(self.output_dim[0],input_shape), input_shape)
          self.kernel = self.add_weight(name='kernel', 
                                        shape=(input_shape[2], self.output_dim[0]),
                                        initializer='uniform',
                                        trainable=True)
          super(MultiLayer, self).build(input_shape)  # Be sure to call this at the end
      def call(self, x):
          return K.dot(x, self.kernel)
      def compute_output_shape(self, input_shape):
          return (input_shape[0], self.output_dim)
  
  x_fake = np.random.random((10,28,28))
  y_fake = [np.diag(np.ones(28))]*10
  input_shape = np.shape(x_fake)[1:]
  print(input_shape)
  ipt = Input(name='inputs',shape=input_shape)
  layer = MultiLayer((input_shape[0],input_shape[0]),name=&quot;dev&quot;)(ipt)
  #layer = Flatten()(layer)
  model = Model(inputs=ipt,outputs=layer)
  model.summary()
  rms = RMSprop()
  model.compile(loss=&quot;rms&quot;, optimizer=rms, metrics=['accuracy'])
  model.fit(x_fake,y_fake)
  
  </code></pre>
  <pre><code>_________________________________________________________________
  Layer (type)                 Output Shape              Param #   
  =================================================================
  inputs (InputLayer)          [(None, 28, 28)]          0         
  _________________________________________________________________
  dev (MultiLayer)             (None, 28, 28)            784       
  =================================================================
  Total params: 784
  Trainable params: 784
  Non-trainable params: 0
  _________________________________________________________________
  </code></pre>
  <p>but when I fit this model, error occurs.</p>
  <pre><code>ValueError: Data cardinality is ambiguous:
    x sizes: 10
    y sizes: 28, 28, 28, 28, 28, 28, 28, 28, 28, 28
  
  Please provide data which shares the same first dimension.
  </code></pre>
  <p>I don't know what the sizes mean.</p>
  <p>How to fix it?</p>