title: How to stack layers in Keras without using Sequential()?
body: <p>If I have a keras layer L, and I want to stack N versions of this layer (with different weights) in a keras model, what's the best way to do that? Please note that here N is large and controlled by a hyper param. If N is small then this not a problem (we can just manually repeat a line N times). So let's assume N > 10 for example.</p>
  
  <p>If the layer has only one input and one output, I can do something like:</p>
  
  <pre><code>m = Sequential()
  for i in range(N):
      m.add(L)
  </code></pre>
  
  <p>But this is not working if my layer actually takes multiple inputs. For example, if my layer has the form z = L(x, y), and I would like my model to do:</p>
  
  <pre><code>x_1 = L(x_0, y)
  x_2 = L(x_1, y)
  ...
  x_N = L(x_N-1, y)
  </code></pre>
  
  <p>Then Sequential wouldn't do the job. I think I can subclass a keras model, but I don't know what's the cleanest way to put N layers into the class. I can use a list, for example:</p>
  
  <pre><code>class MyModel(Model):
      def __init__(self):
          super(MyModel, self).__init__()
          self.layers = []
          for i in range(N):
              self.layers.append(L)
      def call(self, inputs):
          x = inputs[0]
          y = inputs[1]
          for i in range(N):
              x = self.layers[i](x, y)
          return x
  </code></pre>
  
  <p>But this is not ideal, as keras won't recognize these layers (it seems not thinking list of layers as "checkpointables"). For example, MyModel.variables would be empty, and MyModel.Save() won't save anything. </p>
  
  <p>I also tried to define the model using the functional API, but it won't work in my case as well. In fact if we do</p>
  
  <pre><code>def MyModel():
      input = Input(shape=...)
      output = SomeLayer(input)
      return Model(inputs=input, outputs=output)
  </code></pre>
  
  <p>It won't run if SomeLayer itself is a custom model (it raises NotImplementedError).</p>
  
  <p>Any suggestions?</p>