title: NARX implementation using keras
body: <p>I am trying to implement a simple NARX network in keras (backend Tensoflow). I am building the model by subclassing <code>tf.keras.Model</code>. A NARX network can be trained as a simple feedforward network (series-parallel architecture), therefore I did not have any problem with this definition. However for prediction I need to close the loop and give back the output as input to the model. How can I modify the network so that it is able to make predictions and forecastings?</p>
  
  <p>Here is a minimal version of the code:</p>
  
  <pre><code>import tensorflow as tf
  from tensorflow import keras
  
  import numpy as np
  import pandas as pd
  import matplotlib.pyplot as plt
  numPreviousSteps = 8
  inputShape = (None, numPreviousSteps + 2)
  
  class Narx(keras.Model):
  
      def __init__(self):
          super(Narx, self).__init__(name='narx')
          self.dense = keras.layers.Dense(10, input_shape=inputShape,
                                          activation=keras.activations.tanh)
          self.outputLayer = keras.layers.Dense(1, activation=keras.activations.linear)
  
      def call(self, inputs, training = False):
          if (training):
              x = self.dense(inputs)
              return self.outputLayer(x)
          else: # TODO: what should the network do when used for prediction
              x = self.dense(inputs)
              return self.outputLayer(x)
  
  
  model = Narx()
  model.compile(optimizer=keras.training.RMSPropOptimizer(0.001),
                loss=tf.losses.mean_squared_error,
                metric=tf.metrics.mean_absolute_error)
  
  # input data generation
  numTsSamples = 1000
  
  # time series to learn from
  y = np.random.random((numTsSamples + numPreviousSteps + 1,))
  x = np.random.random((numTsSamples,)) # exogenous input
  
  # creation of tapped delay
  data = [np.roll(y, -i)[:numTsSamples] for i in range(numPreviousSteps, -1, -1)]
  data = [x] + data
  
  # training data
  data = np.stack(data, axis=1)
  
  # expected results
  yNext = y[numPreviousSteps : -1]
  
  # model training
  model.fit(data, yNext)
  </code></pre>