<p>This phenomenon is called catastrophic forgetting of neural networks. </p>

<p>You can read a paper on this: <a href="https://arxiv.org/pdf/1708.02072.pdf" rel="nofollow noreferrer">https://arxiv.org/pdf/1708.02072.pdf</a></p>

<p>Your <code>toyX</code> and <code>toyX2</code> has a completely different distribution. When you re-train your model with ToyX2 for 1000 epochs, your model has completely forgotten the mapping from <code>toyX</code> to <code>toyY</code>.</p>

<p>You should train only for very few epochs with a really small learning rate if you want to make sure the knowledge of previous training stays or just mix both of the sets and re-train.</p>
Prediction by Neural network Keras is not working
<p>I want to write a NN program for recognition using Keras.</p>

<p>I have used 2 sets of data for training:</p>

<pre><code>toyX = [1, 2, 3, 4, 5, 6, 7, 8]
toyX2 = [18, 17, 16, 15, 14, 13, 12, 11].
</code></pre>

<p>After  training with <code>toyX</code> and then <code>toyX2</code>, the output of <code>model.predict(toyX)</code> is <code>[[0.56053144 1.0758346  1.7890009 ]]</code>. However, it should have been <code>[6, 11, 14]</code>.</p>

<p>Should I add more layers or change parameters to improve prediction? Which parameters I should change?</p>

<p>Please help me to solve this problem.</p>

<pre><code>from keras.models import Sequential
from keras.optimizers import Adam
from keras.layers import Conv1D, MaxPooling1D
from keras.layers import Dense, Flatten
from keras.layers import Dropout
import numpy as np

model = Sequential()
model.add(Conv1D(filters=64, kernel_size=3, activation='relu', input_shape=(8, 1)))
model.add(Conv1D(filters=64, kernel_size=3, activation='relu'))
model.add(Dropout(0.5))
model.add(MaxPooling1D(pool_size=2))
model.add(Flatten())
model.add(Dense(100, activation='relu'))
#model.add(Dense(3, activation='softmax'))
model.add(Dense(3))
#model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
model.compile(loss='mean_squared_error', optimizer='adam', metrics=['accuracy'])
print(model.summary())


toyX = np.array([1, 2, 3, 4, 5, 6, 7, 8]).reshape(1, 8, 1)
toyX2 = np.array([18, 17, 16, 15, 14, 13, 12, 11]).reshape(1, 8, 1)
#print (toyX.shape)
toyY = np.array([6, 11, 14]).reshape(1, 3)
#print (toyY.shape)
toyY2 = np.array([1, 2, 3]).reshape(1, 3) # if flatten is active

model.fit(toyX, toyY, epochs = 1000, verbose = 0)
model.fit(toyX2, toyY2, epochs = 1000, verbose = 0)

print (model.predict(toyX))
</code></pre>