<p>Well, this is a situation ("negative images") where, as it was revealed relatively recently, the results are not what we may seem to expect them to be...</p>

<p>There is an unpublished paper @ ArXiv, which shows exactly that CNN models that have achieved almost perfect test accuracy at datasets like MNIST &amp; CIFAR-10, fail to give similar performance in the respective "negative" images (i.e. with inverted background &amp; foreground, like your case here):</p>

<p><a href="https://arxiv.org/abs/1703.06857" rel="nofollow noreferrer">On the Limitation of Convolutional Neural Networks in Recognizing Negative Images</a></p>

<p>Here is the main result of the paper:</p>

<p><a href="https://i.stack.imgur.com/MyeOc.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/MyeOc.png" alt="enter image description here"></a></p>

<p>The issue is rather non-trivial, and there has been strong disagreement in the community as to if this result is indeed expected &amp; unsurprising or not; see the (now archived) relevant discussion @ <a href="https://www.reddit.com/r/MachineLearning/comments/60tex8/r170306857_deep_neural_networks_do_not_recognize/" rel="nofollow noreferrer">Reddit</a>, as well as a relevant piece @ <a href="https://www.kdnuggets.com/2017/04/negative-results-images-flaw-deep-learning.html" rel="nofollow noreferrer">KDNuggets</a>.</p>

<p>All in all, as the paper also suggests, you can do it with <em>one</em> model, but you will need to include at least some such "negative" images in your training. See also the SO thread <a href="https://stackoverflow.com/questions/48839618/high-training-accuracy-but-low-prediction-performance-for-tensorflows-official">High training accuracy but low prediction performance for Tensorflow's official MNIST model</a>.</p>
Does training set's colour profile play role in Pattern Recognition with CNN?
<p>I've recently started with Deep Learning and CNN which as quoted, attempts to extract the most optimal features from samples on it's own.
I made a model to recognize characters where the training set had images with black background and script in white.
<a href="https://i.stack.imgur.com/HxsbJ.png" rel="nofollow noreferrer">Image Sample</a></p>

<p>This type of model though fails to recognize images with pattern in black on white background(I tried with my own input and the negative of previous set also).<a href="https://i.stack.imgur.com/DL9YS.png" rel="nofollow noreferrer">Negative of Image Sample</a></p>

<p>Is it possible to recognize both types of images using the same model or do I need to train two separate models?
I don't know if it's possible using ImageDataGenerator class.
Following is the current code snippet:</p>

<pre><code>train_datagen = ImageDataGenerator(rescale = 1./255, shear_range = 0.2, zoom_range = 0.2, horizontal_flip = True)
test_datagen = ImageDataGenerator(rescale = 1./255)
</code></pre>