<p>If I had to do a thing like that I would concatenate image features and numerical features once they have the same form - a feature vector. For that, you can view the convolutional part of the network as a feature extractor that turns into a list of features after the last pooling layer i.e. it will have a shape like [batch_size, 1, 1, N]. At this point you can easily append/concatenate your regular numerical features before feeding them into a dense layer.</p>

<p>Couple things I would be on the lookout for:</p>

<ul>
<li>be sure that numerical and convolutional features are from the same distribution i.e. BatchNorm is applied to both</li>
<li>be sure that they have roughly the same size i.e. if you have 2048 conv features and only 5 numerical features it might not quite work as is.</li>
</ul>

<p>You can get more inspiration from <a href="https://arxiv.org/abs/1606.07792" rel="nofollow noreferrer">Wide &amp; Deep Learning for Recommender Systems</a>.</p>
How can I use values, along with images, as input for training a Keras image classifier?
<p>I believe this is my first question here.</p>

<p>I am very new to Neural Networks. I just started working on one in Python that is supposed to look at levels of glucose in patients with a risk of diabetes and rank them from 1 to 3 on their risk of developing the disease. With 1 being high risk, and 3 being low risk.</p>

<p>Right now, I have ~110 graphs previously ranked by a doctor (42 risk 1, 51 risk 2, 10 of risk 3). I randomly took 25% of each group as the test set, and put the rest as training, then gave it to a Keras for learning. </p>

<p>It works just fine. Here's my code:</p>

<pre class="lang-py prettyprint-override"><code>        print("Convoluting")

        classifier.add(Convolution2D(32, 3, 3, input_shape = (64, 64, 3), activation = 'relu'))

        print("Pooling")

        classifier.add(MaxPooling2D(pool_size = (2,2)))

        print("Flattening")

        classifier.add(Flatten())

        print("Connecting")

        classifier.add(Dense(activation = 'relu', units=128))

        classifier.add(Dense(activation = 'softmax', units=3))

        print("Compiling CNN")

        classifier.compile(optimizer = 'adam', loss = 'categorical_crossentropy', metrics = ['accuracy'])

        print("Generating images")

        from keras.preprocessing.image import ImageDataGenerator

        train_datagen = ImageDataGenerator()

        test_datagen = ImageDataGenerator()

        print("Setting sets")

        training_set = train_datagen.flow_from_directory(
                        'dataset/train_set',
                        target_size=(64,64),
                        batch_size=Batches,
                        class_mode='categorical')

        test_set = test_datagen.flow_from_directory(
                        'dataset/test_set',
                        target_size=(64,64),
                        batch_size=Batches,
                        class_mode='categorical')

        print("training nn...")

        from IPython.display import display
        from PIL import Image

        classifier.fit_generator(
                    training_set,
                    steps_per_epoch=StepsPerEpoch,
                    epochs=Epochs,
                    validation_data=test_set,
                    validation_steps=ValidationSteps)
</code></pre>

<p>However, the accuracy after the training won't go above 0.4. Now, I know I have a relatively small sample for training a neural network, but I currently don't have access to information from more patients. I do, however, have access to demographic data from those patients, like weight, height, and age.</p>

<p><strong>Basically, I would like to somehow include the weight, height, and age of each patient along with the graph showing their levels of glucose. So my program knows to take that information into account when making a judgement.</strong></p>

<p>I haven't been able to find anything similar when searching online, though it may be due to my little knowledge on the matter. What should I do?</p>

<p>Thanks for your time.</p>