<p>It looks like multiple instance learning might be your approach. Check out these two papers:</p>

<p><a href="https://arxiv.org/pdf/1610.03155.pdf" rel="nofollow noreferrer">Multiple Instance Learning Convolutional Neural
Networks for Object Recognition</a></p>

<p><a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4908336/pdf/btw252.pdf" rel="nofollow noreferrer">Classifying and segmenting microscopy images
with deep multiple instance learning</a></p>

<p>The last one is implemented by @dancsalo (not sure if he has a stack overflow account) <a href="https://github.com/dancsalo/TensorFlow-MIL" rel="nofollow noreferrer">here</a>.</p>

<p>I looks like the second paper deals with very large images and breaks them into sub images, but labels the entire image. So, it is like labeling a bag of images with a label instead of having to make a label for each sub-image. In your case, you might be able to construct a matrix of images, i.e. a 10 image x 10 image master image for each of the scans...</p>

<p>Let us know if you do this and if it works well on your data set!</p>
Tensorflow - classify based on multiple image as input, not signle one
<p>I'm building CNN that will tell me if a person has brain damage. I'm planning to use <a href="https://github.com/tensorflow/models/tree/f87a58cd96d45de73c9a8330a06b2ab56749a7fa/research/inception" rel="nofollow noreferrer">tf inception v3</a> model, and <a href="https://github.com/tensorflow/models/blob/f87a58cd96d45de73c9a8330a06b2ab56749a7fa/research/inception/inception/data/build_image_data.py" rel="nofollow noreferrer">build_image_data.py</a> script to build <em>TFRecord</em>.</p>

<p>Dataset is composed of brain scans. Every scan has about 100 images(different head poses, angles). On some images, damage is visible, but on some is not. I can't label all images from the scan as a damage positive(or negative), because some of them would be labeled wrong(if scan is positive on damage, but that is not visible on specific image).</p>

<p>Is there a way to label the whole scan as positive/negative and in that way train the network? 
And after training is done, pass scan as input to network(not single image) and classify it.</p>