tested several different configurations for the top layers. For the following configurations, we removed the heatmap and added two Resnet or VGG blocks on top of the patch classifier layers, followed by a global average pooling layer and the classification output. The models were evaluated by per-image AUCs on the test set.

**Resnet-based networks** \[resnet\_auc\] To evaluate whether the patch classifiers trained on the S1 and S10 sets are equally useful for whole image classification, the Resnet50 patch classifiers were used, followed by two Resnet blocks of the same configuration of $[512-512-2048] \times 1$ (Table \[tab:resnet\_auc\]). The whole image classifier based on the S10 set (AUC=0.85) performed much better than the one based on the S1 set (AUC=0.63). The S10 set contains more information about the ROIs as well as their adjacent regions and background regions than the S1 set, which can be important for whole image classification. For the rest of the study, only patch classifiers trained on the S10 set were used. Varying the configuration by using two Resnet blocks of the same configuration of $[512-512-1024] \times 2$ yielded the same AUC of 0.85 while reducing the depths of the two Resnet blocks decreased the AUC by 0.01-0.02 