<p><strong>Segmentation Accuracy</strong></p>

<p>This is a pretty common problem addressed in image segmentation literature, e.g., <a href="https://stackoverflow.com/questions/13974167/how-to-test-accuracy-of-segmentation-algorithm">here is a StackOverflow post</a></p>

<p>One common approach is to consider the ratio of "correct pixels" to "incorrect pixels," which is common in <strong>image segmentation</strong> for safety domain, e.g., <a href="https://arxiv.org/abs/1703.06870" rel="nofollow noreferrer">Mask RCNN</a>, <a href="http://www.cs.cmu.edu/~aayushb/pixelNet/" rel="nofollow noreferrer">PixelNet</a>. </p>

<p>Treating it as more of an <strong>object detection</strong> task, you could take the overlap of the hull of the objects and just measure <a href="https://dspace2.flinders.edu.au/xmlui/bitstream/handle/2328/27165/Powers%20Evaluation.pdf?sequence=1&amp;isAllowed=y" rel="nofollow noreferrer">accuracy</a> (commonly broken down into <a href="https://en.wikipedia.org/wiki/Precision_and_recall" rel="nofollow noreferrer">precision, recall</a>, <a href="https://en.wikipedia.org/wiki/F1_score" rel="nofollow noreferrer">f-score</a>, and other measures with <a href="https://dspace2.flinders.edu.au/xmlui/handle/2328/27165" rel="nofollow noreferrer">various bias/skews</a>). This allows you to produce an <a href="https://en.wikipedia.org/wiki/Receiver_operating_characteristic" rel="nofollow noreferrer">ROC curve</a> that can be calibrated for false positives/false negatives.</p>

<p>There is no domain-agnostic consensus on what's correct. <a href="http://www.cvlibs.net/datasets/kitti/eval_semantics.php" rel="nofollow noreferrer">KITTI provides both.</a></p>

<p>Mask RCNN is open source state-of-the-art, and provides implemenations 
<strong>in python</strong> of</p>

<ul>
<li><a href="https://github.com/matterport/Mask_RCNN/blob/master/mrcnn/utils.py#L665" rel="nofollow noreferrer">Computing image matching between segmented and original</a></li>
<li><a href="https://github.com/matterport/Mask_RCNN/blob/master/mrcnn/visualize.py#L171" rel="nofollow noreferrer">Displaying the differences</a></li>
</ul>

<p>In your domain (medicine), standard statistical rules apply. Use a holdout set. Cross validate. Etc. (*)</p>

<p><em>Note:</em> although the literature space is dauntingly large, I'd caution you to take a look at some domain-relevant papers, as they may take fewer "statistical short cuts" than other vision (digit recognition e.g.) projects accept. </p>

<ul>
<li>"<a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4533825/" rel="nofollow noreferrer">Metrics for evaluating 3D medical image segmentation: analysis, selection, and tool</a>" provides some summary methods in your your domain  </li>
<li>"<a href="https://www.annualreviews.org/doi/abs/10.1146/annurev.bioeng.2.1.315" rel="nofollow noreferrer">Current methods in image segmentation</a>" has about 2500 citations but is a little older. </li>
<li>"<a href="https://aapm.onlinelibrary.wiley.com/doi/abs/10.1118/1.597000" rel="nofollow noreferrer">Review of MR image segmentation techniques using pattern recognition</a>" is a little older still and will get you safely into "traditional" vision models. </li>
<li><a href="https://pubs.rsna.org/doi/abs/10.1148/radiology.218.2.r01fe44586" rel="nofollow noreferrer">Automated Segmentation of MR Images of Brain Tumors</a> is largely about its segmentation validation process </li>
</ul>

<hr>

<p><strong>Python</strong></p>

<p>Besides the mask rcnn links above, <a href="http://scikit-learn.org/stable/" rel="nofollow noreferrer">scikit-learn</a> provides some extremely user friendly tools and is considered part of the standard science "stack" for python.</p>

<p>Implementing the difference between images in python is trivial (using numpy). Here's an overkill <a href="https://stackoverflow.com/questions/189943/how-can-i-quantify-difference-between-two-images">SO link</a>. </p>

<p>Bounding box intersection in python <a href="https://www.pyimagesearch.com/2016/11/07/intersection-over-union-iou-for-object-detection/" rel="nofollow noreferrer">is easy to implement on one's own</a>; I'd use a library like <a href="https://stackoverflow.com/questions/14697442/faster-way-of-polygon-intersection-with-shapely">shapely if you want to measure general polygon intersection</a>.</p>

<p>Scikit-learn has some nice machine-learning evaluation tools, for example,</p>

<ul>
<li><a href="http://scikit-learn.org/stable/auto_examples/model_selection/plot_roc.html" rel="nofollow noreferrer">ROC curves</a></li>
<li><a href="http://scikit-learn.org/stable/modules/cross_validation.html" rel="nofollow noreferrer">Cross validation</a></li>
<li><a href="http://scikit-learn.org/stable/model_selection.html" rel="nofollow noreferrer">Model selection</a></li>
<li><a href="http://scikit-learn.org/stable/modules/classes.html" rel="nofollow noreferrer">A million others</a></li>
</ul>

<hr>

<p><strong>Literature Searching</strong></p>

<p>One reason that you may have trouble searching for the answer is because you're trying to measure performance of an unsupervised method, clustering, in a <strong>supervised learning</strong> arena. "Clusters" are fundamentally under-defined in mathematics (**). You want to be looking at the supervised learning literature for accuracy measures.</p>

<p>There is literature on unsupervised learning/clustering, too, which looks for topological structure, generally. <a href="https://nlp.stanford.edu/IR-book/html/htmledition/evaluation-of-clustering-1.html" rel="nofollow noreferrer">Here's a very introductory summary</a>. I don't think that is what you want.</p>

<p>A common problem, especially at scale, is that supervised methods require labels, which can be <a href="http://openaccess.thecvf.com/content_cvpr_2018/papers/Zlateski_On_the_Importance_CVPR_2018_paper.pdf" rel="nofollow noreferrer">time consuming to produce accurately</a> for dense segmentation. Object detection <a href="http://vision.stanford.edu/documents/Russakovsky_PhD_thesis_2015.pdf" rel="nofollow noreferrer">makes it a little easier</a>.</p>

<p>There are some existing datasets for medicine (<a href="https://www2.eecs.berkeley.edu/Research/Projects/CS/vision/bsds/" rel="nofollow noreferrer">[1]</a>, <a href="https://www.kaggle.com/paultimothymooney/identification-and-segmentation-of-nuclei-in-cells" rel="nofollow noreferrer">[2]</a>, e.g.) and <a href="https://arxiv.org/pdf/1702.03407.pdf" rel="nofollow noreferrer">some ongoing research in label-less metrics</a>. If none of these are options for you, then you may have to revert to considering it an unsupervised problem, but evaluation becomes very different in scope and utility.</p>

<hr>

<p><strong>Footnotes</strong></p>

<p>[*] Vision people sometimes skip cross validation even though they shouldn't, mainly because the models are slow to fit and they're a lazy bunch. <em>Please don't skip a <a href="https://stats.stackexchange.com/questions/19048/what-is-the-difference-between-test-set-and-validation-set">train/test/validation split</a></em>, or your results may be dangerously useless</p>

<p>[**] You can find all sorts of "formal" definitions, but never two people to agree on which one is correct or most useful. <a href="http://www.stat.cmu.edu/~larry/=sml/clustering.pdf" rel="nofollow noreferrer">Here's denser reading</a></p>
How to find the success rate of a clustering algorithm?
<p>I have implemented several clustering algorithms on an image dataset.
I'm interested in deriving the success rate of clustering. I have to detect the tumor area, in the original image I know where the tumor is located, I would like to compare the two images and obtain the percentage of success.
Following images:</p>

<p>Original image: I know the position of cancer<img src="https://i.stack.imgur.com/TlMJl.png" alt=""></p>

<p>Image after clustering algorithm<img src="https://i.stack.imgur.com/hDT1b.png" alt=""></p>

<p>I'm using python 2.7.</p>