I have such image

I want to select all mark items from it. Of course, I can use such code to implement it

DeleteSmallComponents[ColorNegate[Binarize[img]], 700]

But I'm very don't like using such magic threshold value 700 in my code. I mean such value will decrease the range of application about the method, for example, 700 cannot deal with this image or this. Any better method can do this?


We can do this without too much trouble by using the tendency of the "filled" answer choices to be:

  • high in pixel count compared to unfilled answers
  • filled in
  • rectangular

Import the multiple-choice test image:

mcTestIm = Import["https://i.stack.imgur.com/vSTAv.png"];

Negate it, so the dark sections can be treated as components:

answerComps = ColorNegate[mcTestIm];

Define a helper function:

ruleReverse[rule_] := Values[rule] -> Keys[rule]

Cluster the components based on size, rectangularity, and how filled they are:

similarShapes = FindClusters[ruleReverse /@ ComponentMeasurements[answerComps, {"Count", "ConvexCoverage", "Rectangularity"}]];

Define a function that looks at a set of component labels and finds the Median size of the named components:

componentSizes[cluster_] := Median[Values[ComponentMeasurements[SelectComponents[answerComps, MemberQ[cluster, #Label] &], "Count"]]]

Sort the clustered component labels so that the one containing the largest components is last:

componentSizeSortedClusters = SortBy[similarShapes, componentSizes];

Select the components in your base Image from that cluster:

SelectComponents[answerComps, MemberQ[Last[componentSizeSortedClusters], #Label] &]

Result of selecting the cluster of components with high pixel count.

EDIT: I hadn't seen your additional images to test for the robustness of our solutions. I am adding the results for those images below:


 componentSizes[cluster_, binIm_] := Median[Values[ComponentMeasurements[SelectComponents[binIm, MemberQ[cluster, #Label] &], "Count"]]]

 filledRectangularChoiceSelector[image_] := Block[
 {cnIm, srtdShpClstrs},
 cnIm = ColorNegate[image];
 srtdShpClstrs = SortBy[FindClusters[ruleReverse /@ ComponentMeasurements[cnIm, {"Count", "ConvexCoverage", "Rectangularity"}]], medianComponentSize[#, cnIm] &];
 SelectComponents[cnIm, MemberQ[Last[srtdShpClstrs], #Label] &]

 filledRectangularChoiceSelector /@ Import /@ {"https://i.stack.imgur.com/cfXU8.png", "https://i.stack.imgur.com/SDrud.png", "https://i.stack.imgur.com/vSTAv.png"}

Rasterize of the output of applying my selector to yode's example images

  • $\begingroup$ It is a good jod, but I have to say it is toooo slow. :) $\endgroup$ – yode Mar 21 '18 at 17:10

One solution:

img = RemoveAlphaChannel@Import["https://i.stack.imgur.com/vSTAv.png"];
kernel = Import["http://i.stack.imgur.com/DnGGR.png"];
corr = ImageAdjust@ImageCorrelate[img, kernel];
mask = MaxDetect[ColorNegate[corr], 0.2];
ImageDifference[img, FillingTransform[img, mask]]

Mathematica graphics

Of course, this has a magic constant on its own: the 0.2 in MaxDetect. We also have to specify kernel. Perhaps this isn't a problem if the method is still robust enough for your purpose, i.e. the constant and the kernel don't require fine tuning when using them on other images.

  • $\begingroup$ Thanks anyway, but it is not my expectation. :) $\endgroup$ – yode Mar 18 '18 at 12:04
  • 1
    $\begingroup$ @yode You have to specify what you are looking for in the image somehow, what is acceptable? Is the use of kernel acceptable? $\endgroup$ – C. E. Mar 18 '18 at 12:05
  • $\begingroup$ kernel hard to apply to all case to select mark items I think. $\endgroup$ – yode Mar 18 '18 at 12:13
  • 1
    $\begingroup$ @yode I think it's better if you post several images of different types and ask for a solution that works for all of them. Right now it is difficult to understand what is good enough. $\endgroup$ – C. E. Mar 18 '18 at 17:41
  • $\begingroup$ I'm afraid that will make the post be mess. But I support a new image for test again. $\endgroup$ – yode Mar 20 '18 at 2:03

A quick and dirty solution that works surprisingly satisfying and fast:

DeleteSmallComponents@Image@Total@ComponentMeasurements[ColorNegate@#, "Mask", #Area > .9 #FilledCount &][[;;,2]]&


I don't think 0.9 should be called as a magic number because it could be changed to any number from 0.73 to 0.93 without influencing the result.

Students like me are quite creative in the aspect of filling such cards. But no matter how we fill these cards, one thing is for certain: We would always draw a huge and densely filled and huge shape. and I used only these two rules to judge which is the filled part.

The first is that it must be densely filled instead of leaving a lot of blanks everywhere. So it's crucial for it to has a extraordinary high area-filled area ratio. This can successfully filter out a lot of unfilled blanks no matter what shape they take, and also this would tolerate situations when students paint a huge black region directly instead of filling out blanks one by one.

The second is that numbers or other elements that slips in are usually much smaller than painted area (though they could be not that small visually). So a direct DeleteSmallComponents would work perfectly.

Further improvements may include judging the threshold of DeleteSmallComponents manually to avoid recognition failure when students paint a tremendously huge black region. In this case DeleteSmallComponents may consider painted blocks as small components. But anyway, who would do that in an intense exam just to fool your program and make him or herself get a zero?


I think Erosion can be useful. The following works, except for the last image whose boxes are merged in the bottom right. However, these rare cases can be dealt with in the analysis part (if the white box is too high, it means the two rows have the same values).

filter[img_] := 
    MorphologicalComponents[Erosion[ColorNegate[img], DiskMatrix[4]], 
     Method -> "ConvexHull"] // Colorize // Binarize
img = Import["https://i.stack.imgur.com/vSTAv.png"];
img2 = Import["https://i.stack.imgur.com/cfXU8.png"];
img3 = Import["https://i.stack.imgur.com/SDrud.png"];
GraphicsRow[filter /@ {img, img2, img3}]

enter image description here

The only tuning parameter is the DiskMatrix, but 4 works for all.

Edit Another strategy that should be very robust (I have not tried to implement it):

  • first identity all the possible boxes (four per row). This can be done easily with MorphologicalComponents

  • then for each row, select the box that has the most probability to be ticked. This can be done by extracting, in the original image, the pixels for the four boxes, and then computing the average color.


Your Answer

