title: How to seperate coloured boxes from the main box
body: <p>Here is the main input image(named 1.png)</p>
  
  <p><a href="https://drive.google.com/open?id=1Ga5sP89jttP4pWhdnBet9l_bwYxC97f5" rel="nofollow noreferrer">https://drive.google.com/open?id=1Ga5sP89jttP4pWhdnBet9l_bwYxC97f5</a></p>
  
  <p>Now, I want to identify which box is filled with color and which box is empty,
  in the center big box and not the 20 side boxes that have number.</p>
  
  <p>I wrote the code to extract main big box from image </p>
  
  <pre><code>image = cv2.imread(path)
  
  gray = cv2.cvtColor(image , cv2.COLOR_BGR2GRAY)
  
  edge = cv2.Canny(gray.copy() , 10 ,70)
  
  _,contours,_ = cv2.findContours(edge.copy() , cv2.RETR_EXTERNAL ,cv2.CHAIN_APPROX_SIMPLE)
  
  cv2.drawContours(image, contours , -1, (0,0,255) , 3)
  plt.imshow(image)
  cv2.imwrite('image.jpg',image)
  </code></pre>
  
  <p>Now the image looks like this </p>
  
  <p><a href="https://drive.google.com/open?id=1bbqUac1Da2gLuMLly_tTAKPdkutQ5jJ-" rel="nofollow noreferrer">https://drive.google.com/open?id=1bbqUac1Da2gLuMLly_tTAKPdkutQ5jJ-</a></p>
  
  <p>Then I sorted the contours to get main matrix box which has all the colored and non colored boxes.</p>
  
  <pre><code>sorted_contours = sorted(contours,key = cv2.contourArea,reverse=True)
  </code></pre>
  
  <p>Then I seperated the main box</p>
  
  <pre><code>img = cv2.imread('1.png')
  cnt = sorted_contours[0]
  x,y,w,h = cv2.boundingRect(cnt)
  main_box = img[y:y+h,x:x+w].copy()
  cv2.imwrite('main_box.jpg',main_box)
  plt.imshow(main_box)
  </code></pre>
  
  <p>The main box looks like this:</p>
  
  <p><a href="https://drive.google.com/open?id=1iEhLxBkENWyam0CVFTPx-IbmNOl843MN" rel="nofollow noreferrer">https://drive.google.com/open?id=1iEhLxBkENWyam0CVFTPx-IbmNOl843MN</a></p>
  
  <p>Mark the exterior box</p>
  
  <pre><code>main_box_gray = cv2.cvtColor(main_box,cv2.COLOR_BGR2GRAY)
  _, main_box_gray = cv2.threshold(main_box_gray,5,255,cv2.THRESH_BINARY)
  
  
  _,t_c,_ = cv2.findContours(main_box_gray.copy(),cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)
  
  colored_main_box = img[y:y+h,x:x+w].copy()
  
  cv2.drawContours(colored_main_box,t_c,-1,(255,0,0),2)
  </code></pre>
  
  <p>The outer box is marked,now I separated the outer contour to get boxes inside:</p>
  
  <pre><code>sorted_box = sorted(t_c,key = cv2.contourArea,reverse=True)
  
  colored_main_box = img[y:y+h,x:x+w].copy()
  cnt = sorted_box[0]
  
  x2,y2,w2,h2 = cv2.boundingRect(cnt)
  
  temp_image = colored_main_box[y2:y2+h2,x2:x2+w2].copy()
  
  
  edge_temp = cv2.Canny(temp_image,100,200)
  
  _,t_c_1,_ = cv2.findContours(edge_temp.copy(),cv2.RETR_LIST,cv2.CHAIN_APPROX_SIMPLE)
  
  colored_main_box = img[y:y+h,x:x+w].copy()
  
  colored_main_box = colored_main_box[y2:y2+h2,x2:x2+w2].copy()
  
  cv2.drawContours(colored_main_box,t_c_1,-1,(0,0,255),1)
  
  plt.imshow(colored_main_box)
  cv2.imwrite("full_marked.jpg",colored_main_box)
  </code></pre>
  
  <p>colored_main_box(Full marked ):</p>
  
  <p><a href="https://drive.google.com/open?id=1lGKXlc_7g9kYWSWfXnfYIZQZonNygKer" rel="nofollow noreferrer">https://drive.google.com/open?id=1lGKXlc_7g9kYWSWfXnfYIZQZonNygKer</a></p>
  
  <p>Now I have marked all inner boxes and i can find which box are filled by using  cv2.countNonZero funtion, but my main task is to save color inside the filled box as a separate image without the exterior black border of the box that contains that color fill, and the saved image that contain only color fill,
  should be named after the box that it was extracted.</p>
  
  <p>Eg: in main_box image:</p>
  
  <p><a href="https://drive.google.com/open?id=1iEhLxBkENWyam0CVFTPx-IbmNOl843MN" rel="nofollow noreferrer">https://drive.google.com/open?id=1iEhLxBkENWyam0CVFTPx-IbmNOl843MN</a></p>
  
  <p>The orange color should be saved as  8.jpg</p>
  
  <p>The Red color should be saved as 12.jpg</p>
  
  <p>The green color should be saved as 18.jpg</p>
  
  <p>The yellow should be saved as 19.jpg</p>
  
  <p>The blue color should be number 21.jpg </p>
  
  <p>Please help me with the code to extract and save the color fills inside the boxes with correct number label.</p>
  
  <p>Thanks in advance</p>