title: opencv - cropping handwritten lines (line segmentation)
body: <p>I'm trying to build a handwriting recognition system using python and opencv.
  The recognition of the characters is not the problem but the segmentation.
  I have successfully :</p>
  
  <ul>
  <li>segmented a word into single characters</li>
  <li>segmented a <strong>single sentence</strong> into words in the required order.</li>
  </ul>
  
  <p>But I couldn't segment different lines in the document. I tried sorting the contours (to avoid line segmentation and use only word segmentation) but it didnt work.
  I have used the following code to segment words contained in a handwritten document , but it returns the words out-of-order(it returns words in left-to-right sorted manner) :</p>
  
  <pre><code>import cv2
  import numpy as np
  #import image
  image = cv2.imread('input.jpg')
  #cv2.imshow('orig',image)
  #cv2.waitKey(0)
  
  #grayscale
  gray = cv2.cvtColor(image,cv2.COLOR_BGR2GRAY)
  cv2.imshow('gray',gray)
  cv2.waitKey(0)
  
  #binary
  ret,thresh = cv2.threshold(gray,127,255,cv2.THRESH_BINARY_INV)
  cv2.imshow('second',thresh)
  cv2.waitKey(0)
  
  #dilation
  kernel = np.ones((5,5), np.uint8)
  img_dilation = cv2.dilate(thresh, kernel, iterations=1)
  cv2.imshow('dilated',img_dilation)
  cv2.waitKey(0)
  
  #find contours
  im2,ctrs, hier = cv2.findContours(img_dilation.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
  
  #sort contours
  sorted_ctrs = sorted(ctrs, key=lambda ctr: cv2.boundingRect(ctr)[0])
  
  for i, ctr in enumerate(sorted_ctrs):
      # Get bounding box
      x, y, w, h = cv2.boundingRect(ctr)
  
      # Getting ROI
      roi = image[y:y+h, x:x+w]
  
      # show ROI
      cv2.imshow('segment no:'+str(i),roi)
      cv2.rectangle(image,(x,y),( x + w, y + h ),(90,0,255),2)
      cv2.waitKey(0)
  
  cv2.imshow('marked areas',image)
  cv2.waitKey(0)
  </code></pre>
  
  <p>Please note that i am able to segment all the words here <strong>but they appear out order.Is there any way to sort these contours in order of top to bottom</strong></p>
  
  <p><strong>OR</strong></p>
  
  <p><strong>segment the image into separate lines so that each line can be segmented into words using above code?</strong></p>