{"action":"created","issue":{"url":"https://api.github.com/repos/tesseract-ocr/tesseract/issues/1192","repository_url":"https://api.github.com/repos/tesseract-ocr/tesseract","labels_url":"https://api.github.com/repos/tesseract-ocr/tesseract/issues/1192/labels{/name}","comments_url":"https://api.github.com/repos/tesseract-ocr/tesseract/issues/1192/comments","events_url":"https://api.github.com/repos/tesseract-ocr/tesseract/issues/1192/events","html_url":"https://github.com/tesseract-ocr/tesseract/issues/1192","id":269418183,"node_id":"MDU6SXNzdWUyNjk0MTgxODM=","number":1192,"title":"Noise characters recognized with bbox as the entire page","user":{"login":"TerryZH","id":399202,"node_id":"MDQ6VXNlcjM5OTIwMg==","avatar_url":"https://avatars1.githubusercontent.com/u/399202?v=4","gravatar_id":"","url":"https://api.github.com/users/TerryZH","html_url":"https://github.com/TerryZH","followers_url":"https://api.github.com/users/TerryZH/followers","following_url":"https://api.github.com/users/TerryZH/following{/other_user}","gists_url":"https://api.github.com/users/TerryZH/gists{/gist_id}","starred_url":"https://api.github.com/users/TerryZH/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/TerryZH/subscriptions","organizations_url":"https://api.github.com/users/TerryZH/orgs","repos_url":"https://api.github.com/users/TerryZH/repos","events_url":"https://api.github.com/users/TerryZH/events{/privacy}","received_events_url":"https://api.github.com/users/TerryZH/received_events","type":"User","site_admin":false},"labels":[{"id":880965165,"node_id":"MDU6TGFiZWw4ODA5NjUxNjU=","url":"https://api.github.com/repos/tesseract-ocr/tesseract/labels/accuracy","name":"accuracy","color":"7FDBFF","default":false},{"id":121383680,"node_id":"MDU6TGFiZWwxMjEzODM2ODA=","url":"https://api.github.com/repos/tesseract-ocr/tesseract/labels/bug","name":"bug","color":"FF4136","default":true}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":46,"created_at":"2017-10-29T17:33:27Z","updated_at":"2018-11-09T18:33:52Z","closed_at":null,"author_association":"NONE","body":"### Environment\r\n\r\n* **Tesseract Version**: v4.00.00dev-692-gad5ee18 with Leptonica\r\n* **Commit Number**: ad5ee18\r\n* **Platform**: MAC OS 16.7.0 Darwin Kernel Version 16.7.0: Thu Jun 15 17:36:27 PDT 2017; root:xnu-3789.70.16~2/RELEASE_X86_64 x86_64\r\n\r\n### Current Behavior:\r\n\r\nLine 1, unexpected '__' recognized between 1941 and Ritter, with bbox as the entire page.\r\n\r\n![sample](https://user-images.githubusercontent.com/399202/32145860-7bc02544-bca5-11e7-85a1-a510efded921.jpg)\r\nCorresponding HOCR line:\r\n     <span class='ocr_line' id='line_1_45' title=\"bbox 387 940 1989 1007; baseline -0.001 -19; x_size 65; x_descenders 17; x_ascenders 17\"><span class='ocrx_word' id='word_1_94' title='bbox 387 940 508 1007; x_wconf 74'>GS</span> <span class='ocrx_word' id='word_1_95' title='bbox 601 959 613 987; x_wconf 58'>1</span> <span class='ocrx_word' id='word_1_96' title='bbox 714 957 881 993; x_wconf 33'>2,261,002</span> <span class='ocrx_word' id='word_1_97' title='bbox 1007 957 1075 988; x_wconf 24'>Oct.</span> <span class='ocrx_word' id='word_1_98' title='bbox 1088 957 1232 991; x_wconf 63'>28,1941</span> <span class='ocrx_word' id='word_1_99' title='bbox 0 0 2710 3423; x_wconf 27'>__</span> <span class='ocrx_word' id='word_1_100' title='bbox 1318 956 1407 988; x_wconf 53'>Ritter</span> <span class='ocrx_word' id='word_1_101' title='bbox 1714 956 1815 1005; x_wconf 88'>760</span> <span class='ocrx_word' id='word_1_102' title='bbox 1841 941 1989 1000; x_wconf 62'>$FO</span> \r\n\r\n\r\n### Expected Behavior:\r\n\r\n'__' is not supposed to be recognized in the first place. If the false positive recognition is inevitable, the bbox information should be accurate.\r\n\r\n### Suggested Fix:\r\n\r\nn/a"},"comment":{"url":"https://api.github.com/repos/tesseract-ocr/tesseract/issues/comments/437452990","html_url":"https://github.com/tesseract-ocr/tesseract/issues/1192#issuecomment-437452990","issue_url":"https://api.github.com/repos/tesseract-ocr/tesseract/issues/1192","id":437452990,"node_id":"MDEyOklzc3VlQ29tbWVudDQzNzQ1Mjk5MA==","user":{"login":"jbreiden","id":4961958,"node_id":"MDQ6VXNlcjQ5NjE5NTg=","avatar_url":"https://avatars2.githubusercontent.com/u/4961958?v=4","gravatar_id":"","url":"https://api.github.com/users/jbreiden","html_url":"https://github.com/jbreiden","followers_url":"https://api.github.com/users/jbreiden/followers","following_url":"https://api.github.com/users/jbreiden/following{/other_user}","gists_url":"https://api.github.com/users/jbreiden/gists{/gist_id}","starred_url":"https://api.github.com/users/jbreiden/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jbreiden/subscriptions","organizations_url":"https://api.github.com/users/jbreiden/orgs","repos_url":"https://api.github.com/users/jbreiden/repos","events_url":"https://api.github.com/users/jbreiden/events{/privacy}","received_events_url":"https://api.github.com/users/jbreiden/received_events","type":"User","site_admin":false},"created_at":"2018-11-09T18:33:52Z","updated_at":"2018-11-09T18:33:52Z","author_association":"CONTRIBUTOR","body":"From Ray. Feedback welcome.\r\n\r\nFix github issue 1192 (as best as it can be without retraining).  Blob bounding boxes are clipped to the\r\ncorresponding word in the page layout analysis, provided there is a clear set of 1 or more significantly overlapping source \twords.  Where the LSTM has resegmented the words, there is no choice, but to use the bounding boxes provided by the LSTM model, which will be inaccurate.\r\n\r\n```diff\r\n--- tesseract/ccstruct/pageres.cpp\t2016-12-13 16:56:14.000000000 -0800\r\n+++ tesseract/ccstruct/pageres.cpp\t2018-11-09 10:28:33.000000000 -0800\r\n@@ -21,13 +21,14 @@\r\n  ** limitations under the License.\r\n  *\r\n  **********************************************************************/\r\n-#include          <stdlib.h>\r\n+#include \"pageres.h\"\r\n+#include <stdlib.h>\r\n #ifdef __UNIX__\r\n-#include          <assert.h>\r\n+#include <assert.h>\r\n #endif\r\n-#include          \"blamer.h\"\r\n-#include          \"pageres.h\"\r\n-#include          \"blobs.h\"\r\n+#include \"blamer.h\"\r\n+#include \"blobs.h\"\r\n+#include \"helpers.h\"\r\n \r\n ELISTIZE (BLOCK_RES)\r\n CLISTIZE (BLOCK_RES) ELISTIZE (ROW_RES) ELISTIZE (WERD_RES)\r\n@@ -1293,7 +1294,8 @@\r\n // Helper computes the boundaries between blobs in the word. The blob bounds\r\n // are likely very poor, if they come from LSTM, where it only outputs the\r\n // character at one pixel within it, so we find the midpoints between them.\r\n-static void ComputeBlobEnds(const WERD_RES& word, C_BLOB_LIST* next_word_blobs,\r\n+static void ComputeBlobEnds(const WERD_RES& word, const TBOX& clip_box,\r\n+                            C_BLOB_LIST* next_word_blobs,\r\n                             GenericVector<int>* blob_ends) {\r\n   C_BLOB_IT blob_it(word.word->cblob_list());\r\n   for (int i = 0; i < word.best_state.size(); ++i) {\r\n@@ -1313,8 +1315,74 @@\r\n         blob_it.set_to_list(next_word_blobs);\r\n       blob_end = (blob_box.right() + blob_it.data()->bounding_box().left()) / 2;\r\n     }\r\n+    blob_end = ClipToRange<int>(blob_end, clip_box.left(), clip_box.right());\r\n     blob_ends->push_back(blob_end);\r\n   }\r\n+  blob_ends->back() = clip_box.right();\r\n+}\r\n+\r\n+// Helper computes the bounds of a word by restricting it to existing words\r\n+// that significantly overlap.\r\n+static TBOX ComputeWordBounds(const tesseract::PointerVector<WERD_RES>& words,\r\n+                              int w_index, TBOX prev_box, WERD_RES_IT w_it) {\r\n+  constexpr int kSignificantOverlapFraction = 4;\r\n+  TBOX clipped_box;\r\n+  TBOX current_box = words[w_index]->word->bounding_box();\r\n+  TBOX next_box;\r\n+  if (w_index + 1 < words.size() && words[w_index + 1] != nullptr &&\r\n+      words[w_index + 1]->word != nullptr)\r\n+    next_box = words[w_index + 1]->word->bounding_box();\r\n+  for (w_it.forward(); !w_it.at_first() && w_it.data()->part_of_combo;\r\n+       w_it.forward()) {\r\n+    if (w_it.data() == nullptr || w_it.data()->word == nullptr) continue;\r\n+    TBOX w_box = w_it.data()->word->bounding_box();\r\n+    int height_limit = std::min<int>(w_box.height(), w_box.width() / 2);\r\n+    int width_limit = w_box.width() / kSignificantOverlapFraction;\r\n+    int min_significant_overlap = std::max(height_limit, width_limit);\r\n+    int overlap = w_box.intersection(current_box).width();\r\n+    int prev_overlap = w_box.intersection(prev_box).width();\r\n+    int next_overlap = w_box.intersection(next_box).width();\r\n+    if (overlap > min_significant_overlap) {\r\n+      if (prev_overlap > min_significant_overlap) {\r\n+        // We have no choice but to use the LSTM word edge.\r\n+        clipped_box.set_left(current_box.left());\r\n+      } else if (next_overlap > min_significant_overlap) {\r\n+        // We have no choice but to use the LSTM word edge.\r\n+        clipped_box.set_right(current_box.right());\r\n+      } else {\r\n+        clipped_box += w_box;\r\n+      }\r\n+    }\r\n+  }\r\n+  if (clipped_box.height() <= 0) {\r\n+    clipped_box.set_top(current_box.top());\r\n+    clipped_box.set_bottom(current_box.bottom());\r\n+  }\r\n+  if (clipped_box.width() <= 0) clipped_box = current_box;\r\n+  return clipped_box;\r\n+}\r\n+\r\n+// Helper moves the blob from src to dest. If it isn't contained by clip_box,\r\n+// the blob is replaced by a fake that is contained.\r\n+static TBOX MoveAndClipBlob(C_BLOB_IT* src_it, C_BLOB_IT* dest_it,\r\n+                            const TBOX& clip_box) {\r\n+  C_BLOB* src_blob = src_it->extract();\r\n+  TBOX box = src_blob->bounding_box();\r\n+  if (!clip_box.contains(box)) {\r\n+    int left =\r\n+        ClipToRange<int>(box.left(), clip_box.left(), clip_box.right() - 1);\r\n+    int right =\r\n+        ClipToRange<int>(box.right(), clip_box.left() + 1, clip_box.right());\r\n+    int top =\r\n+        ClipToRange<int>(box.top(), clip_box.bottom() + 1, clip_box.top());\r\n+    int bottom =\r\n+        ClipToRange<int>(box.bottom(), clip_box.bottom(), clip_box.top() - 1);\r\n+    box = TBOX(left, bottom, right, top);\r\n+    delete src_blob;\r\n+    src_blob = C_BLOB::FakeBlob(box);\r\n+  }\r\n+  dest_it->add_after_then_move(src_blob);\r\n+  return box;\r\n }\r\n \r\n // Replaces the current WERD/WERD_RES with the given words. The given words\r\n@@ -1365,66 +1433,45 @@\r\n   src_b_it.sort(&C_BLOB::SortByXMiddle);\r\n   C_BLOB_IT rej_b_it(input_word->word->rej_cblob_list());\r\n   rej_b_it.sort(&C_BLOB::SortByXMiddle);\r\n+  TBOX clip_box;\r\n   for (int w = 0; w < words->size(); ++w) {\r\n     WERD_RES* word_w = (*words)[w];\r\n+    clip_box = ComputeWordBounds(*words, w, clip_box, wr_it_of_current_word);\r\n     // Compute blob boundaries.\r\n     GenericVector<int> blob_ends;\r\n     C_BLOB_LIST* next_word_blobs =\r\n         w + 1 < words->size() ? (*words)[w + 1]->word->cblob_list() : NULL;\r\n-    ComputeBlobEnds(*word_w, next_word_blobs, &blob_ends);\r\n-    // Delete the fake blobs on the current word.\r\n+    ComputeBlobEnds(*word_w, clip_box, next_word_blobs, &blob_ends);\r\n+    // Remove the fake blobs on the current word, but keep safe for back-up if\r\n+    // no blob can be found.\r\n+    C_BLOB_LIST fake_blobs;\r\n+    C_BLOB_IT fake_b_it(&fake_blobs);\r\n+    fake_b_it.add_list_after(word_w->word->cblob_list());\r\n+    fake_b_it.move_to_first();\r\n     word_w->word->cblob_list()->clear();\r\n     C_BLOB_IT dest_it(word_w->word->cblob_list());\r\n     // Build the box word as we move the blobs.\r\n     tesseract::BoxWord* box_word = new tesseract::BoxWord;\r\n-    for (int i = 0; i < blob_ends.size(); ++i) {\r\n+    for (int i = 0; i < blob_ends.size(); ++i, fake_b_it.forward()) {\r\n       int end_x = blob_ends[i];\r\n       TBOX blob_box;\r\n       // Add the blobs up to end_x.\r\n       while (!src_b_it.empty() &&\r\n              src_b_it.data()->bounding_box().x_middle() < end_x) {\r\n-        blob_box += src_b_it.data()->bounding_box();\r\n-        dest_it.add_after_then_move(src_b_it.extract());\r\n+        blob_box += MoveAndClipBlob(&src_b_it, &dest_it, clip_box);\r\n         src_b_it.forward();\r\n       }\r\n       while (!rej_b_it.empty() &&\r\n              rej_b_it.data()->bounding_box().x_middle() < end_x) {\r\n-        blob_box += rej_b_it.data()->bounding_box();\r\n-        dest_it.add_after_then_move(rej_b_it.extract());\r\n+        blob_box += MoveAndClipBlob(&rej_b_it, &dest_it, clip_box);\r\n         rej_b_it.forward();\r\n       }\r\n-      // Clip to the previously computed bounds. Although imperfectly accurate,\r\n-      // it is good enough, and much more complicated to determine where else\r\n-      // to clip.\r\n-      if (i > 0 && blob_box.left() < blob_ends[i - 1])\r\n-        blob_box.set_left(blob_ends[i - 1]);\r\n-      if (blob_box.right() > end_x)\r\n-        blob_box.set_right(end_x);\r\n+      if (blob_box.null_box()) {\r\n+        // Use the original box as a back-up.\r\n+        blob_box = MoveAndClipBlob(&fake_b_it, &dest_it, clip_box);\r\n+      }\r\n       box_word->InsertBox(i, blob_box);\r\n     }\r\n-    // Fix empty boxes. If a very joined blob sits over multiple characters,\r\n-    // then we will have some empty boxes from using the middle, so look for\r\n-    // overlaps.\r\n-    for (int i = 0; i < box_word->length(); ++i) {\r\n-      TBOX box = box_word->BlobBox(i);\r\n-      if (box.null_box()) {\r\n-        // Nothing has its middle in the bounds of this blob, so use anything\r\n-        // that overlaps.\r\n-        for (dest_it.mark_cycle_pt(); !dest_it.cycled_list();\r\n-             dest_it.forward()) {\r\n-          TBOX blob_box = dest_it.data()->bounding_box();\r\n-          if (blob_box.left() < blob_ends[i] &&\r\n-              (i == 0 || blob_box.right() >= blob_ends[i - 1])) {\r\n-            if (i > 0 && blob_box.left() < blob_ends[i - 1])\r\n-              blob_box.set_left(blob_ends[i - 1]);\r\n-            if (blob_box.right() > blob_ends[i])\r\n-              blob_box.set_right(blob_ends[i]);\r\n-            box_word->ChangeBox(i, blob_box);\r\n-            break;\r\n-          }\r\n-        }\r\n-      }\r\n-    }\r\n     delete word_w->box_word;\r\n     word_w->box_word = box_word;\r\n     if (!input_word->combination) {\r\n@@ -1545,6 +1592,7 @@\r\n       }\r\n     }\r\n     ASSERT_HOST(!word_res_it.cycled_list());\r\n+    wr_it_of_next_word = word_res_it;\r\n     word_res_it.forward();\r\n   } else {\r\n     // word_res_it is OK, but reset word_res and prev_word_res if needed.\r\n@@ -1582,6 +1630,7 @@\r\n   block_res = next_block_res;\r\n   row_res = next_row_res;\r\n   word_res = next_word_res;\r\n+  wr_it_of_current_word = wr_it_of_next_word;\r\n   next_block_res = NULL;\r\n   next_row_res = NULL;\r\n   next_word_res = NULL;\r\n@@ -1610,6 +1659,7 @@\r\n         next_block_res = block_res_it.data();\r\n         next_row_res = row_res_it.data();\r\n         next_word_res = word_res_it.data();\r\n+        wr_it_of_next_word = word_res_it;\r\n         word_res_it.forward();\r\n         goto foundword;\r\n       }\r\n==== tesseract/ccstruct/pageres.h#60 - tesseract/ccstruct/pageres.h#61 ====\r\n# action=edit type=text\r\n--- tesseract/ccstruct/pageres.h\t2016-11-07 07:44:03.000000000 -0800\r\n+++ tesseract/ccstruct/pageres.h\t2018-11-09 10:28:33.000000000 -0800\r\n@@ -772,5 +772,9 @@\r\n   BLOCK_RES_IT block_res_it;   // iterators\r\n   ROW_RES_IT row_res_it;\r\n   WERD_RES_IT word_res_it;\r\n+  // Iterators used to get the state of word_res_it for the current word.\r\n+  // Since word_res_it is 2 words further on, this is otherwise hard to do.\r\n+  WERD_RES_IT wr_it_of_current_word;\r\n+  WERD_RES_IT wr_it_of_next_word;\r\n };\r\n #endif\r\n```"}}
{"actor":{"display_login":"jbreiden"}}