{"action":"created","comment":{"url":"https://api.github.com/repos/ropensci/roweb2/pulls/comments/209421545","pull_request_review_id":145441385,"id":209421545,"node_id":"MDI0OlB1bGxSZXF1ZXN0UmV2aWV3Q29tbWVudDIwOTQyMTU0NQ==","diff_hunk":"@@ -0,0 +1,411 @@\n+---\n+title: \"What's this bird? Classify old natural history drawings with R\"\n+slug: birds-ocr\n+authors:\n+  - name: Maëlle Salmon\n+    url: https://masalmon.eu/\n+date: 2018-08-28\n+preface: The blog post series corresponds to the material for a talk Maëlle will give at the [Animal Movement Analysis summer school in Radolfzell, Germany on September the 12th](http://animove.org/animove-2019-evening-keynotes/), in a Max Plant Institute of Ornithology.\n+tags:\n+- magick\n+- tesseract\n+- cld2\n+- cld3\n+- taxize\n+- birder\n+output: \n+  md_document:\n+    variant: markdown_github\n+    preserve_yaml: true\n+---\n+\n+In this new post, we’re taking a break from modern birding data in [our\n+birder’s series](https://ropensci.org/tags/birder)… let’s explore\n+gorgeous drawings from a natural history collection! Armed with\n+rOpenSci’s packages binding powerful C++ libraries and open taxonomy\n+data, how much information can we automatically extract from images?\n+Maybe not much, but we’ll at least have explored image manipulation,\n+optical character recognition, language detection, taxonomic name\n+resolution with rOpenSci’s packages.\n+\n+### Free natural history images and appropriate R tooling!\n+\n+A long time ago I had bookmarked the [Flickr account of the Biodiversity\n+Heritage Library](https://www.flickr.com/people/biodivlibrary/). So many\n+beautiful images of biodiversity, moreover free to use! In particular, I\n+[downloaded all pictures from one of the Birds of Australia\n+albums](https://www.flickr.com/photos/biodivlibrary/sets/72157694191194992).\n+\n+I wanted to try to extract the bird names on images using packages of\n+Jeroen Ooms’, rOpenSci’s post-doc hacker & C(++)-bindings wizard. In\n+particular, I wanted to try extract names of\n+[`magick`](https://github.com/ropensci/magick) for image manipulation,\n+[`tesseract`](https://github.com/ropensci/tesseract) for optical\n+character recognition,\n+[`cld2`](https://github.com/ropensci/cld2)/[`cld3`](https://github.com/ropensci/cld3)\n+for language detection… Quite the armory! We’ll also sprinkle some\n+taxonomy magic by Scott Chamberlain, one of rOpenSci’s co-founders, to\n+resolve the names extracted.\n+\n+### OCR-bird naming workflow, piece by piece\n+\n+In this section, we explain the different elements of our R workflow:\n+preparing images, extracting text, resolving taxonomic names.\n+\n+#### Image preparation\n+\n+I saved the pictures locally in a “birds” folder. Yes, I click-buttoned\n+instead of using [the Flickr API](https://www.flickr.com/services/api/)\n+for which e.g. Jim Hester wrote a [minimal R\n+wrapper](https://github.com/jimhester/flickrr)… I don’t do *everything*\n+with R scripts (yet).\n+\n+``` r\n+library(\"magrittr\")\n+filenames <- fs::dir_ls(\"birds\")\n+```\n+\n+Each image shows a bird and its name. Images are either landscape- or\n+portrait- oriented, but for the sake of simplicity, I’ll act as if they\n+were all portrait- oriented. *What I’d do if I weren’t after simplicity\n+but still as lazy would be to duplicate images rotated\n+(`magick::image_rotate`) in all possible directions, applying the\n+workflow, hoping to get good data from one of the rotated versions.*\n+\n+``` r\n+magick::image_read(filenames[1])\n+```\n+\n+<img src=\"/img/blog-images/2018-08-28-birds-ocr/unnamed-chunk-2-1.png\" width=\"1376\" />\n+\n+From that image, I wanted to extract the name indicated below the bird.\n+To maximize the efficiency of Optical Character Recognition, we shall\n+first prepare the image, since the [accuracy of OCR depends on the\n+quality of the input\n+image](https://cran.r-project.org/web/packages/tesseract/vignettes/intro.html#preprocessing_with_magick)\n+which we can influence a bit. This part could be tweaked even more, and\n+in real life examples it’ll be worth spending time trying different\n+`magick` functions and parameter values. Since I have in mind the case\n+of a bunch of images to be batch-processed, nothing is done by hand.\n+\n+``` r\n+crop_bird <- function(filename){\n+  image <- magick::image_read(filename)\n+  \n+  height <- magick::image_info(image)$height\n+  \n+  # crop the top of the image\n+  image <- magick::image_crop(image, \n+                     paste0(\"+0+\",round(0.75*height))) %>%\n+    # convert the image to black and white\n+    magick::image_convert(type = \"grayscale\") %>%\n+    # increase brightness\n+    magick::image_modulate(brightness = 120) %>%\n+    magick::image_enhance() %>%\n+    magick::image_median() %>%\n+    magick::image_contrast() \n+  \n+  # we'll need the filename later\n+  attr(image, \"filename\") <- filename\n+  \n+  return(image)\n+}\n+\n+crop_bird(filenames[1])\n+```\n+\n+<img src=\"/img/blog-images/2018-08-28-birds-ocr/unnamed-chunk-3-1.png\" width=\"1376\" />\n+\n+It does look cleaner now!\n+","path":"content/blog/2018-08-28-birds-ocr.md","position":null,"original_position":120,"commit_id":"f0edf454c8651244c9b751f2fdf108e142a5d417","original_commit_id":"52ad0b022a8388aebe6042080abefb93c9f12ca2","user":{"login":"maelle","id":8360597,"node_id":"MDQ6VXNlcjgzNjA1OTc=","avatar_url":"https://avatars0.githubusercontent.com/u/8360597?v=4","gravatar_id":"","url":"https://api.github.com/users/maelle","html_url":"https://github.com/maelle","followers_url":"https://api.github.com/users/maelle/followers","following_url":"https://api.github.com/users/maelle/following{/other_user}","gists_url":"https://api.github.com/users/maelle/gists{/gist_id}","starred_url":"https://api.github.com/users/maelle/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/maelle/subscriptions","organizations_url":"https://api.github.com/users/maelle/orgs","repos_url":"https://api.github.com/users/maelle/repos","events_url":"https://api.github.com/users/maelle/events{/privacy}","received_events_url":"https://api.github.com/users/maelle/received_events","type":"User","site_admin":false},"body":"I tried `<br>` and before that I had tried comparing them to working headers, no luck. ","created_at":"2018-08-11T09:05:28Z","updated_at":"2018-08-11T09:05:28Z","html_url":"https://github.com/ropensci/roweb2/pull/244#discussion_r209421545","pull_request_url":"https://api.github.com/repos/ropensci/roweb2/pulls/244","author_association":"MEMBER","_links":{"self":{"href":"https://api.github.com/repos/ropensci/roweb2/pulls/comments/209421545"},"html":{"href":"https://github.com/ropensci/roweb2/pull/244#discussion_r209421545"},"pull_request":{"href":"https://api.github.com/repos/ropensci/roweb2/pulls/244"}},"in_reply_to_id":209386449},"pull_request":{"url":"https://api.github.com/repos/ropensci/roweb2/pulls/244","id":206992262,"node_id":"MDExOlB1bGxSZXF1ZXN0MjA2OTkyMjYy","html_url":"https://github.com/ropensci/roweb2/pull/244","diff_url":"https://github.com/ropensci/roweb2/pull/244.diff","patch_url":"https://github.com/ropensci/roweb2/pull/244.patch","issue_url":"https://api.github.com/repos/ropensci/roweb2/issues/244","number":244,"state":"open","locked":false,"title":"3d post of the birder's guide series","user":{"login":"maelle","id":8360597,"node_id":"MDQ6VXNlcjgzNjA1OTc=","avatar_url":"https://avatars0.githubusercontent.com/u/8360597?v=4","gravatar_id":"","url":"https://api.github.com/users/maelle","html_url":"https://github.com/maelle","followers_url":"https://api.github.com/users/maelle/followers","following_url":"https://api.github.com/users/maelle/following{/other_user}","gists_url":"https://api.github.com/users/maelle/gists{/gist_id}","starred_url":"https://api.github.com/users/maelle/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/maelle/subscriptions","organizations_url":"https://api.github.com/users/maelle/orgs","repos_url":"https://api.github.com/users/maelle/repos","events_url":"https://api.github.com/users/maelle/events{/privacy}","received_events_url":"https://api.github.com/users/maelle/received_events","type":"User","site_admin":false},"body":"","created_at":"2018-08-08T12:33:54Z","updated_at":"2018-08-11T09:05:28Z","closed_at":null,"merged_at":null,"merge_commit_sha":"ac95cff063d1a4a9e62529764d77d13a24722686","assignee":null,"assignees":[],"requested_reviewers":[{"login":"jeroen","id":216319,"node_id":"MDQ6VXNlcjIxNjMxOQ==","avatar_url":"https://avatars3.githubusercontent.com/u/216319?v=4","gravatar_id":"","url":"https://api.github.com/users/jeroen","html_url":"https://github.com/jeroen","followers_url":"https://api.github.com/users/jeroen/followers","following_url":"https://api.github.com/users/jeroen/following{/other_user}","gists_url":"https://api.github.com/users/jeroen/gists{/gist_id}","starred_url":"https://api.github.com/users/jeroen/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jeroen/subscriptions","organizations_url":"https://api.github.com/users/jeroen/orgs","repos_url":"https://api.github.com/users/jeroen/repos","events_url":"https://api.github.com/users/jeroen/events{/privacy}","received_events_url":"https://api.github.com/users/jeroen/received_events","type":"User","site_admin":false}],"requested_teams":[],"labels":[],"milestone":null,"commits_url":"https://api.github.com/repos/ropensci/roweb2/pulls/244/commits","review_comments_url":"https://api.github.com/repos/ropensci/roweb2/pulls/244/comments","review_comment_url":"https://api.github.com/repos/ropensci/roweb2/pulls/comments{/number}","comments_url":"https://api.github.com/repos/ropensci/roweb2/issues/244/comments","statuses_url":"https://api.github.com/repos/ropensci/roweb2/statuses/f0edf454c8651244c9b751f2fdf108e142a5d417","head":{"label":"maelle:birds-ocr","ref":"birds-ocr","sha":"f0edf454c8651244c9b751f2fdf108e142a5d417","user":{"login":"maelle","id":8360597,"node_id":"MDQ6VXNlcjgzNjA1OTc=","avatar_url":"https://avatars0.githubusercontent.com/u/8360597?v=4","gravatar_id":"","url":"https://api.github.com/users/maelle","html_url":"https://github.com/maelle","followers_url":"https://api.github.com/users/maelle/followers","following_url":"https://api.github.com/users/maelle/following{/other_user}","gists_url":"https://api.github.com/users/maelle/gists{/gist_id}","starred_url":"https://api.github.com/users/maelle/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/maelle/subscriptions","organizations_url":"https://api.github.com/users/maelle/orgs","repos_url":"https://api.github.com/users/maelle/repos","events_url":"https://api.github.com/users/maelle/events{/privacy}","received_events_url":"https://api.github.com/users/maelle/received_events","type":"User","site_admin":false},"repo":{"id":142394312,"node_id":"MDEwOlJlcG9zaXRvcnkxNDIzOTQzMTI=","name":"roweb2","full_name":"maelle/roweb2","owner":{"login":"maelle","id":8360597,"node_id":"MDQ6VXNlcjgzNjA1OTc=","avatar_url":"https://avatars0.githubusercontent.com/u/8360597?v=4","gravatar_id":"","url":"https://api.github.com/users/maelle","html_url":"https://github.com/maelle","followers_url":"https://api.github.com/users/maelle/followers","following_url":"https://api.github.com/users/maelle/following{/other_user}","gists_url":"https://api.github.com/users/maelle/gists{/gist_id}","starred_url":"https://api.github.com/users/maelle/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/maelle/subscriptions","organizations_url":"https://api.github.com/users/maelle/orgs","repos_url":"https://api.github.com/users/maelle/repos","events_url":"https://api.github.com/users/maelle/events{/privacy}","received_events_url":"https://api.github.com/users/maelle/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/maelle/roweb2","description":"rOpenSci website","fork":true,"url":"https://api.github.com/repos/maelle/roweb2","forks_url":"https://api.github.com/repos/maelle/roweb2/forks","keys_url":"https://api.github.com/repos/maelle/roweb2/keys{/key_id}","collaborators_url":"https://api.github.com/repos/maelle/roweb2/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/maelle/roweb2/teams","hooks_url":"https://api.github.com/repos/maelle/roweb2/hooks","issue_events_url":"https://api.github.com/repos/maelle/roweb2/issues/events{/number}","events_url":"https://api.github.com/repos/maelle/roweb2/events","assignees_url":"https://api.github.com/repos/maelle/roweb2/assignees{/user}","branches_url":"https://api.github.com/repos/maelle/roweb2/branches{/branch}","tags_url":"https://api.github.com/repos/maelle/roweb2/tags","blobs_url":"https://api.github.com/repos/maelle/roweb2/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/maelle/roweb2/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/maelle/roweb2/git/refs{/sha}","trees_url":"https://api.github.com/repos/maelle/roweb2/git/trees{/sha}","statuses_url":"https://api.github.com/repos/maelle/roweb2/statuses/{sha}","languages_url":"https://api.github.com/repos/maelle/roweb2/languages","stargazers_url":"https://api.github.com/repos/maelle/roweb2/stargazers","contributors_url":"https://api.github.com/repos/maelle/roweb2/contributors","subscribers_url":"https://api.github.com/repos/maelle/roweb2/subscribers","subscription_url":"https://api.github.com/repos/maelle/roweb2/subscription","commits_url":"https://api.github.com/repos/maelle/roweb2/commits{/sha}","git_commits_url":"https://api.github.com/repos/maelle/roweb2/git/commits{/sha}","comments_url":"https://api.github.com/repos/maelle/roweb2/comments{/number}","issue_comment_url":"https://api.github.com/repos/maelle/roweb2/issues/comments{/number}","contents_url":"https://api.github.com/repos/maelle/roweb2/contents/{+path}","compare_url":"https://api.github.com/repos/maelle/roweb2/compare/{base}...{head}","merges_url":"https://api.github.com/repos/maelle/roweb2/merges","archive_url":"https://api.github.com/repos/maelle/roweb2/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/maelle/roweb2/downloads","issues_url":"https://api.github.com/repos/maelle/roweb2/issues{/number}","pulls_url":"https://api.github.com/repos/maelle/roweb2/pulls{/number}","milestones_url":"https://api.github.com/repos/maelle/roweb2/milestones{/number}","notifications_url":"https://api.github.com/repos/maelle/roweb2/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/maelle/roweb2/labels{/name}","releases_url":"https://api.github.com/repos/maelle/roweb2/releases{/id}","deployments_url":"https://api.github.com/repos/maelle/roweb2/deployments","created_at":"2018-07-26T05:46:41Z","updated_at":"2018-07-26T05:46:46Z","pushed_at":"2018-08-11T09:03:51Z","git_url":"git://github.com/maelle/roweb2.git","ssh_url":"git@github.com:maelle/roweb2.git","clone_url":"https://github.com/maelle/roweb2.git","svn_url":"https://github.com/maelle/roweb2","homepage":"https://ropensci.org","size":101255,"stargazers_count":0,"watchers_count":0,"language":"HTML","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"archived":false,"open_issues_count":0,"license":null,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"ropensci:master","ref":"master","sha":"44693295421551101b99e52a0a2468dfb3cacecf","user":{"login":"ropensci","id":1200269,"node_id":"MDEyOk9yZ2FuaXphdGlvbjEyMDAyNjk=","avatar_url":"https://avatars3.githubusercontent.com/u/1200269?v=4","gravatar_id":"","url":"https://api.github.com/users/ropensci","html_url":"https://github.com/ropensci","followers_url":"https://api.github.com/users/ropensci/followers","following_url":"https://api.github.com/users/ropensci/following{/other_user}","gists_url":"https://api.github.com/users/ropensci/gists{/gist_id}","starred_url":"https://api.github.com/users/ropensci/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ropensci/subscriptions","organizations_url":"https://api.github.com/users/ropensci/orgs","repos_url":"https://api.github.com/users/ropensci/repos","events_url":"https://api.github.com/users/ropensci/events{/privacy}","received_events_url":"https://api.github.com/users/ropensci/received_events","type":"Organization","site_admin":false},"repo":{"id":95706744,"node_id":"MDEwOlJlcG9zaXRvcnk5NTcwNjc0NA==","name":"roweb2","full_name":"ropensci/roweb2","owner":{"login":"ropensci","id":1200269,"node_id":"MDEyOk9yZ2FuaXphdGlvbjEyMDAyNjk=","avatar_url":"https://avatars3.githubusercontent.com/u/1200269?v=4","gravatar_id":"","url":"https://api.github.com/users/ropensci","html_url":"https://github.com/ropensci","followers_url":"https://api.github.com/users/ropensci/followers","following_url":"https://api.github.com/users/ropensci/following{/other_user}","gists_url":"https://api.github.com/users/ropensci/gists{/gist_id}","starred_url":"https://api.github.com/users/ropensci/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ropensci/subscriptions","organizations_url":"https://api.github.com/users/ropensci/orgs","repos_url":"https://api.github.com/users/ropensci/repos","events_url":"https://api.github.com/users/ropensci/events{/privacy}","received_events_url":"https://api.github.com/users/ropensci/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/ropensci/roweb2","description":"rOpenSci website","fork":false,"url":"https://api.github.com/repos/ropensci/roweb2","forks_url":"https://api.github.com/repos/ropensci/roweb2/forks","keys_url":"https://api.github.com/repos/ropensci/roweb2/keys{/key_id}","collaborators_url":"https://api.github.com/repos/ropensci/roweb2/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/ropensci/roweb2/teams","hooks_url":"https://api.github.com/repos/ropensci/roweb2/hooks","issue_events_url":"https://api.github.com/repos/ropensci/roweb2/issues/events{/number}","events_url":"https://api.github.com/repos/ropensci/roweb2/events","assignees_url":"https://api.github.com/repos/ropensci/roweb2/assignees{/user}","branches_url":"https://api.github.com/repos/ropensci/roweb2/branches{/branch}","tags_url":"https://api.github.com/repos/ropensci/roweb2/tags","blobs_url":"https://api.github.com/repos/ropensci/roweb2/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/ropensci/roweb2/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/ropensci/roweb2/git/refs{/sha}","trees_url":"https://api.github.com/repos/ropensci/roweb2/git/trees{/sha}","statuses_url":"https://api.github.com/repos/ropensci/roweb2/statuses/{sha}","languages_url":"https://api.github.com/repos/ropensci/roweb2/languages","stargazers_url":"https://api.github.com/repos/ropensci/roweb2/stargazers","contributors_url":"https://api.github.com/repos/ropensci/roweb2/contributors","subscribers_url":"https://api.github.com/repos/ropensci/roweb2/subscribers","subscription_url":"https://api.github.com/repos/ropensci/roweb2/subscription","commits_url":"https://api.github.com/repos/ropensci/roweb2/commits{/sha}","git_commits_url":"https://api.github.com/repos/ropensci/roweb2/git/commits{/sha}","comments_url":"https://api.github.com/repos/ropensci/roweb2/comments{/number}","issue_comment_url":"https://api.github.com/repos/ropensci/roweb2/issues/comments{/number}","contents_url":"https://api.github.com/repos/ropensci/roweb2/contents/{+path}","compare_url":"https://api.github.com/repos/ropensci/roweb2/compare/{base}...{head}","merges_url":"https://api.github.com/repos/ropensci/roweb2/merges","archive_url":"https://api.github.com/repos/ropensci/roweb2/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/ropensci/roweb2/downloads","issues_url":"https://api.github.com/repos/ropensci/roweb2/issues{/number}","pulls_url":"https://api.github.com/repos/ropensci/roweb2/pulls{/number}","milestones_url":"https://api.github.com/repos/ropensci/roweb2/milestones{/number}","notifications_url":"https://api.github.com/repos/ropensci/roweb2/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/ropensci/roweb2/labels{/name}","releases_url":"https://api.github.com/repos/ropensci/roweb2/releases{/id}","deployments_url":"https://api.github.com/repos/ropensci/roweb2/deployments","created_at":"2017-06-28T20:00:15Z","updated_at":"2018-08-10T19:39:37Z","pushed_at":"2018-08-11T09:03:52Z","git_url":"git://github.com/ropensci/roweb2.git","ssh_url":"git@github.com:ropensci/roweb2.git","clone_url":"https://github.com/ropensci/roweb2.git","svn_url":"https://github.com/ropensci/roweb2","homepage":"https://ropensci.org","size":96322,"stargazers_count":6,"watchers_count":6,"language":"HTML","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":38,"mirror_url":null,"archived":false,"open_issues_count":43,"license":null,"forks":38,"open_issues":43,"watchers":6,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/ropensci/roweb2/pulls/244"},"html":{"href":"https://github.com/ropensci/roweb2/pull/244"},"issue":{"href":"https://api.github.com/repos/ropensci/roweb2/issues/244"},"comments":{"href":"https://api.github.com/repos/ropensci/roweb2/issues/244/comments"},"review_comments":{"href":"https://api.github.com/repos/ropensci/roweb2/pulls/244/comments"},"review_comment":{"href":"https://api.github.com/repos/ropensci/roweb2/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/ropensci/roweb2/pulls/244/commits"},"statuses":{"href":"https://api.github.com/repos/ropensci/roweb2/statuses/f0edf454c8651244c9b751f2fdf108e142a5d417"}},"author_association":"MEMBER"}}
{"actor":{"display_login":"maelle"}}