{"action":"created","comment":{"url":"https://api.github.com/repos/s-rah/onionscan/pulls/comments/81688869","pull_request_review_id":2650314,"id":81688869,"diff_hunk":"@@ -2,45 +2,40 @@ package deanonymization\n \n import (\n \t\"bytes\"\n+\t\"github.com/rwcarlsen/goexif/exif\"\n+\t\"github.com/rwcarlsen/goexif/tiff\"\n \t\"github.com/s-rah/onionscan/config\"\n \t\"github.com/s-rah/onionscan/report\"\n-\t\"github.com/xiam/exif\"\n-\t\"io\"\n \t\"net/url\"\n \t\"strings\"\n )\n \n-func CheckExif(osreport *report.OnionScanReport, anonreport *report.AnonymityReport, osc *config.OnionScanConfig) {\n+type ExifWalker struct {\n+\tosreport   *report.OnionScanReport\n+\tanonreport *report.AnonymityReport\n+\tosc        *config.OnionScanConfig\n+}\n+\n+func (w *ExifWalker) Walk(name exif.FieldName, val *tiff.Tag) error {\n+\tw.anonreport.AddExifTag(string(name), val.String())\n+\treturn nil\n+}\n \n+func CheckExif(osreport *report.OnionScanReport, anonreport *report.AnonymityReport, osc *config.OnionScanConfig) {\n \tfor _, id := range osreport.Crawls {\n \t\tcrawlRecord, _ := osc.Database.GetCrawlRecord(id)\n \n \t\tif crawlRecord.Page.Status == 200 && strings.Contains(crawlRecord.Page.Headers.Get(\"Content-Type\"), \"image/jpeg\") {\n-\t\t\treader := exif.New()\n-\t\t\t_, err := io.Copy(reader, bytes.NewReader(crawlRecord.Page.Raw))\n-\n-\t\t\t// exif.FoundExifInData is a signal that the EXIF parser has all it needs,\n-\t\t\t// it doesn't need to be given the whole image.\n-\t\t\tif err != nil && err != exif.ErrFoundExifInData {\n-\t\t\t\t// We don't care if we fail\n-\t\t\t\treturn\n-\t\t\t}\n-\n-\t\t\terr = reader.Parse()\n+\t\t\texifdata, err := exif.Decode(bytes.NewReader(crawlRecord.Page.Raw))\n \n \t\t\tif err != nil {\n \t\t\t\t// We don't care if we fail\n \t\t\t\treturn","path":"deanonymization/check_exif.go","position":46,"original_position":46,"commit_id":"0efe1118a8c6fcd9ef835e8a1ffe8f9eeca673c9","original_commit_id":"0efe1118a8c6fcd9ef835e8a1ffe8f9eeca673c9","user":{"login":"s-rah","id":106626,"avatar_url":"https://avatars.githubusercontent.com/u/106626?v=3","gravatar_id":"","url":"https://api.github.com/users/s-rah","html_url":"https://github.com/s-rah","followers_url":"https://api.github.com/users/s-rah/followers","following_url":"https://api.github.com/users/s-rah/following{/other_user}","gists_url":"https://api.github.com/users/s-rah/gists{/gist_id}","starred_url":"https://api.github.com/users/s-rah/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/s-rah/subscriptions","organizations_url":"https://api.github.com/users/s-rah/orgs","repos_url":"https://api.github.com/users/s-rah/repos","events_url":"https://api.github.com/users/s-rah/events{/privacy}","received_events_url":"https://api.github.com/users/s-rah/received_events","type":"User","site_admin":false},"body":"continue, not return, we can continue evaluating other images - this is actually a bug in the current implementation too.","created_at":"2016-10-04T05:20:20Z","updated_at":"2016-10-04T05:21:38Z","html_url":"https://github.com/s-rah/onionscan/pull/68#discussion_r81688869","pull_request_url":"https://api.github.com/repos/s-rah/onionscan/pulls/68","_links":{"self":{"href":"https://api.github.com/repos/s-rah/onionscan/pulls/comments/81688869"},"html":{"href":"https://github.com/s-rah/onionscan/pull/68#discussion_r81688869"},"pull_request":{"href":"https://api.github.com/repos/s-rah/onionscan/pulls/68"}}},"pull_request":{"url":"https://api.github.com/repos/s-rah/onionscan/pulls/68","id":87634228,"html_url":"https://github.com/s-rah/onionscan/pull/68","diff_url":"https://github.com/s-rah/onionscan/pull/68.diff","patch_url":"https://github.com/s-rah/onionscan/pull/68.patch","issue_url":"https://api.github.com/repos/s-rah/onionscan/issues/68","number":68,"state":"open","locked":false,"title":"Switch to native-go EXIF library","user":{"login":"laanwj","id":126646,"avatar_url":"https://avatars.githubusercontent.com/u/126646?v=3","gravatar_id":"","url":"https://api.github.com/users/laanwj","html_url":"https://github.com/laanwj","followers_url":"https://api.github.com/users/laanwj/followers","following_url":"https://api.github.com/users/laanwj/following{/other_user}","gists_url":"https://api.github.com/users/laanwj/gists{/gist_id}","starred_url":"https://api.github.com/users/laanwj/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/laanwj/subscriptions","organizations_url":"https://api.github.com/users/laanwj/orgs","repos_url":"https://api.github.com/users/laanwj/repos","events_url":"https://api.github.com/users/laanwj/events{/privacy}","received_events_url":"https://api.github.com/users/laanwj/received_events","type":"User","site_admin":false},"body":"Use [goexif](https://github.com/rwcarlsen/goexif/) instead of libexif for EXIF parsing.\r\n\r\nBuilds on #67. Implements #63.\r\n\r\nThis is a bit of a proof of concept. The goexif page mentions that the software is still in alpha. As it looks like the upstream maintainer isn't very active maintaining it there are [a lot of forks](https://github.com/rwcarlsen/goexif/network) of this library that add extra functionality or maybe fix bugs. I haven't looked deeply into this.\r\n\r\nThe change affects the output. For example:\r\n- [4hwyik7xxwb6pbvb_1.txt](https://github.com/s-rah/onionscan/files/504962/4hwyik7xxwb6pbvb_1.txt)  Example with libexif\r\n- [4hwyik7xxwb6pbvb_2.txt](https://github.com/s-rah/onionscan/files/504963/4hwyik7xxwb6pbvb_2.txt) Example with goexif\r\n\r\nThere is the same information, but the conversion to strings happens differently. E.g. `F-Number` in the one is `FNumber` in the other. This may break current postprocessing. It would be possible to map them to the same names and value representation in an additional processing step, ofcourse.\r\n","created_at":"2016-10-02T13:39:31Z","updated_at":"2016-10-04T05:21:38Z","closed_at":null,"merged_at":null,"merge_commit_sha":"f6aee7c0ff2654a8779fb2daf4d37f388b6fd4a6","assignee":null,"assignees":[],"milestone":null,"commits_url":"https://api.github.com/repos/s-rah/onionscan/pulls/68/commits","review_comments_url":"https://api.github.com/repos/s-rah/onionscan/pulls/68/comments","review_comment_url":"https://api.github.com/repos/s-rah/onionscan/pulls/comments{/number}","comments_url":"https://api.github.com/repos/s-rah/onionscan/issues/68/comments","statuses_url":"https://api.github.com/repos/s-rah/onionscan/statuses/0efe1118a8c6fcd9ef835e8a1ffe8f9eeca673c9","head":{"label":"laanwj:2016_10_newexiflib","ref":"2016_10_newexiflib","sha":"0efe1118a8c6fcd9ef835e8a1ffe8f9eeca673c9","user":{"login":"laanwj","id":126646,"avatar_url":"https://avatars.githubusercontent.com/u/126646?v=3","gravatar_id":"","url":"https://api.github.com/users/laanwj","html_url":"https://github.com/laanwj","followers_url":"https://api.github.com/users/laanwj/followers","following_url":"https://api.github.com/users/laanwj/following{/other_user}","gists_url":"https://api.github.com/users/laanwj/gists{/gist_id}","starred_url":"https://api.github.com/users/laanwj/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/laanwj/subscriptions","organizations_url":"https://api.github.com/users/laanwj/orgs","repos_url":"https://api.github.com/users/laanwj/repos","events_url":"https://api.github.com/users/laanwj/events{/privacy}","received_events_url":"https://api.github.com/users/laanwj/received_events","type":"User","site_admin":false},"repo":{"id":69292130,"name":"onionscan","full_name":"laanwj/onionscan","owner":{"login":"laanwj","id":126646,"avatar_url":"https://avatars.githubusercontent.com/u/126646?v=3","gravatar_id":"","url":"https://api.github.com/users/laanwj","html_url":"https://github.com/laanwj","followers_url":"https://api.github.com/users/laanwj/followers","following_url":"https://api.github.com/users/laanwj/following{/other_user}","gists_url":"https://api.github.com/users/laanwj/gists{/gist_id}","starred_url":"https://api.github.com/users/laanwj/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/laanwj/subscriptions","organizations_url":"https://api.github.com/users/laanwj/orgs","repos_url":"https://api.github.com/users/laanwj/repos","events_url":"https://api.github.com/users/laanwj/events{/privacy}","received_events_url":"https://api.github.com/users/laanwj/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/laanwj/onionscan","description":"Scan Onion Services for Security Issues","fork":true,"url":"https://api.github.com/repos/laanwj/onionscan","forks_url":"https://api.github.com/repos/laanwj/onionscan/forks","keys_url":"https://api.github.com/repos/laanwj/onionscan/keys{/key_id}","collaborators_url":"https://api.github.com/repos/laanwj/onionscan/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/laanwj/onionscan/teams","hooks_url":"https://api.github.com/repos/laanwj/onionscan/hooks","issue_events_url":"https://api.github.com/repos/laanwj/onionscan/issues/events{/number}","events_url":"https://api.github.com/repos/laanwj/onionscan/events","assignees_url":"https://api.github.com/repos/laanwj/onionscan/assignees{/user}","branches_url":"https://api.github.com/repos/laanwj/onionscan/branches{/branch}","tags_url":"https://api.github.com/repos/laanwj/onionscan/tags","blobs_url":"https://api.github.com/repos/laanwj/onionscan/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/laanwj/onionscan/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/laanwj/onionscan/git/refs{/sha}","trees_url":"https://api.github.com/repos/laanwj/onionscan/git/trees{/sha}","statuses_url":"https://api.github.com/repos/laanwj/onionscan/statuses/{sha}","languages_url":"https://api.github.com/repos/laanwj/onionscan/languages","stargazers_url":"https://api.github.com/repos/laanwj/onionscan/stargazers","contributors_url":"https://api.github.com/repos/laanwj/onionscan/contributors","subscribers_url":"https://api.github.com/repos/laanwj/onionscan/subscribers","subscription_url":"https://api.github.com/repos/laanwj/onionscan/subscription","commits_url":"https://api.github.com/repos/laanwj/onionscan/commits{/sha}","git_commits_url":"https://api.github.com/repos/laanwj/onionscan/git/commits{/sha}","comments_url":"https://api.github.com/repos/laanwj/onionscan/comments{/number}","issue_comment_url":"https://api.github.com/repos/laanwj/onionscan/issues/comments{/number}","contents_url":"https://api.github.com/repos/laanwj/onionscan/contents/{+path}","compare_url":"https://api.github.com/repos/laanwj/onionscan/compare/{base}...{head}","merges_url":"https://api.github.com/repos/laanwj/onionscan/merges","archive_url":"https://api.github.com/repos/laanwj/onionscan/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/laanwj/onionscan/downloads","issues_url":"https://api.github.com/repos/laanwj/onionscan/issues{/number}","pulls_url":"https://api.github.com/repos/laanwj/onionscan/pulls{/number}","milestones_url":"https://api.github.com/repos/laanwj/onionscan/milestones{/number}","notifications_url":"https://api.github.com/repos/laanwj/onionscan/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/laanwj/onionscan/labels{/name}","releases_url":"https://api.github.com/repos/laanwj/onionscan/releases{/id}","deployments_url":"https://api.github.com/repos/laanwj/onionscan/deployments","created_at":"2016-09-26T20:56:38Z","updated_at":"2016-09-26T20:56:39Z","pushed_at":"2016-10-03T17:28:10Z","git_url":"git://github.com/laanwj/onionscan.git","ssh_url":"git@github.com:laanwj/onionscan.git","clone_url":"https://github.com/laanwj/onionscan.git","svn_url":"https://github.com/laanwj/onionscan","homepage":"","size":146,"stargazers_count":0,"watchers_count":0,"language":"Go","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"s-rah:onionscan-0.2","ref":"onionscan-0.2","sha":"c735333dedf7126b71e7833386aecd8db69ab7af","user":{"login":"s-rah","id":106626,"avatar_url":"https://avatars.githubusercontent.com/u/106626?v=3","gravatar_id":"","url":"https://api.github.com/users/s-rah","html_url":"https://github.com/s-rah","followers_url":"https://api.github.com/users/s-rah/followers","following_url":"https://api.github.com/users/s-rah/following{/other_user}","gists_url":"https://api.github.com/users/s-rah/gists{/gist_id}","starred_url":"https://api.github.com/users/s-rah/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/s-rah/subscriptions","organizations_url":"https://api.github.com/users/s-rah/orgs","repos_url":"https://api.github.com/users/s-rah/repos","events_url":"https://api.github.com/users/s-rah/events{/privacy}","received_events_url":"https://api.github.com/users/s-rah/received_events","type":"User","site_admin":false},"repo":{"id":55757553,"name":"onionscan","full_name":"s-rah/onionscan","owner":{"login":"s-rah","id":106626,"avatar_url":"https://avatars.githubusercontent.com/u/106626?v=3","gravatar_id":"","url":"https://api.github.com/users/s-rah","html_url":"https://github.com/s-rah","followers_url":"https://api.github.com/users/s-rah/followers","following_url":"https://api.github.com/users/s-rah/following{/other_user}","gists_url":"https://api.github.com/users/s-rah/gists{/gist_id}","starred_url":"https://api.github.com/users/s-rah/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/s-rah/subscriptions","organizations_url":"https://api.github.com/users/s-rah/orgs","repos_url":"https://api.github.com/users/s-rah/repos","events_url":"https://api.github.com/users/s-rah/events{/privacy}","received_events_url":"https://api.github.com/users/s-rah/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/s-rah/onionscan","description":"Scan Onion Services for Security Issues","fork":false,"url":"https://api.github.com/repos/s-rah/onionscan","forks_url":"https://api.github.com/repos/s-rah/onionscan/forks","keys_url":"https://api.github.com/repos/s-rah/onionscan/keys{/key_id}","collaborators_url":"https://api.github.com/repos/s-rah/onionscan/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/s-rah/onionscan/teams","hooks_url":"https://api.github.com/repos/s-rah/onionscan/hooks","issue_events_url":"https://api.github.com/repos/s-rah/onionscan/issues/events{/number}","events_url":"https://api.github.com/repos/s-rah/onionscan/events","assignees_url":"https://api.github.com/repos/s-rah/onionscan/assignees{/user}","branches_url":"https://api.github.com/repos/s-rah/onionscan/branches{/branch}","tags_url":"https://api.github.com/repos/s-rah/onionscan/tags","blobs_url":"https://api.github.com/repos/s-rah/onionscan/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/s-rah/onionscan/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/s-rah/onionscan/git/refs{/sha}","trees_url":"https://api.github.com/repos/s-rah/onionscan/git/trees{/sha}","statuses_url":"https://api.github.com/repos/s-rah/onionscan/statuses/{sha}","languages_url":"https://api.github.com/repos/s-rah/onionscan/languages","stargazers_url":"https://api.github.com/repos/s-rah/onionscan/stargazers","contributors_url":"https://api.github.com/repos/s-rah/onionscan/contributors","subscribers_url":"https://api.github.com/repos/s-rah/onionscan/subscribers","subscription_url":"https://api.github.com/repos/s-rah/onionscan/subscription","commits_url":"https://api.github.com/repos/s-rah/onionscan/commits{/sha}","git_commits_url":"https://api.github.com/repos/s-rah/onionscan/git/commits{/sha}","comments_url":"https://api.github.com/repos/s-rah/onionscan/comments{/number}","issue_comment_url":"https://api.github.com/repos/s-rah/onionscan/issues/comments{/number}","contents_url":"https://api.github.com/repos/s-rah/onionscan/contents/{+path}","compare_url":"https://api.github.com/repos/s-rah/onionscan/compare/{base}...{head}","merges_url":"https://api.github.com/repos/s-rah/onionscan/merges","archive_url":"https://api.github.com/repos/s-rah/onionscan/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/s-rah/onionscan/downloads","issues_url":"https://api.github.com/repos/s-rah/onionscan/issues{/number}","pulls_url":"https://api.github.com/repos/s-rah/onionscan/pulls{/number}","milestones_url":"https://api.github.com/repos/s-rah/onionscan/milestones{/number}","notifications_url":"https://api.github.com/repos/s-rah/onionscan/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/s-rah/onionscan/labels{/name}","releases_url":"https://api.github.com/repos/s-rah/onionscan/releases{/id}","deployments_url":"https://api.github.com/repos/s-rah/onionscan/deployments","created_at":"2016-04-08T06:58:16Z","updated_at":"2016-10-04T01:40:08Z","pushed_at":"2016-10-04T01:36:24Z","git_url":"git://github.com/s-rah/onionscan.git","ssh_url":"git@github.com:s-rah/onionscan.git","clone_url":"https://github.com/s-rah/onionscan.git","svn_url":"https://github.com/s-rah/onionscan","homepage":"","size":174,"stargazers_count":427,"watchers_count":427,"language":"Go","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":63,"mirror_url":null,"open_issues_count":22,"forks":63,"open_issues":22,"watchers":427,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/s-rah/onionscan/pulls/68"},"html":{"href":"https://github.com/s-rah/onionscan/pull/68"},"issue":{"href":"https://api.github.com/repos/s-rah/onionscan/issues/68"},"comments":{"href":"https://api.github.com/repos/s-rah/onionscan/issues/68/comments"},"review_comments":{"href":"https://api.github.com/repos/s-rah/onionscan/pulls/68/comments"},"review_comment":{"href":"https://api.github.com/repos/s-rah/onionscan/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/s-rah/onionscan/pulls/68/commits"},"statuses":{"href":"https://api.github.com/repos/s-rah/onionscan/statuses/0efe1118a8c6fcd9ef835e8a1ffe8f9eeca673c9"}}}}
{"actor":{"display_login":"s-rah"}}