{"action":"created","comment":{"url":"https://api.github.com/repos/broadinstitute/picard/pulls/comments/197512189","pull_request_review_id":44889699,"id":197512189,"node_id":"MDI0OlB1bGxSZXF1ZXN0UmV2aWV3Q29tbWVudDE5NzUxMjE4OQ==","diff_hunk":"@@ -311,5 +343,27 @@ private static void trackOpticalDuplicates(final List<? extends ReadEnds> list,\n         if (opticalDuplicates > 0) {\n             opticalDuplicatesByLibraryId.increment(list.get(0).getLibraryId(), opticalDuplicates);\n         }\n+        return opticalDuplicates;\n     }\n+\n+    private static void trackDuplicateCounts(final int listSize,\n+                                                   final int optDupCnt,\n+                                                    final Histogram<Double> duplicatesCountHist,\n+                                                    final Histogram<Double> nonOpticalDuplicatesCountHist,\n+                                                    final Histogram<Double> opticalDuplicatesCountHist) {\n+        duplicatesCountHist.increment(new Double(listSize));\n+        if ( (listSize - optDupCnt) > 0) {\n+            nonOpticalDuplicatesCountHist.increment(new Double(listSize - optDupCnt));\n+        }\n+        if (optDupCnt > 0) {\n+            opticalDuplicatesCountHist.increment(new Double(optDupCnt+1));\n+        }\n+    }\n+\n+    private static void addSingletonToCount(final Histogram<Double> duplicatesCountHist,\n+                                            final Histogram<Double> nonOpticalDuplicatesCountHist) {\n+        duplicatesCountHist.increment((double) 1);","path":"src/main/java/picard/sam/markduplicates/util/AbstractMarkDuplicatesCommandLineProgram.java","position":102,"original_position":102,"commit_id":"b80fe676e27de60f569b40f35858a174b3bf5f3b","original_commit_id":"b80fe676e27de60f569b40f35858a174b3bf5f3b","user":{"login":"fleharty","id":10714381,"node_id":"MDQ6VXNlcjEwNzE0Mzgx","avatar_url":"https://avatars3.githubusercontent.com/u/10714381?v=4","gravatar_id":"","url":"https://api.github.com/users/fleharty","html_url":"https://github.com/fleharty","followers_url":"https://api.github.com/users/fleharty/followers","following_url":"https://api.github.com/users/fleharty/following{/other_user}","gists_url":"https://api.github.com/users/fleharty/gists{/gist_id}","starred_url":"https://api.github.com/users/fleharty/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fleharty/subscriptions","organizations_url":"https://api.github.com/users/fleharty/orgs","repos_url":"https://api.github.com/users/fleharty/repos","events_url":"https://api.github.com/users/fleharty/events{/privacy}","received_events_url":"https://api.github.com/users/fleharty/received_events","type":"User","site_admin":false},"body":"use 1.0 rather than casting (double)","created_at":"2018-06-22T17:17:14Z","updated_at":"2018-06-22T17:52:18Z","html_url":"https://github.com/broadinstitute/picard/pull/569#discussion_r197512189","pull_request_url":"https://api.github.com/repos/broadinstitute/picard/pulls/569","author_association":"CONTRIBUTOR","_links":{"self":{"href":"https://api.github.com/repos/broadinstitute/picard/pulls/comments/197512189"},"html":{"href":"https://github.com/broadinstitute/picard/pull/569#discussion_r197512189"},"pull_request":{"href":"https://api.github.com/repos/broadinstitute/picard/pulls/569"}}},"pull_request":{"url":"https://api.github.com/repos/broadinstitute/picard/pulls/569","id":74340531,"node_id":"MDExOlB1bGxSZXF1ZXN0NzQzNDA1MzE=","html_url":"https://github.com/broadinstitute/picard/pull/569","diff_url":"https://github.com/broadinstitute/picard/pull/569.diff","patch_url":"https://github.com/broadinstitute/picard/pull/569.patch","issue_url":"https://api.github.com/repos/broadinstitute/picard/issues/569","number":569,"state":"open","locked":false,"title":"Histogram of duplicate set sizes to be reported in the MarkDuplicates metrics file","user":{"login":"lhogstrom","id":3091474,"node_id":"MDQ6VXNlcjMwOTE0NzQ=","avatar_url":"https://avatars0.githubusercontent.com/u/3091474?v=4","gravatar_id":"","url":"https://api.github.com/users/lhogstrom","html_url":"https://github.com/lhogstrom","followers_url":"https://api.github.com/users/lhogstrom/followers","following_url":"https://api.github.com/users/lhogstrom/following{/other_user}","gists_url":"https://api.github.com/users/lhogstrom/gists{/gist_id}","starred_url":"https://api.github.com/users/lhogstrom/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lhogstrom/subscriptions","organizations_url":"https://api.github.com/users/lhogstrom/orgs","repos_url":"https://api.github.com/users/lhogstrom/repos","events_url":"https://api.github.com/users/lhogstrom/events{/privacy}","received_events_url":"https://api.github.com/users/lhogstrom/received_events","type":"User","site_admin":false},"body":"This change tracks the size of read pairs sets that are found by MarkDuplicates to be mapped to the same location. The feature performs as follows:\n- only sets of mapped read pairs are counted in the histogram\n- a set size of 1 indicates a read pair was not a duplicate\n- set size of 2 indicates two read pairs were mapped to the same location\n- set size of 3 indicates three read pairs were mapped to the same location... and so on\n\nDesign choices where I'd welcome feedback:\n- optional flag to write the set size counts: I could add this if need be. \n- addSingletonToCount: this function directly counts each read pair that does not appear as duplicate. The overall count of these non-duplicate pairs could also be inferred from READ_PAIRS_EXAMINED - READ_PAIR_DUPLICATES. In one sense, I see it as preferable to count them directly, but I could also see the argument for inferring the count of non-duplicate entries to reduce overhead. \n- The structure of the output file is not ideal. The set size counts are printed on adjacent columns to the ROI metric. This follows the convention of 'addHistogram()' to a metrics file, but I see two problems doing this. First, I find this slightly confusing as the single 'BIN' column is used to represent two different things (set size and ROI multiple). Second, values of the duplicate set histogram will always take on integer values and that doesn't have to be the case for ROI sequencing multiples. \n\nHere are three options regarding this last issue 1) Leave the output as it appears in this pull request (ROI output and set sizes printed together) being careful to explain in the documentation that both are represented in the ## HISTOGRAM entry. 2) Change the metrics file to allow histograms to be printed on multiple lines (possible, but probably out of scope for this pull request). 3) Print the set size histograms to an entirely new file. \n","created_at":"2016-06-18T15:57:56Z","updated_at":"2018-06-22T17:52:18Z","closed_at":null,"merged_at":null,"merge_commit_sha":"d33d5f736c5f03c6e1adda6ec4bdb76f0a92e59e","assignee":{"login":"fleharty","id":10714381,"node_id":"MDQ6VXNlcjEwNzE0Mzgx","avatar_url":"https://avatars3.githubusercontent.com/u/10714381?v=4","gravatar_id":"","url":"https://api.github.com/users/fleharty","html_url":"https://github.com/fleharty","followers_url":"https://api.github.com/users/fleharty/followers","following_url":"https://api.github.com/users/fleharty/following{/other_user}","gists_url":"https://api.github.com/users/fleharty/gists{/gist_id}","starred_url":"https://api.github.com/users/fleharty/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fleharty/subscriptions","organizations_url":"https://api.github.com/users/fleharty/orgs","repos_url":"https://api.github.com/users/fleharty/repos","events_url":"https://api.github.com/users/fleharty/events{/privacy}","received_events_url":"https://api.github.com/users/fleharty/received_events","type":"User","site_admin":false},"assignees":[{"login":"fleharty","id":10714381,"node_id":"MDQ6VXNlcjEwNzE0Mzgx","avatar_url":"https://avatars3.githubusercontent.com/u/10714381?v=4","gravatar_id":"","url":"https://api.github.com/users/fleharty","html_url":"https://github.com/fleharty","followers_url":"https://api.github.com/users/fleharty/followers","following_url":"https://api.github.com/users/fleharty/following{/other_user}","gists_url":"https://api.github.com/users/fleharty/gists{/gist_id}","starred_url":"https://api.github.com/users/fleharty/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fleharty/subscriptions","organizations_url":"https://api.github.com/users/fleharty/orgs","repos_url":"https://api.github.com/users/fleharty/repos","events_url":"https://api.github.com/users/fleharty/events{/privacy}","received_events_url":"https://api.github.com/users/fleharty/received_events","type":"User","site_admin":false}],"requested_reviewers":[],"requested_teams":[],"labels":[],"milestone":null,"commits_url":"https://api.github.com/repos/broadinstitute/picard/pulls/569/commits","review_comments_url":"https://api.github.com/repos/broadinstitute/picard/pulls/569/comments","review_comment_url":"https://api.github.com/repos/broadinstitute/picard/pulls/comments{/number}","comments_url":"https://api.github.com/repos/broadinstitute/picard/issues/569/comments","statuses_url":"https://api.github.com/repos/broadinstitute/picard/statuses/b80fe676e27de60f569b40f35858a174b3bf5f3b","head":{"label":"broadinstitute:lh_duplicate_set_size_histogram","ref":"lh_duplicate_set_size_histogram","sha":"b80fe676e27de60f569b40f35858a174b3bf5f3b","user":{"login":"broadinstitute","id":393552,"node_id":"MDEyOk9yZ2FuaXphdGlvbjM5MzU1Mg==","avatar_url":"https://avatars0.githubusercontent.com/u/393552?v=4","gravatar_id":"","url":"https://api.github.com/users/broadinstitute","html_url":"https://github.com/broadinstitute","followers_url":"https://api.github.com/users/broadinstitute/followers","following_url":"https://api.github.com/users/broadinstitute/following{/other_user}","gists_url":"https://api.github.com/users/broadinstitute/gists{/gist_id}","starred_url":"https://api.github.com/users/broadinstitute/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/broadinstitute/subscriptions","organizations_url":"https://api.github.com/users/broadinstitute/orgs","repos_url":"https://api.github.com/users/broadinstitute/repos","events_url":"https://api.github.com/users/broadinstitute/events{/privacy}","received_events_url":"https://api.github.com/users/broadinstitute/received_events","type":"Organization","site_admin":false},"repo":{"id":18225913,"node_id":"MDEwOlJlcG9zaXRvcnkxODIyNTkxMw==","name":"picard","full_name":"broadinstitute/picard","owner":{"login":"broadinstitute","id":393552,"node_id":"MDEyOk9yZ2FuaXphdGlvbjM5MzU1Mg==","avatar_url":"https://avatars0.githubusercontent.com/u/393552?v=4","gravatar_id":"","url":"https://api.github.com/users/broadinstitute","html_url":"https://github.com/broadinstitute","followers_url":"https://api.github.com/users/broadinstitute/followers","following_url":"https://api.github.com/users/broadinstitute/following{/other_user}","gists_url":"https://api.github.com/users/broadinstitute/gists{/gist_id}","starred_url":"https://api.github.com/users/broadinstitute/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/broadinstitute/subscriptions","organizations_url":"https://api.github.com/users/broadinstitute/orgs","repos_url":"https://api.github.com/users/broadinstitute/repos","events_url":"https://api.github.com/users/broadinstitute/events{/privacy}","received_events_url":"https://api.github.com/users/broadinstitute/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/broadinstitute/picard","description":"A set of command line tools (in Java) for manipulating high-throughput sequencing (HTS) data and formats such as SAM/BAM/CRAM and VCF.","fork":false,"url":"https://api.github.com/repos/broadinstitute/picard","forks_url":"https://api.github.com/repos/broadinstitute/picard/forks","keys_url":"https://api.github.com/repos/broadinstitute/picard/keys{/key_id}","collaborators_url":"https://api.github.com/repos/broadinstitute/picard/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/broadinstitute/picard/teams","hooks_url":"https://api.github.com/repos/broadinstitute/picard/hooks","issue_events_url":"https://api.github.com/repos/broadinstitute/picard/issues/events{/number}","events_url":"https://api.github.com/repos/broadinstitute/picard/events","assignees_url":"https://api.github.com/repos/broadinstitute/picard/assignees{/user}","branches_url":"https://api.github.com/repos/broadinstitute/picard/branches{/branch}","tags_url":"https://api.github.com/repos/broadinstitute/picard/tags","blobs_url":"https://api.github.com/repos/broadinstitute/picard/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/broadinstitute/picard/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/broadinstitute/picard/git/refs{/sha}","trees_url":"https://api.github.com/repos/broadinstitute/picard/git/trees{/sha}","statuses_url":"https://api.github.com/repos/broadinstitute/picard/statuses/{sha}","languages_url":"https://api.github.com/repos/broadinstitute/picard/languages","stargazers_url":"https://api.github.com/repos/broadinstitute/picard/stargazers","contributors_url":"https://api.github.com/repos/broadinstitute/picard/contributors","subscribers_url":"https://api.github.com/repos/broadinstitute/picard/subscribers","subscription_url":"https://api.github.com/repos/broadinstitute/picard/subscription","commits_url":"https://api.github.com/repos/broadinstitute/picard/commits{/sha}","git_commits_url":"https://api.github.com/repos/broadinstitute/picard/git/commits{/sha}","comments_url":"https://api.github.com/repos/broadinstitute/picard/comments{/number}","issue_comment_url":"https://api.github.com/repos/broadinstitute/picard/issues/comments{/number}","contents_url":"https://api.github.com/repos/broadinstitute/picard/contents/{+path}","compare_url":"https://api.github.com/repos/broadinstitute/picard/compare/{base}...{head}","merges_url":"https://api.github.com/repos/broadinstitute/picard/merges","archive_url":"https://api.github.com/repos/broadinstitute/picard/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/broadinstitute/picard/downloads","issues_url":"https://api.github.com/repos/broadinstitute/picard/issues{/number}","pulls_url":"https://api.github.com/repos/broadinstitute/picard/pulls{/number}","milestones_url":"https://api.github.com/repos/broadinstitute/picard/milestones{/number}","notifications_url":"https://api.github.com/repos/broadinstitute/picard/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/broadinstitute/picard/labels{/name}","releases_url":"https://api.github.com/repos/broadinstitute/picard/releases{/id}","deployments_url":"https://api.github.com/repos/broadinstitute/picard/deployments","created_at":"2014-03-28T20:43:35Z","updated_at":"2018-06-21T16:29:41Z","pushed_at":"2018-06-22T15:47:12Z","git_url":"git://github.com/broadinstitute/picard.git","ssh_url":"git@github.com:broadinstitute/picard.git","clone_url":"https://github.com/broadinstitute/picard.git","svn_url":"https://github.com/broadinstitute/picard","homepage":"http://broadinstitute.github.io/picard/","size":148870,"stargazers_count":380,"watchers_count":380,"language":"Java","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":true,"forks_count":237,"mirror_url":null,"archived":false,"open_issues_count":138,"license":{"key":"mit","name":"MIT License","spdx_id":"MIT","url":"https://api.github.com/licenses/mit","node_id":"MDc6TGljZW5zZTEz"},"forks":237,"open_issues":138,"watchers":380,"default_branch":"master"}},"base":{"label":"broadinstitute:master","ref":"master","sha":"e4344e0a20c5909796fd48994577f69fef53771c","user":{"login":"broadinstitute","id":393552,"node_id":"MDEyOk9yZ2FuaXphdGlvbjM5MzU1Mg==","avatar_url":"https://avatars0.githubusercontent.com/u/393552?v=4","gravatar_id":"","url":"https://api.github.com/users/broadinstitute","html_url":"https://github.com/broadinstitute","followers_url":"https://api.github.com/users/broadinstitute/followers","following_url":"https://api.github.com/users/broadinstitute/following{/other_user}","gists_url":"https://api.github.com/users/broadinstitute/gists{/gist_id}","starred_url":"https://api.github.com/users/broadinstitute/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/broadinstitute/subscriptions","organizations_url":"https://api.github.com/users/broadinstitute/orgs","repos_url":"https://api.github.com/users/broadinstitute/repos","events_url":"https://api.github.com/users/broadinstitute/events{/privacy}","received_events_url":"https://api.github.com/users/broadinstitute/received_events","type":"Organization","site_admin":false},"repo":{"id":18225913,"node_id":"MDEwOlJlcG9zaXRvcnkxODIyNTkxMw==","name":"picard","full_name":"broadinstitute/picard","owner":{"login":"broadinstitute","id":393552,"node_id":"MDEyOk9yZ2FuaXphdGlvbjM5MzU1Mg==","avatar_url":"https://avatars0.githubusercontent.com/u/393552?v=4","gravatar_id":"","url":"https://api.github.com/users/broadinstitute","html_url":"https://github.com/broadinstitute","followers_url":"https://api.github.com/users/broadinstitute/followers","following_url":"https://api.github.com/users/broadinstitute/following{/other_user}","gists_url":"https://api.github.com/users/broadinstitute/gists{/gist_id}","starred_url":"https://api.github.com/users/broadinstitute/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/broadinstitute/subscriptions","organizations_url":"https://api.github.com/users/broadinstitute/orgs","repos_url":"https://api.github.com/users/broadinstitute/repos","events_url":"https://api.github.com/users/broadinstitute/events{/privacy}","received_events_url":"https://api.github.com/users/broadinstitute/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/broadinstitute/picard","description":"A set of command line tools (in Java) for manipulating high-throughput sequencing (HTS) data and formats such as SAM/BAM/CRAM and VCF.","fork":false,"url":"https://api.github.com/repos/broadinstitute/picard","forks_url":"https://api.github.com/repos/broadinstitute/picard/forks","keys_url":"https://api.github.com/repos/broadinstitute/picard/keys{/key_id}","collaborators_url":"https://api.github.com/repos/broadinstitute/picard/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/broadinstitute/picard/teams","hooks_url":"https://api.github.com/repos/broadinstitute/picard/hooks","issue_events_url":"https://api.github.com/repos/broadinstitute/picard/issues/events{/number}","events_url":"https://api.github.com/repos/broadinstitute/picard/events","assignees_url":"https://api.github.com/repos/broadinstitute/picard/assignees{/user}","branches_url":"https://api.github.com/repos/broadinstitute/picard/branches{/branch}","tags_url":"https://api.github.com/repos/broadinstitute/picard/tags","blobs_url":"https://api.github.com/repos/broadinstitute/picard/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/broadinstitute/picard/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/broadinstitute/picard/git/refs{/sha}","trees_url":"https://api.github.com/repos/broadinstitute/picard/git/trees{/sha}","statuses_url":"https://api.github.com/repos/broadinstitute/picard/statuses/{sha}","languages_url":"https://api.github.com/repos/broadinstitute/picard/languages","stargazers_url":"https://api.github.com/repos/broadinstitute/picard/stargazers","contributors_url":"https://api.github.com/repos/broadinstitute/picard/contributors","subscribers_url":"https://api.github.com/repos/broadinstitute/picard/subscribers","subscription_url":"https://api.github.com/repos/broadinstitute/picard/subscription","commits_url":"https://api.github.com/repos/broadinstitute/picard/commits{/sha}","git_commits_url":"https://api.github.com/repos/broadinstitute/picard/git/commits{/sha}","comments_url":"https://api.github.com/repos/broadinstitute/picard/comments{/number}","issue_comment_url":"https://api.github.com/repos/broadinstitute/picard/issues/comments{/number}","contents_url":"https://api.github.com/repos/broadinstitute/picard/contents/{+path}","compare_url":"https://api.github.com/repos/broadinstitute/picard/compare/{base}...{head}","merges_url":"https://api.github.com/repos/broadinstitute/picard/merges","archive_url":"https://api.github.com/repos/broadinstitute/picard/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/broadinstitute/picard/downloads","issues_url":"https://api.github.com/repos/broadinstitute/picard/issues{/number}","pulls_url":"https://api.github.com/repos/broadinstitute/picard/pulls{/number}","milestones_url":"https://api.github.com/repos/broadinstitute/picard/milestones{/number}","notifications_url":"https://api.github.com/repos/broadinstitute/picard/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/broadinstitute/picard/labels{/name}","releases_url":"https://api.github.com/repos/broadinstitute/picard/releases{/id}","deployments_url":"https://api.github.com/repos/broadinstitute/picard/deployments","created_at":"2014-03-28T20:43:35Z","updated_at":"2018-06-21T16:29:41Z","pushed_at":"2018-06-22T15:47:12Z","git_url":"git://github.com/broadinstitute/picard.git","ssh_url":"git@github.com:broadinstitute/picard.git","clone_url":"https://github.com/broadinstitute/picard.git","svn_url":"https://github.com/broadinstitute/picard","homepage":"http://broadinstitute.github.io/picard/","size":148870,"stargazers_count":380,"watchers_count":380,"language":"Java","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":true,"forks_count":237,"mirror_url":null,"archived":false,"open_issues_count":138,"license":{"key":"mit","name":"MIT License","spdx_id":"MIT","url":"https://api.github.com/licenses/mit","node_id":"MDc6TGljZW5zZTEz"},"forks":237,"open_issues":138,"watchers":380,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/broadinstitute/picard/pulls/569"},"html":{"href":"https://github.com/broadinstitute/picard/pull/569"},"issue":{"href":"https://api.github.com/repos/broadinstitute/picard/issues/569"},"comments":{"href":"https://api.github.com/repos/broadinstitute/picard/issues/569/comments"},"review_comments":{"href":"https://api.github.com/repos/broadinstitute/picard/pulls/569/comments"},"review_comment":{"href":"https://api.github.com/repos/broadinstitute/picard/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/broadinstitute/picard/pulls/569/commits"},"statuses":{"href":"https://api.github.com/repos/broadinstitute/picard/statuses/b80fe676e27de60f569b40f35858a174b3bf5f3b"}},"author_association":"CONTRIBUTOR"}}
{"actor":{"display_login":"fleharty"}}