{"action":"created","comment":{"url":"https://api.github.com/repos/ceph/ceph/pulls/comments/66304981","id":66304981,"diff_hunk":"@@ -1007,8 +1013,12 @@ void PGLog::read_log(ObjectStore *store, coll_t pg_coll,\n \tobject_info_t oi(bv);\n \tif (oi.version < i->version) {\n \t  ldpp_dout(dpp, 15) << \"read_log  missing \" << *i\n-\t\t\t     << \" (have \" << oi.version << \")\" << dendl;\n+\t\t\t     << \" (have \" << oi.version << \")\"\n+\t\t\t     << \" unmodified_omap \" << i->unmodified_omap\n+\t\t\t     << \" unmodified_extents \" << i->unmodified_extents << dendl;\n \t  missing.add(i->soid, i->version, oi.version);\n+\t  missing.missing[i->soid].unmodified_omap = i->unmodified_omap;\n+\t  missing.missing[i->soid].unmodified_extents.insert(i->unmodified_extents);","path":"src/osd/PGLog.cc","position":27,"original_position":27,"commit_id":"90f48bba0133cc5dc1bcfa55806c56ffa84163d1","original_commit_id":"90f48bba0133cc5dc1bcfa55806c56ffa84163d1","user":{"login":"athanatos","id":544782,"avatar_url":"https://avatars.githubusercontent.com/u/544782?v=3","gravatar_id":"","url":"https://api.github.com/users/athanatos","html_url":"https://github.com/athanatos","followers_url":"https://api.github.com/users/athanatos/followers","following_url":"https://api.github.com/users/athanatos/following{/other_user}","gists_url":"https://api.github.com/users/athanatos/gists{/gist_id}","starred_url":"https://api.github.com/users/athanatos/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/athanatos/subscriptions","organizations_url":"https://api.github.com/users/athanatos/orgs","repos_url":"https://api.github.com/users/athanatos/repos","events_url":"https://api.github.com/users/athanatos/events{/privacy}","received_events_url":"https://api.github.com/users/athanatos/received_events","type":"User","site_admin":false},"body":"This isn't correct.  Suppose the log is:\r\n\r\n(10'5, 10'8]\r\n\r\nwith entries\r\n\r\n10'6 modify foo unmodified_omap=true unmodified_extents=[0, 1024]\r\n10'7 modify foo unmodified_omap=false unmodified_extents=[1024, 2048]\r\n10'8 modify foo unmodified_omap=true unmodified_extents=[0, MAX]\r\n\r\nIf the on disk version if 10'6, this code will erroneously mark unmodified_omap=true and unmodified_extents=[0,MAX] causing recovery to *only recover the xattrs* even though it needs to also recover the omap and [0, 1024] of the data payload.  You need to walk the log from oi.version up to i->version and combine all of the log entries.  That logic should be encapsulated in PGLog and should be covered in the PGLog unit tests.  Those PGLog unit tests also need to be modified to test the logic for merging log entries with unmodified_omap and unmodified_extents into the missing entries.","created_at":"2016-06-08T17:51:49Z","updated_at":"2016-06-08T17:51:49Z","html_url":"https://github.com/ceph/ceph/pull/7325#discussion_r66304981","pull_request_url":"https://api.github.com/repos/ceph/ceph/pulls/7325","_links":{"self":{"href":"https://api.github.com/repos/ceph/ceph/pulls/comments/66304981"},"html":{"href":"https://github.com/ceph/ceph/pull/7325#discussion_r66304981"},"pull_request":{"href":"https://api.github.com/repos/ceph/ceph/pulls/7325"}}},"pull_request":{"url":"https://api.github.com/repos/ceph/ceph/pulls/7325","id":56877234,"html_url":"https://github.com/ceph/ceph/pull/7325","diff_url":"https://github.com/ceph/ceph/pull/7325.diff","patch_url":"https://github.com/ceph/ceph/pull/7325.patch","issue_url":"https://api.github.com/repos/ceph/ceph/issues/7325","number":7325,"state":"open","locked":false,"title":"DNM: osd: recovery optimization  for overwrite Ops","user":{"login":"mslovy","id":9523883,"avatar_url":"https://avatars.githubusercontent.com/u/9523883?v=3","gravatar_id":"","url":"https://api.github.com/users/mslovy","html_url":"https://github.com/mslovy","followers_url":"https://api.github.com/users/mslovy/followers","following_url":"https://api.github.com/users/mslovy/following{/other_user}","gists_url":"https://api.github.com/users/mslovy/gists{/gist_id}","starred_url":"https://api.github.com/users/mslovy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mslovy/subscriptions","organizations_url":"https://api.github.com/users/mslovy/orgs","repos_url":"https://api.github.com/users/mslovy/repos","events_url":"https://api.github.com/users/mslovy/events{/privacy}","received_events_url":"https://api.github.com/users/mslovy/received_events","type":"User","site_admin":false},"body":"This PR is based on https://github.com/ceph/ceph/pull/3837\r\nintroduce a can_recover_partial bit to make sure that only overwrite Ops can be recover as partial content. \r\nOtherwise, run recovery as the normal process\r\n\r\nSigned-off-by: Ning Yao yaoning@unitedstack.com","created_at":"2016-01-22T11:37:52Z","updated_at":"2016-06-08T17:51:49Z","closed_at":null,"merged_at":null,"merge_commit_sha":"996970e2890c119f49e5d7b77a3bfebdebda8cc2","assignee":{"login":"athanatos","id":544782,"avatar_url":"https://avatars.githubusercontent.com/u/544782?v=3","gravatar_id":"","url":"https://api.github.com/users/athanatos","html_url":"https://github.com/athanatos","followers_url":"https://api.github.com/users/athanatos/followers","following_url":"https://api.github.com/users/athanatos/following{/other_user}","gists_url":"https://api.github.com/users/athanatos/gists{/gist_id}","starred_url":"https://api.github.com/users/athanatos/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/athanatos/subscriptions","organizations_url":"https://api.github.com/users/athanatos/orgs","repos_url":"https://api.github.com/users/athanatos/repos","events_url":"https://api.github.com/users/athanatos/events{/privacy}","received_events_url":"https://api.github.com/users/athanatos/received_events","type":"User","site_admin":false},"milestone":null,"commits_url":"https://api.github.com/repos/ceph/ceph/pulls/7325/commits","review_comments_url":"https://api.github.com/repos/ceph/ceph/pulls/7325/comments","review_comment_url":"https://api.github.com/repos/ceph/ceph/pulls/comments{/number}","comments_url":"https://api.github.com/repos/ceph/ceph/issues/7325/comments","statuses_url":"https://api.github.com/repos/ceph/ceph/statuses/90f48bba0133cc5dc1bcfa55806c56ffa84163d1","head":{"label":"mslovy:wip-new-recovery","ref":"wip-new-recovery","sha":"90f48bba0133cc5dc1bcfa55806c56ffa84163d1","user":{"login":"mslovy","id":9523883,"avatar_url":"https://avatars.githubusercontent.com/u/9523883?v=3","gravatar_id":"","url":"https://api.github.com/users/mslovy","html_url":"https://github.com/mslovy","followers_url":"https://api.github.com/users/mslovy/followers","following_url":"https://api.github.com/users/mslovy/following{/other_user}","gists_url":"https://api.github.com/users/mslovy/gists{/gist_id}","starred_url":"https://api.github.com/users/mslovy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mslovy/subscriptions","organizations_url":"https://api.github.com/users/mslovy/orgs","repos_url":"https://api.github.com/users/mslovy/repos","events_url":"https://api.github.com/users/mslovy/events{/privacy}","received_events_url":"https://api.github.com/users/mslovy/received_events","type":"User","site_admin":false},"repo":{"id":29449534,"name":"ceph","full_name":"mslovy/ceph","owner":{"login":"mslovy","id":9523883,"avatar_url":"https://avatars.githubusercontent.com/u/9523883?v=3","gravatar_id":"","url":"https://api.github.com/users/mslovy","html_url":"https://github.com/mslovy","followers_url":"https://api.github.com/users/mslovy/followers","following_url":"https://api.github.com/users/mslovy/following{/other_user}","gists_url":"https://api.github.com/users/mslovy/gists{/gist_id}","starred_url":"https://api.github.com/users/mslovy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mslovy/subscriptions","organizations_url":"https://api.github.com/users/mslovy/orgs","repos_url":"https://api.github.com/users/mslovy/repos","events_url":"https://api.github.com/users/mslovy/events{/privacy}","received_events_url":"https://api.github.com/users/mslovy/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/mslovy/ceph","description":"Ceph is a distributed object, block, and file storage platform ","fork":true,"url":"https://api.github.com/repos/mslovy/ceph","forks_url":"https://api.github.com/repos/mslovy/ceph/forks","keys_url":"https://api.github.com/repos/mslovy/ceph/keys{/key_id}","collaborators_url":"https://api.github.com/repos/mslovy/ceph/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/mslovy/ceph/teams","hooks_url":"https://api.github.com/repos/mslovy/ceph/hooks","issue_events_url":"https://api.github.com/repos/mslovy/ceph/issues/events{/number}","events_url":"https://api.github.com/repos/mslovy/ceph/events","assignees_url":"https://api.github.com/repos/mslovy/ceph/assignees{/user}","branches_url":"https://api.github.com/repos/mslovy/ceph/branches{/branch}","tags_url":"https://api.github.com/repos/mslovy/ceph/tags","blobs_url":"https://api.github.com/repos/mslovy/ceph/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/mslovy/ceph/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/mslovy/ceph/git/refs{/sha}","trees_url":"https://api.github.com/repos/mslovy/ceph/git/trees{/sha}","statuses_url":"https://api.github.com/repos/mslovy/ceph/statuses/{sha}","languages_url":"https://api.github.com/repos/mslovy/ceph/languages","stargazers_url":"https://api.github.com/repos/mslovy/ceph/stargazers","contributors_url":"https://api.github.com/repos/mslovy/ceph/contributors","subscribers_url":"https://api.github.com/repos/mslovy/ceph/subscribers","subscription_url":"https://api.github.com/repos/mslovy/ceph/subscription","commits_url":"https://api.github.com/repos/mslovy/ceph/commits{/sha}","git_commits_url":"https://api.github.com/repos/mslovy/ceph/git/commits{/sha}","comments_url":"https://api.github.com/repos/mslovy/ceph/comments{/number}","issue_comment_url":"https://api.github.com/repos/mslovy/ceph/issues/comments{/number}","contents_url":"https://api.github.com/repos/mslovy/ceph/contents/{+path}","compare_url":"https://api.github.com/repos/mslovy/ceph/compare/{base}...{head}","merges_url":"https://api.github.com/repos/mslovy/ceph/merges","archive_url":"https://api.github.com/repos/mslovy/ceph/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/mslovy/ceph/downloads","issues_url":"https://api.github.com/repos/mslovy/ceph/issues{/number}","pulls_url":"https://api.github.com/repos/mslovy/ceph/pulls{/number}","milestones_url":"https://api.github.com/repos/mslovy/ceph/milestones{/number}","notifications_url":"https://api.github.com/repos/mslovy/ceph/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/mslovy/ceph/labels{/name}","releases_url":"https://api.github.com/repos/mslovy/ceph/releases{/id}","deployments_url":"https://api.github.com/repos/mslovy/ceph/deployments","created_at":"2015-01-19T02:14:49Z","updated_at":"2015-01-19T02:15:01Z","pushed_at":"2016-06-07T13:36:37Z","git_url":"git://github.com/mslovy/ceph.git","ssh_url":"git@github.com:mslovy/ceph.git","clone_url":"https://github.com/mslovy/ceph.git","svn_url":"https://github.com/mslovy/ceph","homepage":"http://ceph.com","size":155291,"stargazers_count":0,"watchers_count":0,"language":"C++","has_issues":false,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"ceph:master","ref":"master","sha":"4e5a0186a1bb775844a1b4c2065f409b887de773","user":{"login":"ceph","id":1015767,"avatar_url":"https://avatars.githubusercontent.com/u/1015767?v=3","gravatar_id":"","url":"https://api.github.com/users/ceph","html_url":"https://github.com/ceph","followers_url":"https://api.github.com/users/ceph/followers","following_url":"https://api.github.com/users/ceph/following{/other_user}","gists_url":"https://api.github.com/users/ceph/gists{/gist_id}","starred_url":"https://api.github.com/users/ceph/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ceph/subscriptions","organizations_url":"https://api.github.com/users/ceph/orgs","repos_url":"https://api.github.com/users/ceph/repos","events_url":"https://api.github.com/users/ceph/events{/privacy}","received_events_url":"https://api.github.com/users/ceph/received_events","type":"Organization","site_admin":false},"repo":{"id":2310495,"name":"ceph","full_name":"ceph/ceph","owner":{"login":"ceph","id":1015767,"avatar_url":"https://avatars.githubusercontent.com/u/1015767?v=3","gravatar_id":"","url":"https://api.github.com/users/ceph","html_url":"https://github.com/ceph","followers_url":"https://api.github.com/users/ceph/followers","following_url":"https://api.github.com/users/ceph/following{/other_user}","gists_url":"https://api.github.com/users/ceph/gists{/gist_id}","starred_url":"https://api.github.com/users/ceph/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ceph/subscriptions","organizations_url":"https://api.github.com/users/ceph/orgs","repos_url":"https://api.github.com/users/ceph/repos","events_url":"https://api.github.com/users/ceph/events{/privacy}","received_events_url":"https://api.github.com/users/ceph/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/ceph/ceph","description":"Ceph is a distributed object, block, and file storage platform ","fork":false,"url":"https://api.github.com/repos/ceph/ceph","forks_url":"https://api.github.com/repos/ceph/ceph/forks","keys_url":"https://api.github.com/repos/ceph/ceph/keys{/key_id}","collaborators_url":"https://api.github.com/repos/ceph/ceph/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/ceph/ceph/teams","hooks_url":"https://api.github.com/repos/ceph/ceph/hooks","issue_events_url":"https://api.github.com/repos/ceph/ceph/issues/events{/number}","events_url":"https://api.github.com/repos/ceph/ceph/events","assignees_url":"https://api.github.com/repos/ceph/ceph/assignees{/user}","branches_url":"https://api.github.com/repos/ceph/ceph/branches{/branch}","tags_url":"https://api.github.com/repos/ceph/ceph/tags","blobs_url":"https://api.github.com/repos/ceph/ceph/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/ceph/ceph/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/ceph/ceph/git/refs{/sha}","trees_url":"https://api.github.com/repos/ceph/ceph/git/trees{/sha}","statuses_url":"https://api.github.com/repos/ceph/ceph/statuses/{sha}","languages_url":"https://api.github.com/repos/ceph/ceph/languages","stargazers_url":"https://api.github.com/repos/ceph/ceph/stargazers","contributors_url":"https://api.github.com/repos/ceph/ceph/contributors","subscribers_url":"https://api.github.com/repos/ceph/ceph/subscribers","subscription_url":"https://api.github.com/repos/ceph/ceph/subscription","commits_url":"https://api.github.com/repos/ceph/ceph/commits{/sha}","git_commits_url":"https://api.github.com/repos/ceph/ceph/git/commits{/sha}","comments_url":"https://api.github.com/repos/ceph/ceph/comments{/number}","issue_comment_url":"https://api.github.com/repos/ceph/ceph/issues/comments{/number}","contents_url":"https://api.github.com/repos/ceph/ceph/contents/{+path}","compare_url":"https://api.github.com/repos/ceph/ceph/compare/{base}...{head}","merges_url":"https://api.github.com/repos/ceph/ceph/merges","archive_url":"https://api.github.com/repos/ceph/ceph/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/ceph/ceph/downloads","issues_url":"https://api.github.com/repos/ceph/ceph/issues{/number}","pulls_url":"https://api.github.com/repos/ceph/ceph/pulls{/number}","milestones_url":"https://api.github.com/repos/ceph/ceph/milestones{/number}","notifications_url":"https://api.github.com/repos/ceph/ceph/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/ceph/ceph/labels{/name}","releases_url":"https://api.github.com/repos/ceph/ceph/releases{/id}","deployments_url":"https://api.github.com/repos/ceph/ceph/deployments","created_at":"2011-09-01T21:41:26Z","updated_at":"2016-06-08T02:36:10Z","pushed_at":"2016-06-08T16:43:51Z","git_url":"git://github.com/ceph/ceph.git","ssh_url":"git@github.com:ceph/ceph.git","clone_url":"https://github.com/ceph/ceph.git","svn_url":"https://github.com/ceph/ceph","homepage":"http://ceph.com","size":217773,"stargazers_count":2222,"watchers_count":2222,"language":"C++","has_issues":false,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":1311,"mirror_url":null,"open_issues_count":396,"forks":1311,"open_issues":396,"watchers":2222,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/ceph/ceph/pulls/7325"},"html":{"href":"https://github.com/ceph/ceph/pull/7325"},"issue":{"href":"https://api.github.com/repos/ceph/ceph/issues/7325"},"comments":{"href":"https://api.github.com/repos/ceph/ceph/issues/7325/comments"},"review_comments":{"href":"https://api.github.com/repos/ceph/ceph/pulls/7325/comments"},"review_comment":{"href":"https://api.github.com/repos/ceph/ceph/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/ceph/ceph/pulls/7325/commits"},"statuses":{"href":"https://api.github.com/repos/ceph/ceph/statuses/90f48bba0133cc5dc1bcfa55806c56ffa84163d1"}}}}
{"actor":{"display_login":"athanatos"}}