{"action":"created","comment":{"url":"https://api.github.com/repos/ceph/ceph/pulls/comments/56825365","id":56825365,"diff_hunk":"@@ -1310,9 +1310,19 @@ void PGMonitor::_try_mark_pg_stale(\n \n bool PGMonitor::check_down_pgs()\n {\n-  dout(10) << \"check_down_pgs\" << dendl;\n+  dout(10) << \"check_down_pgs last_osdmap_epoch \"\n+\t   << pg_map.last_osdmap_epoch << dendl;\n+  if (pg_map.last_osdmap_epoch == 0)\n+    return false;\n+\n+  // use the OSDMap that matches the one pg_map has consumed.\n+  std::unique_ptr<OSDMap> osdmap;\n+  bufferlist bl;\n+  int err = mon->osdmon()->get_version_full(pg_map.last_osdmap_epoch, bl);","path":"src/mon/PGMonitor.cc","position":97,"original_position":13,"commit_id":"a6adb882a9cb6ce90c7a29462ccc0684e205067d","original_commit_id":"ad25029be0240035869ec70eff2ea12b8320e311","user":{"login":"tchaikov","id":59071,"avatar_url":"https://avatars.githubusercontent.com/u/59071?v=3","gravatar_id":"","url":"https://api.github.com/users/tchaikov","html_url":"https://github.com/tchaikov","followers_url":"https://api.github.com/users/tchaikov/followers","following_url":"https://api.github.com/users/tchaikov/following{/other_user}","gists_url":"https://api.github.com/users/tchaikov/gists{/gist_id}","starred_url":"https://api.github.com/users/tchaikov/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tchaikov/subscriptions","organizations_url":"https://api.github.com/users/tchaikov/orgs","repos_url":"https://api.github.com/users/tchaikov/repos","events_url":"https://api.github.com/users/tchaikov/events{/privacy}","received_events_url":"https://api.github.com/users/tchaikov/received_events","type":"User","site_admin":false},"body":"the OSDMonitor kicks pgmon in `OSDMonitor::update_from_paxos()`, so the pgmon can consume (`check_osd_map()`) the latest osdmaps once the monitor is **active**[1]. and pg_map.last_osdmap_epoch is updated in `PGMonitor::update_from_paxo()` when the monitor is still **updating**.\r\n\r\nso, is there a chance that `check_down_pgs()` gets called **after** `pg_map.last_osdmap_epoch` is updated, and **before** `check_osd_map()` is called? in other words:\r\n\r\n1. pg_map.set_last_osdmap_epoch(last_osdmap_epoch);\r\n1. PGMonitor::check_down_pgs()\r\n1. check_osd_map()\r\n\r\n---\r\n[1] see `PGMonitor::check_osd_map()`\r\n```c++\r\n  if (!is_writeable()) {\r\n    dout(10) << __func__ << \" -- pgmap not writeable, waiting\" << dendl;\r\n    wait_for_writeable_ctx(new RetryCheckOSDMap(this, epoch));\r\n    return;\r\n  }\r\n```\r\n\r\nthe monitor is not writable until it is done with refresh (the `update_from_paxos()` calls).","created_at":"2016-03-21T13:59:03Z","updated_at":"2016-03-21T13:59:03Z","html_url":"https://github.com/ceph/ceph/pull/8208#discussion_r56825365","pull_request_url":"https://api.github.com/repos/ceph/ceph/pulls/8208","_links":{"self":{"href":"https://api.github.com/repos/ceph/ceph/pulls/comments/56825365"},"html":{"href":"https://github.com/ceph/ceph/pull/8208#discussion_r56825365"},"pull_request":{"href":"https://api.github.com/repos/ceph/ceph/pulls/8208"}}},"pull_request":{"url":"https://api.github.com/repos/ceph/ceph/pulls/8208","id":63376903,"html_url":"https://github.com/ceph/ceph/pull/8208","diff_url":"https://github.com/ceph/ceph/pull/8208.diff","patch_url":"https://github.com/ceph/ceph/pull/8208.patch","issue_url":"https://api.github.com/repos/ceph/ceph/issues/8208","number":8208,"state":"open","locked":false,"title":"mon: mark_down_pgs in lockstep with pg_map's osdmap epoch","user":{"login":"liewegas","id":433031,"avatar_url":"https://avatars.githubusercontent.com/u/433031?v=3","gravatar_id":"","url":"https://api.github.com/users/liewegas","html_url":"https://github.com/liewegas","followers_url":"https://api.github.com/users/liewegas/followers","following_url":"https://api.github.com/users/liewegas/following{/other_user}","gists_url":"https://api.github.com/users/liewegas/gists{/gist_id}","starred_url":"https://api.github.com/users/liewegas/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/liewegas/subscriptions","organizations_url":"https://api.github.com/users/liewegas/orgs","repos_url":"https://api.github.com/users/liewegas/repos","events_url":"https://api.github.com/users/liewegas/events{/privacy}","received_events_url":"https://api.github.com/users/liewegas/received_events","type":"User","site_admin":false},"body":"This fixes the stale PGs we've been seeing.  And probably\nseveral corner cases that are hard to see.","created_at":"2016-03-18T13:52:53Z","updated_at":"2016-03-21T13:59:03Z","closed_at":null,"merged_at":null,"merge_commit_sha":"06c4db2c2d2df5bf8b09a36594e6f92f36a393a0","assignee":null,"milestone":{"url":"https://api.github.com/repos/ceph/ceph/milestones/8","html_url":"https://github.com/ceph/ceph/milestones/jewel","labels_url":"https://api.github.com/repos/ceph/ceph/milestones/8/labels","id":1403256,"number":8,"title":"jewel","description":"","creator":{"login":"dillaman","id":4739042,"avatar_url":"https://avatars.githubusercontent.com/u/4739042?v=3","gravatar_id":"","url":"https://api.github.com/users/dillaman","html_url":"https://github.com/dillaman","followers_url":"https://api.github.com/users/dillaman/followers","following_url":"https://api.github.com/users/dillaman/following{/other_user}","gists_url":"https://api.github.com/users/dillaman/gists{/gist_id}","starred_url":"https://api.github.com/users/dillaman/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dillaman/subscriptions","organizations_url":"https://api.github.com/users/dillaman/orgs","repos_url":"https://api.github.com/users/dillaman/repos","events_url":"https://api.github.com/users/dillaman/events{/privacy}","received_events_url":"https://api.github.com/users/dillaman/received_events","type":"User","site_admin":false},"open_issues":32,"closed_issues":215,"state":"open","created_at":"2015-11-09T22:29:48Z","updated_at":"2016-03-21T13:35:27Z","due_on":"2016-03-20T04:00:00Z","closed_at":null},"commits_url":"https://api.github.com/repos/ceph/ceph/pulls/8208/commits","review_comments_url":"https://api.github.com/repos/ceph/ceph/pulls/8208/comments","review_comment_url":"https://api.github.com/repos/ceph/ceph/pulls/comments{/number}","comments_url":"https://api.github.com/repos/ceph/ceph/issues/8208/comments","statuses_url":"https://api.github.com/repos/ceph/ceph/statuses/a6adb882a9cb6ce90c7a29462ccc0684e205067d","head":{"label":"liewegas:wip-15027","ref":"wip-15027","sha":"a6adb882a9cb6ce90c7a29462ccc0684e205067d","user":{"login":"liewegas","id":433031,"avatar_url":"https://avatars.githubusercontent.com/u/433031?v=3","gravatar_id":"","url":"https://api.github.com/users/liewegas","html_url":"https://github.com/liewegas","followers_url":"https://api.github.com/users/liewegas/followers","following_url":"https://api.github.com/users/liewegas/following{/other_user}","gists_url":"https://api.github.com/users/liewegas/gists{/gist_id}","starred_url":"https://api.github.com/users/liewegas/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/liewegas/subscriptions","organizations_url":"https://api.github.com/users/liewegas/orgs","repos_url":"https://api.github.com/users/liewegas/repos","events_url":"https://api.github.com/users/liewegas/events{/privacy}","received_events_url":"https://api.github.com/users/liewegas/received_events","type":"User","site_admin":false},"repo":{"id":2430918,"name":"ceph","full_name":"liewegas/ceph","owner":{"login":"liewegas","id":433031,"avatar_url":"https://avatars.githubusercontent.com/u/433031?v=3","gravatar_id":"","url":"https://api.github.com/users/liewegas","html_url":"https://github.com/liewegas","followers_url":"https://api.github.com/users/liewegas/followers","following_url":"https://api.github.com/users/liewegas/following{/other_user}","gists_url":"https://api.github.com/users/liewegas/gists{/gist_id}","starred_url":"https://api.github.com/users/liewegas/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/liewegas/subscriptions","organizations_url":"https://api.github.com/users/liewegas/orgs","repos_url":"https://api.github.com/users/liewegas/repos","events_url":"https://api.github.com/users/liewegas/events{/privacy}","received_events_url":"https://api.github.com/users/liewegas/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/liewegas/ceph","description":"Ceph distributed file system","fork":true,"url":"https://api.github.com/repos/liewegas/ceph","forks_url":"https://api.github.com/repos/liewegas/ceph/forks","keys_url":"https://api.github.com/repos/liewegas/ceph/keys{/key_id}","collaborators_url":"https://api.github.com/repos/liewegas/ceph/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/liewegas/ceph/teams","hooks_url":"https://api.github.com/repos/liewegas/ceph/hooks","issue_events_url":"https://api.github.com/repos/liewegas/ceph/issues/events{/number}","events_url":"https://api.github.com/repos/liewegas/ceph/events","assignees_url":"https://api.github.com/repos/liewegas/ceph/assignees{/user}","branches_url":"https://api.github.com/repos/liewegas/ceph/branches{/branch}","tags_url":"https://api.github.com/repos/liewegas/ceph/tags","blobs_url":"https://api.github.com/repos/liewegas/ceph/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/liewegas/ceph/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/liewegas/ceph/git/refs{/sha}","trees_url":"https://api.github.com/repos/liewegas/ceph/git/trees{/sha}","statuses_url":"https://api.github.com/repos/liewegas/ceph/statuses/{sha}","languages_url":"https://api.github.com/repos/liewegas/ceph/languages","stargazers_url":"https://api.github.com/repos/liewegas/ceph/stargazers","contributors_url":"https://api.github.com/repos/liewegas/ceph/contributors","subscribers_url":"https://api.github.com/repos/liewegas/ceph/subscribers","subscription_url":"https://api.github.com/repos/liewegas/ceph/subscription","commits_url":"https://api.github.com/repos/liewegas/ceph/commits{/sha}","git_commits_url":"https://api.github.com/repos/liewegas/ceph/git/commits{/sha}","comments_url":"https://api.github.com/repos/liewegas/ceph/comments{/number}","issue_comment_url":"https://api.github.com/repos/liewegas/ceph/issues/comments{/number}","contents_url":"https://api.github.com/repos/liewegas/ceph/contents/{+path}","compare_url":"https://api.github.com/repos/liewegas/ceph/compare/{base}...{head}","merges_url":"https://api.github.com/repos/liewegas/ceph/merges","archive_url":"https://api.github.com/repos/liewegas/ceph/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/liewegas/ceph/downloads","issues_url":"https://api.github.com/repos/liewegas/ceph/issues{/number}","pulls_url":"https://api.github.com/repos/liewegas/ceph/pulls{/number}","milestones_url":"https://api.github.com/repos/liewegas/ceph/milestones{/number}","notifications_url":"https://api.github.com/repos/liewegas/ceph/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/liewegas/ceph/labels{/name}","releases_url":"https://api.github.com/repos/liewegas/ceph/releases{/id}","deployments_url":"https://api.github.com/repos/liewegas/ceph/deployments","created_at":"2011-09-21T16:14:23Z","updated_at":"2016-03-01T03:21:25Z","pushed_at":"2016-03-21T12:48:05Z","git_url":"git://github.com/liewegas/ceph.git","ssh_url":"git@github.com:liewegas/ceph.git","clone_url":"https://github.com/liewegas/ceph.git","svn_url":"https://github.com/liewegas/ceph","homepage":"http://ceph.newdream.net/","size":156130,"stargazers_count":11,"watchers_count":11,"language":"C++","has_issues":false,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":4,"mirror_url":null,"open_issues_count":0,"forks":4,"open_issues":0,"watchers":11,"default_branch":"master"}},"base":{"label":"ceph:master","ref":"master","sha":"19aabf4a3047eb63c1646375357054ca12991a06","user":{"login":"ceph","id":1015767,"avatar_url":"https://avatars.githubusercontent.com/u/1015767?v=3","gravatar_id":"","url":"https://api.github.com/users/ceph","html_url":"https://github.com/ceph","followers_url":"https://api.github.com/users/ceph/followers","following_url":"https://api.github.com/users/ceph/following{/other_user}","gists_url":"https://api.github.com/users/ceph/gists{/gist_id}","starred_url":"https://api.github.com/users/ceph/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ceph/subscriptions","organizations_url":"https://api.github.com/users/ceph/orgs","repos_url":"https://api.github.com/users/ceph/repos","events_url":"https://api.github.com/users/ceph/events{/privacy}","received_events_url":"https://api.github.com/users/ceph/received_events","type":"Organization","site_admin":false},"repo":{"id":2310495,"name":"ceph","full_name":"ceph/ceph","owner":{"login":"ceph","id":1015767,"avatar_url":"https://avatars.githubusercontent.com/u/1015767?v=3","gravatar_id":"","url":"https://api.github.com/users/ceph","html_url":"https://github.com/ceph","followers_url":"https://api.github.com/users/ceph/followers","following_url":"https://api.github.com/users/ceph/following{/other_user}","gists_url":"https://api.github.com/users/ceph/gists{/gist_id}","starred_url":"https://api.github.com/users/ceph/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ceph/subscriptions","organizations_url":"https://api.github.com/users/ceph/orgs","repos_url":"https://api.github.com/users/ceph/repos","events_url":"https://api.github.com/users/ceph/events{/privacy}","received_events_url":"https://api.github.com/users/ceph/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/ceph/ceph","description":"Ceph is a distributed object, block, and file storage platform ","fork":false,"url":"https://api.github.com/repos/ceph/ceph","forks_url":"https://api.github.com/repos/ceph/ceph/forks","keys_url":"https://api.github.com/repos/ceph/ceph/keys{/key_id}","collaborators_url":"https://api.github.com/repos/ceph/ceph/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/ceph/ceph/teams","hooks_url":"https://api.github.com/repos/ceph/ceph/hooks","issue_events_url":"https://api.github.com/repos/ceph/ceph/issues/events{/number}","events_url":"https://api.github.com/repos/ceph/ceph/events","assignees_url":"https://api.github.com/repos/ceph/ceph/assignees{/user}","branches_url":"https://api.github.com/repos/ceph/ceph/branches{/branch}","tags_url":"https://api.github.com/repos/ceph/ceph/tags","blobs_url":"https://api.github.com/repos/ceph/ceph/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/ceph/ceph/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/ceph/ceph/git/refs{/sha}","trees_url":"https://api.github.com/repos/ceph/ceph/git/trees{/sha}","statuses_url":"https://api.github.com/repos/ceph/ceph/statuses/{sha}","languages_url":"https://api.github.com/repos/ceph/ceph/languages","stargazers_url":"https://api.github.com/repos/ceph/ceph/stargazers","contributors_url":"https://api.github.com/repos/ceph/ceph/contributors","subscribers_url":"https://api.github.com/repos/ceph/ceph/subscribers","subscription_url":"https://api.github.com/repos/ceph/ceph/subscription","commits_url":"https://api.github.com/repos/ceph/ceph/commits{/sha}","git_commits_url":"https://api.github.com/repos/ceph/ceph/git/commits{/sha}","comments_url":"https://api.github.com/repos/ceph/ceph/comments{/number}","issue_comment_url":"https://api.github.com/repos/ceph/ceph/issues/comments{/number}","contents_url":"https://api.github.com/repos/ceph/ceph/contents/{+path}","compare_url":"https://api.github.com/repos/ceph/ceph/compare/{base}...{head}","merges_url":"https://api.github.com/repos/ceph/ceph/merges","archive_url":"https://api.github.com/repos/ceph/ceph/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/ceph/ceph/downloads","issues_url":"https://api.github.com/repos/ceph/ceph/issues{/number}","pulls_url":"https://api.github.com/repos/ceph/ceph/pulls{/number}","milestones_url":"https://api.github.com/repos/ceph/ceph/milestones{/number}","notifications_url":"https://api.github.com/repos/ceph/ceph/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/ceph/ceph/labels{/name}","releases_url":"https://api.github.com/repos/ceph/ceph/releases{/id}","deployments_url":"https://api.github.com/repos/ceph/ceph/deployments","created_at":"2011-09-01T21:41:26Z","updated_at":"2016-03-21T10:29:57Z","pushed_at":"2016-03-21T13:45:35Z","git_url":"git://github.com/ceph/ceph.git","ssh_url":"git@github.com:ceph/ceph.git","clone_url":"https://github.com/ceph/ceph.git","svn_url":"https://github.com/ceph/ceph","homepage":"http://ceph.com","size":208296,"stargazers_count":1968,"watchers_count":1968,"language":"C++","has_issues":false,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":1200,"mirror_url":null,"open_issues_count":281,"forks":1200,"open_issues":281,"watchers":1968,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/ceph/ceph/pulls/8208"},"html":{"href":"https://github.com/ceph/ceph/pull/8208"},"issue":{"href":"https://api.github.com/repos/ceph/ceph/issues/8208"},"comments":{"href":"https://api.github.com/repos/ceph/ceph/issues/8208/comments"},"review_comments":{"href":"https://api.github.com/repos/ceph/ceph/pulls/8208/comments"},"review_comment":{"href":"https://api.github.com/repos/ceph/ceph/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/ceph/ceph/pulls/8208/commits"},"statuses":{"href":"https://api.github.com/repos/ceph/ceph/statuses/a6adb882a9cb6ce90c7a29462ccc0684e205067d"}}}}