{"action":"created","comment":{"url":"https://api.github.com/repos/ceph/ceph/pulls/comments/65588144","id":65588144,"diff_hunk":"@@ -0,0 +1,199 @@\n+msgr2 protocol\n+==============\n+\n+This is a revision of the legacy Ceph on-wire protocol that was\n+implemented by the SimpleMessenger.  It addresses performance and\n+security issues.\n+\n+Definitions\n+-----------\n+\n+* *client* (C): the party initiating a (TCP) connection\n+* *server* (S): the party accepting a (TCP) connection\n+* *connection*: an instance of a (TCP) connection between two peers\n+* *session*: a stateful session between two peers in which message\n+  exchange is ordered and lossless.  A session might span multiple\n+  connections if there is an interruption (TCP connection disconnect).\n+* *frame*: a discrete message sent between the peers.  Each frame\n+  consists of a tag (type code), payload, and (if signing or\n+  encryption is enabled) some other fields.  See below for the\n+  structure.\n+* *tag*: a single-byte type code associated with a frame.  The tag\n+  determines the structure of the payload.\n+\n+Phases\n+------\n+\n+A connection has four distinct phases:\n+\n+#. banner\n+#. authentication\n+#. message flow handshake\n+#. message exchange\n+\n+Banner\n+------\n+\n+Both the client and server, upon connecting, send a banner::\n+\n+  \"ceph %x %x\\n\", protocol_features_suppored, protocol_features_required\n+\n+The protocol features are a new, distinct namespace.  Initially no\n+features are defined or required, so this will be \"ceph 0 0\\n\".\n+\n+If the remote party advertises features we don't support, we can\n+disconnect.\n+\n+Authentication\n+--------------\n+\n+A series of frames between client and server of the form::\n+\n+  tag byte (TAG_*)\n+  payload\n+\n+where the payload is determiend by the tag.\n+\n+* TAG_AUTH_METHODS (server only): list authentication methods (none, cephx, ...)::\n+\n+    __le32 num_methods;\n+    __le32 methods[num_methods];   // CEPH_AUTH_{NONE, CEPHX}\n+\n+* TAG_AUTH_SET_METHOD (client only): set auth method for this connection::\n+\n+    __le32 method;\n+\n+  - The selected auth method determines the sig_size and block_size in any\n+    subsequent messages (TAG_AUTH_DONE and non-auth messages).\n+\n+* TAG_AUTH_BAD_METHOD (server only): reject client-selected auth method::\n+\n+    __le32 method\n+\n+* TAG_AUTH: client->server or server->client auth message::\n+\n+    __le32 len;\n+    method specific payload\n+\n+* TAG_AUTH_DONE::\n+    \n+    __le64 flags\n+      FLAG_ENCRYPTED  1\n+      FLAG_SIGNED     2\n+    signature\n+\n+  - The client first says AUTH_DONE, and the server replies to\n+    acknowledge it.\n+\n+\n+Message frame format\n+--------------------\n+\n+Each frame can take one of two forms.  If FLAG_SIGNED or\n+FLAG_ENCRYPTED has been specified::\n+\n+  confounder (block_size bytes of random garbage)\n+  __le32 length\n+  tag byte\n+  payload\n+  signature (sig_size bytes)\n+  more confounder padding (to pad data from start of __le32 length out to block size)\n+\n+Note that the padding ensures that the total frame (with or without\n+the leading confounder) is a multiple of the auth method's block_size.\n+This is usually something like 16 bytes.\n+\n+If neither FLAG_SIGNED or FLAG_ENCRYPTED is specified, things are simple::\n+\n+  tag byte\n+  payload\n+    \n+Message flow handshake\n+----------------------\n+\n+In this phase the peers identify each other and (if desired) reconnect to\n+an established session.\n+\n+* TAG_IDENT: identify ourselves::\n+\n+    entity_addrvec_t addr(s)\n+    __u8   my type (CEPH_ENTITY_TYPE_*)\n+    __le32 protocol version\n+    __le64 features supported (CEPH_FEATURE_* bitmask)\n+    __le64 features required (CEPH_FEATURE_* bitmask)\n+    __le64 flags (CEPH_MSG_CONNECT_* bitmask)\n+    __le64 cookie (a client identifier, assigned by the sender. unique on the sender.)\n+\n+  - client will send first, server will reply with same.\n+\n+* TAG_IDENT_MISSING_FEATURES (server only): complain about a TAG_IDENT with too few features::\n+\n+    __le64 features we require that peer didn't advertise\n+\n+* TAG_IDENT_BAD_PROTOCOL (server only): complain about an old protocol version::\n+\n+    __le32 protocol_version (our protocol version)\n+\n+* TAG_RECONNECT (client only): reconnect to an established session::\n+\n+    __le64 cookie\n+    __le64 global_seq\n+    __le64 connect_seq\n+    __le64 msg_seq (the last msg seq received)\n+\n+* TAG_RECONNECT_OK (server only): acknowledge a reconnect attempt::\n+\n+    __le64 msg_seq (last msg seq received)\n+\n+* TAG_RECONNECT_RETRY_SESSION (server only): fail reconnect due to stale connect_seq\n+\n+* TAG_RECONNECT_RETRY_GLOBAL (server only): fail reconnect due to stale global_seq\n+\n+* TAG_RECONNECT_WAIT (server only): fail reconnect due to connect race.\n+\n+  - Indicates that the server is already connecting to the client, and\n+    that direction should win the race.  The client should wait for that\n+    connection to complete.\n+\n+* TAG_START: client is ready to start the message exchange session::\n+\n+    __le64 flags\n+\n+  - Sent by client when it is ready.  Server replies with same.\n+  - No flags are defined now, but we might use them later.\n+  - I'm not sure this is necessary.","path":"doc/dev/msgr2.rst","position":164,"original_position":164,"commit_id":"ff64bbffa0f618663ff01ca75a375c152cd730cd","original_commit_id":"ff64bbffa0f618663ff01ca75a375c152cd730cd","user":{"login":"gregsfortytwo","id":908479,"avatar_url":"https://avatars.githubusercontent.com/u/908479?v=3","gravatar_id":"","url":"https://api.github.com/users/gregsfortytwo","html_url":"https://github.com/gregsfortytwo","followers_url":"https://api.github.com/users/gregsfortytwo/followers","following_url":"https://api.github.com/users/gregsfortytwo/following{/other_user}","gists_url":"https://api.github.com/users/gregsfortytwo/gists{/gist_id}","starred_url":"https://api.github.com/users/gregsfortytwo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gregsfortytwo/subscriptions","organizations_url":"https://api.github.com/users/gregsfortytwo/orgs","repos_url":"https://api.github.com/users/gregsfortytwo/repos","events_url":"https://api.github.com/users/gregsfortytwo/events{/privacy}","received_events_url":"https://api.github.com/users/gregsfortytwo/received_events","type":"User","site_admin":false},"body":"I'm not either. What flags might we want to include here that aren't specified elsewhere?\r\nI just want to cut down the round-trip counts and I think the only reason to have explicit tags would be if we need to do one of\r\n*) authenticate data that was previously sent in the clear (and doesn't apply by this point)\r\n*) make a decision that can't be deterministically done on both sides based on established data (which I don't think applies by this point)\r\nAre there cases I'm missing? Does having a TAG_START here enable either of those cases?","created_at":"2016-06-02T17:59:12Z","updated_at":"2016-06-02T17:59:12Z","html_url":"https://github.com/ceph/ceph/pull/9461#discussion_r65588144","pull_request_url":"https://api.github.com/repos/ceph/ceph/pulls/9461","_links":{"self":{"href":"https://api.github.com/repos/ceph/ceph/pulls/comments/65588144"},"html":{"href":"https://github.com/ceph/ceph/pull/9461#discussion_r65588144"},"pull_request":{"href":"https://api.github.com/repos/ceph/ceph/pulls/9461"}}},"pull_request":{"url":"https://api.github.com/repos/ceph/ceph/pulls/9461","id":72367324,"html_url":"https://github.com/ceph/ceph/pull/9461","diff_url":"https://github.com/ceph/ceph/pull/9461.diff","patch_url":"https://github.com/ceph/ceph/pull/9461.patch","issue_url":"https://api.github.com/repos/ceph/ceph/issues/9461","number":9461,"state":"open","locked":false,"title":"doc: msgr2 protocol spec","user":{"login":"liewegas","id":433031,"avatar_url":"https://avatars.githubusercontent.com/u/433031?v=3","gravatar_id":"","url":"https://api.github.com/users/liewegas","html_url":"https://github.com/liewegas","followers_url":"https://api.github.com/users/liewegas/followers","following_url":"https://api.github.com/users/liewegas/following{/other_user}","gists_url":"https://api.github.com/users/liewegas/gists{/gist_id}","starred_url":"https://api.github.com/users/liewegas/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/liewegas/subscriptions","organizations_url":"https://api.github.com/users/liewegas/orgs","repos_url":"https://api.github.com/users/liewegas/repos","events_url":"https://api.github.com/users/liewegas/events{/privacy}","received_events_url":"https://api.github.com/users/liewegas/received_events","type":"User","site_admin":false},"body":null,"created_at":"2016-06-02T15:30:58Z","updated_at":"2016-06-02T17:59:12Z","closed_at":null,"merged_at":null,"merge_commit_sha":"e9af065654839255a4a77a6f0dd8e5b847856824","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/ceph/ceph/pulls/9461/commits","review_comments_url":"https://api.github.com/repos/ceph/ceph/pulls/9461/comments","review_comment_url":"https://api.github.com/repos/ceph/ceph/pulls/comments{/number}","comments_url":"https://api.github.com/repos/ceph/ceph/issues/9461/comments","statuses_url":"https://api.github.com/repos/ceph/ceph/statuses/ff64bbffa0f618663ff01ca75a375c152cd730cd","head":{"label":"liewegas:wip-msgr2","ref":"wip-msgr2","sha":"ff64bbffa0f618663ff01ca75a375c152cd730cd","user":{"login":"liewegas","id":433031,"avatar_url":"https://avatars.githubusercontent.com/u/433031?v=3","gravatar_id":"","url":"https://api.github.com/users/liewegas","html_url":"https://github.com/liewegas","followers_url":"https://api.github.com/users/liewegas/followers","following_url":"https://api.github.com/users/liewegas/following{/other_user}","gists_url":"https://api.github.com/users/liewegas/gists{/gist_id}","starred_url":"https://api.github.com/users/liewegas/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/liewegas/subscriptions","organizations_url":"https://api.github.com/users/liewegas/orgs","repos_url":"https://api.github.com/users/liewegas/repos","events_url":"https://api.github.com/users/liewegas/events{/privacy}","received_events_url":"https://api.github.com/users/liewegas/received_events","type":"User","site_admin":false},"repo":{"id":2430918,"name":"ceph","full_name":"liewegas/ceph","owner":{"login":"liewegas","id":433031,"avatar_url":"https://avatars.githubusercontent.com/u/433031?v=3","gravatar_id":"","url":"https://api.github.com/users/liewegas","html_url":"https://github.com/liewegas","followers_url":"https://api.github.com/users/liewegas/followers","following_url":"https://api.github.com/users/liewegas/following{/other_user}","gists_url":"https://api.github.com/users/liewegas/gists{/gist_id}","starred_url":"https://api.github.com/users/liewegas/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/liewegas/subscriptions","organizations_url":"https://api.github.com/users/liewegas/orgs","repos_url":"https://api.github.com/users/liewegas/repos","events_url":"https://api.github.com/users/liewegas/events{/privacy}","received_events_url":"https://api.github.com/users/liewegas/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/liewegas/ceph","description":"Ceph distributed file system","fork":true,"url":"https://api.github.com/repos/liewegas/ceph","forks_url":"https://api.github.com/repos/liewegas/ceph/forks","keys_url":"https://api.github.com/repos/liewegas/ceph/keys{/key_id}","collaborators_url":"https://api.github.com/repos/liewegas/ceph/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/liewegas/ceph/teams","hooks_url":"https://api.github.com/repos/liewegas/ceph/hooks","issue_events_url":"https://api.github.com/repos/liewegas/ceph/issues/events{/number}","events_url":"https://api.github.com/repos/liewegas/ceph/events","assignees_url":"https://api.github.com/repos/liewegas/ceph/assignees{/user}","branches_url":"https://api.github.com/repos/liewegas/ceph/branches{/branch}","tags_url":"https://api.github.com/repos/liewegas/ceph/tags","blobs_url":"https://api.github.com/repos/liewegas/ceph/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/liewegas/ceph/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/liewegas/ceph/git/refs{/sha}","trees_url":"https://api.github.com/repos/liewegas/ceph/git/trees{/sha}","statuses_url":"https://api.github.com/repos/liewegas/ceph/statuses/{sha}","languages_url":"https://api.github.com/repos/liewegas/ceph/languages","stargazers_url":"https://api.github.com/repos/liewegas/ceph/stargazers","contributors_url":"https://api.github.com/repos/liewegas/ceph/contributors","subscribers_url":"https://api.github.com/repos/liewegas/ceph/subscribers","subscription_url":"https://api.github.com/repos/liewegas/ceph/subscription","commits_url":"https://api.github.com/repos/liewegas/ceph/commits{/sha}","git_commits_url":"https://api.github.com/repos/liewegas/ceph/git/commits{/sha}","comments_url":"https://api.github.com/repos/liewegas/ceph/comments{/number}","issue_comment_url":"https://api.github.com/repos/liewegas/ceph/issues/comments{/number}","contents_url":"https://api.github.com/repos/liewegas/ceph/contents/{+path}","compare_url":"https://api.github.com/repos/liewegas/ceph/compare/{base}...{head}","merges_url":"https://api.github.com/repos/liewegas/ceph/merges","archive_url":"https://api.github.com/repos/liewegas/ceph/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/liewegas/ceph/downloads","issues_url":"https://api.github.com/repos/liewegas/ceph/issues{/number}","pulls_url":"https://api.github.com/repos/liewegas/ceph/pulls{/number}","milestones_url":"https://api.github.com/repos/liewegas/ceph/milestones{/number}","notifications_url":"https://api.github.com/repos/liewegas/ceph/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/liewegas/ceph/labels{/name}","releases_url":"https://api.github.com/repos/liewegas/ceph/releases{/id}","deployments_url":"https://api.github.com/repos/liewegas/ceph/deployments","created_at":"2011-09-21T16:14:23Z","updated_at":"2016-05-02T15:47:21Z","pushed_at":"2016-06-02T17:43:17Z","git_url":"git://github.com/liewegas/ceph.git","ssh_url":"git@github.com:liewegas/ceph.git","clone_url":"https://github.com/liewegas/ceph.git","svn_url":"https://github.com/liewegas/ceph","homepage":"http://ceph.newdream.net/","size":153937,"stargazers_count":11,"watchers_count":11,"language":"C++","has_issues":false,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":4,"mirror_url":null,"open_issues_count":1,"forks":4,"open_issues":1,"watchers":11,"default_branch":"master"}},"base":{"label":"ceph:master","ref":"master","sha":"f92e8d0762d656d4ac8b6b01fcace7a6ceeb4d36","user":{"login":"ceph","id":1015767,"avatar_url":"https://avatars.githubusercontent.com/u/1015767?v=3","gravatar_id":"","url":"https://api.github.com/users/ceph","html_url":"https://github.com/ceph","followers_url":"https://api.github.com/users/ceph/followers","following_url":"https://api.github.com/users/ceph/following{/other_user}","gists_url":"https://api.github.com/users/ceph/gists{/gist_id}","starred_url":"https://api.github.com/users/ceph/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ceph/subscriptions","organizations_url":"https://api.github.com/users/ceph/orgs","repos_url":"https://api.github.com/users/ceph/repos","events_url":"https://api.github.com/users/ceph/events{/privacy}","received_events_url":"https://api.github.com/users/ceph/received_events","type":"Organization","site_admin":false},"repo":{"id":2310495,"name":"ceph","full_name":"ceph/ceph","owner":{"login":"ceph","id":1015767,"avatar_url":"https://avatars.githubusercontent.com/u/1015767?v=3","gravatar_id":"","url":"https://api.github.com/users/ceph","html_url":"https://github.com/ceph","followers_url":"https://api.github.com/users/ceph/followers","following_url":"https://api.github.com/users/ceph/following{/other_user}","gists_url":"https://api.github.com/users/ceph/gists{/gist_id}","starred_url":"https://api.github.com/users/ceph/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ceph/subscriptions","organizations_url":"https://api.github.com/users/ceph/orgs","repos_url":"https://api.github.com/users/ceph/repos","events_url":"https://api.github.com/users/ceph/events{/privacy}","received_events_url":"https://api.github.com/users/ceph/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/ceph/ceph","description":"Ceph is a distributed object, block, and file storage platform ","fork":false,"url":"https://api.github.com/repos/ceph/ceph","forks_url":"https://api.github.com/repos/ceph/ceph/forks","keys_url":"https://api.github.com/repos/ceph/ceph/keys{/key_id}","collaborators_url":"https://api.github.com/repos/ceph/ceph/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/ceph/ceph/teams","hooks_url":"https://api.github.com/repos/ceph/ceph/hooks","issue_events_url":"https://api.github.com/repos/ceph/ceph/issues/events{/number}","events_url":"https://api.github.com/repos/ceph/ceph/events","assignees_url":"https://api.github.com/repos/ceph/ceph/assignees{/user}","branches_url":"https://api.github.com/repos/ceph/ceph/branches{/branch}","tags_url":"https://api.github.com/repos/ceph/ceph/tags","blobs_url":"https://api.github.com/repos/ceph/ceph/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/ceph/ceph/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/ceph/ceph/git/refs{/sha}","trees_url":"https://api.github.com/repos/ceph/ceph/git/trees{/sha}","statuses_url":"https://api.github.com/repos/ceph/ceph/statuses/{sha}","languages_url":"https://api.github.com/repos/ceph/ceph/languages","stargazers_url":"https://api.github.com/repos/ceph/ceph/stargazers","contributors_url":"https://api.github.com/repos/ceph/ceph/contributors","subscribers_url":"https://api.github.com/repos/ceph/ceph/subscribers","subscription_url":"https://api.github.com/repos/ceph/ceph/subscription","commits_url":"https://api.github.com/repos/ceph/ceph/commits{/sha}","git_commits_url":"https://api.github.com/repos/ceph/ceph/git/commits{/sha}","comments_url":"https://api.github.com/repos/ceph/ceph/comments{/number}","issue_comment_url":"https://api.github.com/repos/ceph/ceph/issues/comments{/number}","contents_url":"https://api.github.com/repos/ceph/ceph/contents/{+path}","compare_url":"https://api.github.com/repos/ceph/ceph/compare/{base}...{head}","merges_url":"https://api.github.com/repos/ceph/ceph/merges","archive_url":"https://api.github.com/repos/ceph/ceph/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/ceph/ceph/downloads","issues_url":"https://api.github.com/repos/ceph/ceph/issues{/number}","pulls_url":"https://api.github.com/repos/ceph/ceph/pulls{/number}","milestones_url":"https://api.github.com/repos/ceph/ceph/milestones{/number}","notifications_url":"https://api.github.com/repos/ceph/ceph/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/ceph/ceph/labels{/name}","releases_url":"https://api.github.com/repos/ceph/ceph/releases{/id}","deployments_url":"https://api.github.com/repos/ceph/ceph/deployments","created_at":"2011-09-01T21:41:26Z","updated_at":"2016-06-02T15:05:05Z","pushed_at":"2016-06-02T17:54:24Z","git_url":"git://github.com/ceph/ceph.git","ssh_url":"git@github.com:ceph/ceph.git","clone_url":"https://github.com/ceph/ceph.git","svn_url":"https://github.com/ceph/ceph","homepage":"http://ceph.com","size":216323,"stargazers_count":2207,"watchers_count":2207,"language":"C++","has_issues":false,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":1301,"mirror_url":null,"open_issues_count":371,"forks":1301,"open_issues":371,"watchers":2207,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/ceph/ceph/pulls/9461"},"html":{"href":"https://github.com/ceph/ceph/pull/9461"},"issue":{"href":"https://api.github.com/repos/ceph/ceph/issues/9461"},"comments":{"href":"https://api.github.com/repos/ceph/ceph/issues/9461/comments"},"review_comments":{"href":"https://api.github.com/repos/ceph/ceph/pulls/9461/comments"},"review_comment":{"href":"https://api.github.com/repos/ceph/ceph/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/ceph/ceph/pulls/9461/commits"},"statuses":{"href":"https://api.github.com/repos/ceph/ceph/statuses/ff64bbffa0f618663ff01ca75a375c152cd730cd"}}}}
{"actor":{"display_login":"gregsfortytwo"}}