{"action":"created","comment":{"url":"https://api.github.com/repos/jbenet/multistream/pulls/comments/60112246","id":60112246,"diff_hunk":"@@ -1,84 +1,155 @@\n-# multistream - self-describing protocol streams\n+multistream\n+===========\n \n-multistream is a format -- or simple protocol -- for disambiguating, and\n-layering streams. It is extremely simple.\n+[![](https://img.shields.io/badge/made%20by-Protocol%20Labs-blue.svg?style=flat-square)](http://ipn.io)\n+[![](https://img.shields.io/badge/project-IPFS-blue.svg?style=flat-square)](http://ipfs.io/)\n+[![](https://img.shields.io/badge/freenode-%23ipfs-blue.svg?style=flat-square)](http://webchat.freenode.net/?channels=%23ipfs)\n \n-multistream is one of the `multi-protocols`, a set of protocols that solve\n-problems with self-description.\n+> Friendly protocol multiplexing. It enables a multicodec to be negotiated between two entities.\n \n ## Motivation\n \n-To decode an incoming stream of data, a program must either (a) know the format of the data a priori, or (b) learn the format from the data itself. (a) precludes running protocols that may provide one of many kinds of formats without prior agreement on which. multistream makes (b) neat using self-description.\n+Some protocols have sub-protocols or protocol-suites. Often, these sub protocols are optional extensions. Selecting which protocol to use -- or even knowing what is available to chose from -- is not simple.\n \n-Moreover, this self-description allows straightforward layering of protocols without having to implement support in the parent (or encapsulating) one.\n+What if there was a protocol that allowed mounting or nesting other protocols, and made it easy to select which protocol to use. (This is sort of like ports, but managed at the protocol level -- not the OS -- and human readable).\n \n-## How it works - Protocol Description\n+### Protocol\n \n-A `multistream` stream MUST begin with a simple header, followed by an arbitrary stream of data for the specified protocol:\n+The actual protocol is very simple. It is a multistream protocol itself, it has a multicodec header. And it has a set of other protocols available to be used by the remote side. The remote side must enter:\n \n ```\n-<header>\n-<arbitrary-stream-data>\n+> <multicodec-header-of-multistream>\n+> <multicodec-header-for-whatever-protocol-that-we-want-to-speak>\n ```\n \n-### The header\n+for example:\n \n-The header has three parts:\n-\n-- `hdr-len` - a varint length, in bytes, for security and binary protocols.\n-- `path` - the path of the protocol in a universal namespace. UTF-8. must start with a slash.\n-- `\\n` - a newline at the end, for the benefit of text protocols **(included in hdr-len)**.\n-\n-It looks like this:\n ```\n-<hdr-len><path>\\n\n+> /ipfs/QmdRKVhvzyATs3L6dosSb6w8hKuqfZK2SyPVqcYJ5VLYa2/multistream-select/0.3.0\n+> /ipfs/QmVXZiejj3sXEmxuQxF2RjmFbEiE9w7T82xDn3uYNuhbFb/ipfs-dht/0.2.3\n ```\n \n-So the full protocol is:\n+- The `<multicodec-header-of-multistream>` ensures a protocol selection is happening.\n+- The `<multistream-header-for-whatever-protocol-is-then-selected>` hopefully describes a valid protocol listed. Otherwise we return a `na`(\"not available\") error:\n \n ```\n-<hdr-len><path>\\n\n-<arbitrary-stream-data>\n+na\\n\n+\n+# in hex (note the varint prefix = 3)\n+# 0x036e610a\n ```\n \n-For example:\n+for example:\n \n ```sh\n-# this header:\n+# open connection + send multicodec headers, inc for a protocol not available\n+> /ipfs/QmdRKVhvzyATs3L6dosSb6w8hKuqfZK2SyPVqcYJ5VLYa2/multistream-select/0.3.0\n+> /ipfs/QmVXZiejj3sXEmxuQxF2RjmFbEiE9w7T82xDn3uYNuhbFb/some-protocol-that-is-not-available\n+\n+# open connection + signal protocol not available.\n+< /ipfs/QmdRKVhvzyATs3L6dosSb6w8hKuqfZK2SyPVqcYJ5VLYa2/multistream-select/0.3.0\n+< na\n+\n+# send a selection of a valid protocol + upgrade the conn and send traffic\n+> /ipfs/QmVXZiejj3sXEmxuQxF2RjmFbEiE9w7T82xDn3uYNuhbFb/ipfs-dht/0.2.3\n+> <dht-traffic>\n+> ...\n+\n+# receive a selection of the protocol + sent traffic\n+< /ipfs/QmVXZiejj3sXEmxuQxF2RjmFbEiE9w7T82xDn3uYNuhbFb/ipfs-dht/0.2.3\n+< <dht-traffic>\n+< ...\n+```\n \n-/echo/1.0\n+#### Listing\n \n-# with hexdump -C style inspection:\n-0a 2f 65 63 68 6f 2f 31  2e 30 0a                 |./echo/1.0.|\n+It is also possible to \"list\" the available protocols. A list message is simply:\n \n-0a    - varint hdr len   10 bytes\n-2f-30 - \"/echo/1.0\"       9 bytes\n-0a    - newline           1 byte\n ```\n+ls\\n\n \n-### The protocol path\n+# in hex (note the varint prefix = 3)\n+0x036c730a\n+```\n \n-`multistream` allows us to specify different protocols in a universal namespace, that way being able to recognize, multiplex, and embed them easily. We use the notion of a `path` instead of an `id` because it is meant to be a Unix-friendly URI.\n+So a remote side asking for a protocol listing would look like this:\n \n-A good path name should be decipherable -- meaning that if some machine or developer -- who has no idea about your protocol -- encounters the path string, they should be able to look it up and resolve how to use it.\n+```sh\n+# request\n+<multistream-header-for-multistream-select>\n+ls\\n\n+\n+# response\n+# TODO: maybe include a varint number of protocols here ?","path":"README.md","position":117,"original_position":117,"commit_id":"225ec351eef267d2ae5f5a153ace34aec4c6416f","original_commit_id":"225ec351eef267d2ae5f5a153ace34aec4c6416f","user":{"login":"jbenet","id":138401,"avatar_url":"https://avatars.githubusercontent.com/u/138401?v=3","gravatar_id":"","url":"https://api.github.com/users/jbenet","html_url":"https://github.com/jbenet","followers_url":"https://api.github.com/users/jbenet/followers","following_url":"https://api.github.com/users/jbenet/following{/other_user}","gists_url":"https://api.github.com/users/jbenet/gists{/gist_id}","starred_url":"https://api.github.com/users/jbenet/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jbenet/subscriptions","organizations_url":"https://api.github.com/users/jbenet/orgs","repos_url":"https://api.github.com/users/jbenet/repos","events_url":"https://api.github.com/users/jbenet/events{/privacy}","received_events_url":"https://api.github.com/users/jbenet/received_events","type":"User","site_admin":false},"body":"the total size is useful for buffer allocation. let's do that (final decisions):\r\n\r\n```\r\n<varint-total-response-size-in-bytes><varint-number-of-protocols>\\n\r\n<protocols>\r\n```\r\nthink of this as a pascal-strings style array (which is good selfdescription). We want to show the total buffer size AND the number of entries. knowing both up-front matters for perf and easier parser construction. It's safer at the cost of (on avg) one more byte.","created_at":"2016-04-18T18:43:53Z","updated_at":"2016-04-18T18:43:53Z","html_url":"https://github.com/jbenet/multistream/pull/7#discussion_r60112246","pull_request_url":"https://api.github.com/repos/jbenet/multistream/pulls/7","_links":{"self":{"href":"https://api.github.com/repos/jbenet/multistream/pulls/comments/60112246"},"html":{"href":"https://github.com/jbenet/multistream/pull/7#discussion_r60112246"},"pull_request":{"href":"https://api.github.com/repos/jbenet/multistream/pulls/7"}}},"pull_request":{"url":"https://api.github.com/repos/jbenet/multistream/pulls/7","id":66114809,"html_url":"https://github.com/jbenet/multistream/pull/7","diff_url":"https://github.com/jbenet/multistream/pull/7.diff","patch_url":"https://github.com/jbenet/multistream/pull/7.patch","issue_url":"https://api.github.com/repos/jbenet/multistream/issues/7","number":7,"state":"open","locked":false,"title":"bubble the spec up","user":{"login":"diasdavid","id":1211152,"avatar_url":"https://avatars.githubusercontent.com/u/1211152?v=3","gravatar_id":"","url":"https://api.github.com/users/diasdavid","html_url":"https://github.com/diasdavid","followers_url":"https://api.github.com/users/diasdavid/followers","following_url":"https://api.github.com/users/diasdavid/following{/other_user}","gists_url":"https://api.github.com/users/diasdavid/gists{/gist_id}","starred_url":"https://api.github.com/users/diasdavid/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/diasdavid/subscriptions","organizations_url":"https://api.github.com/users/diasdavid/orgs","repos_url":"https://api.github.com/users/diasdavid/repos","events_url":"https://api.github.com/users/diasdavid/events{/privacy}","received_events_url":"https://api.github.com/users/diasdavid/received_events","type":"User","site_admin":false},"body":"as suggested on https://github.com/jbenet/multistream/pull/3#issuecomment-139106688","created_at":"2016-04-12T07:56:44Z","updated_at":"2016-04-18T18:43:54Z","closed_at":null,"merged_at":null,"merge_commit_sha":"6972d4a3adb66d58d02bc7202a598a12647433f8","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/jbenet/multistream/pulls/7/commits","review_comments_url":"https://api.github.com/repos/jbenet/multistream/pulls/7/comments","review_comment_url":"https://api.github.com/repos/jbenet/multistream/pulls/comments{/number}","comments_url":"https://api.github.com/repos/jbenet/multistream/issues/7/comments","statuses_url":"https://api.github.com/repos/jbenet/multistream/statuses/225ec351eef267d2ae5f5a153ace34aec4c6416f","head":{"label":"jbenet:bubble-spec-up","ref":"bubble-spec-up","sha":"225ec351eef267d2ae5f5a153ace34aec4c6416f","user":{"login":"jbenet","id":138401,"avatar_url":"https://avatars.githubusercontent.com/u/138401?v=3","gravatar_id":"","url":"https://api.github.com/users/jbenet","html_url":"https://github.com/jbenet","followers_url":"https://api.github.com/users/jbenet/followers","following_url":"https://api.github.com/users/jbenet/following{/other_user}","gists_url":"https://api.github.com/users/jbenet/gists{/gist_id}","starred_url":"https://api.github.com/users/jbenet/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jbenet/subscriptions","organizations_url":"https://api.github.com/users/jbenet/orgs","repos_url":"https://api.github.com/users/jbenet/repos","events_url":"https://api.github.com/users/jbenet/events{/privacy}","received_events_url":"https://api.github.com/users/jbenet/received_events","type":"User","site_admin":false},"repo":{"id":36754903,"name":"multistream","full_name":"jbenet/multistream","owner":{"login":"jbenet","id":138401,"avatar_url":"https://avatars.githubusercontent.com/u/138401?v=3","gravatar_id":"","url":"https://api.github.com/users/jbenet","html_url":"https://github.com/jbenet","followers_url":"https://api.github.com/users/jbenet/followers","following_url":"https://api.github.com/users/jbenet/following{/other_user}","gists_url":"https://api.github.com/users/jbenet/gists{/gist_id}","starred_url":"https://api.github.com/users/jbenet/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jbenet/subscriptions","organizations_url":"https://api.github.com/users/jbenet/orgs","repos_url":"https://api.github.com/users/jbenet/repos","events_url":"https://api.github.com/users/jbenet/events{/privacy}","received_events_url":"https://api.github.com/users/jbenet/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/jbenet/multistream","description":"self-describing protocol streams","fork":false,"url":"https://api.github.com/repos/jbenet/multistream","forks_url":"https://api.github.com/repos/jbenet/multistream/forks","keys_url":"https://api.github.com/repos/jbenet/multistream/keys{/key_id}","collaborators_url":"https://api.github.com/repos/jbenet/multistream/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/jbenet/multistream/teams","hooks_url":"https://api.github.com/repos/jbenet/multistream/hooks","issue_events_url":"https://api.github.com/repos/jbenet/multistream/issues/events{/number}","events_url":"https://api.github.com/repos/jbenet/multistream/events","assignees_url":"https://api.github.com/repos/jbenet/multistream/assignees{/user}","branches_url":"https://api.github.com/repos/jbenet/multistream/branches{/branch}","tags_url":"https://api.github.com/repos/jbenet/multistream/tags","blobs_url":"https://api.github.com/repos/jbenet/multistream/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/jbenet/multistream/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/jbenet/multistream/git/refs{/sha}","trees_url":"https://api.github.com/repos/jbenet/multistream/git/trees{/sha}","statuses_url":"https://api.github.com/repos/jbenet/multistream/statuses/{sha}","languages_url":"https://api.github.com/repos/jbenet/multistream/languages","stargazers_url":"https://api.github.com/repos/jbenet/multistream/stargazers","contributors_url":"https://api.github.com/repos/jbenet/multistream/contributors","subscribers_url":"https://api.github.com/repos/jbenet/multistream/subscribers","subscription_url":"https://api.github.com/repos/jbenet/multistream/subscription","commits_url":"https://api.github.com/repos/jbenet/multistream/commits{/sha}","git_commits_url":"https://api.github.com/repos/jbenet/multistream/git/commits{/sha}","comments_url":"https://api.github.com/repos/jbenet/multistream/comments{/number}","issue_comment_url":"https://api.github.com/repos/jbenet/multistream/issues/comments{/number}","contents_url":"https://api.github.com/repos/jbenet/multistream/contents/{+path}","compare_url":"https://api.github.com/repos/jbenet/multistream/compare/{base}...{head}","merges_url":"https://api.github.com/repos/jbenet/multistream/merges","archive_url":"https://api.github.com/repos/jbenet/multistream/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/jbenet/multistream/downloads","issues_url":"https://api.github.com/repos/jbenet/multistream/issues{/number}","pulls_url":"https://api.github.com/repos/jbenet/multistream/pulls{/number}","milestones_url":"https://api.github.com/repos/jbenet/multistream/milestones{/number}","notifications_url":"https://api.github.com/repos/jbenet/multistream/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/jbenet/multistream/labels{/name}","releases_url":"https://api.github.com/repos/jbenet/multistream/releases{/id}","deployments_url":"https://api.github.com/repos/jbenet/multistream/deployments","created_at":"2015-06-02T18:57:54Z","updated_at":"2016-04-18T15:42:09Z","pushed_at":"2016-04-12T07:59:45Z","git_url":"git://github.com/jbenet/multistream.git","ssh_url":"git@github.com:jbenet/multistream.git","clone_url":"https://github.com/jbenet/multistream.git","svn_url":"https://github.com/jbenet/multistream","homepage":"","size":14,"stargazers_count":15,"watchers_count":15,"language":null,"has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":3,"mirror_url":null,"open_issues_count":3,"forks":3,"open_issues":3,"watchers":15,"default_branch":"master"}},"base":{"label":"jbenet:master","ref":"master","sha":"81d80ef97384892674c0a7ededcf484f5a9c8f7d","user":{"login":"jbenet","id":138401,"avatar_url":"https://avatars.githubusercontent.com/u/138401?v=3","gravatar_id":"","url":"https://api.github.com/users/jbenet","html_url":"https://github.com/jbenet","followers_url":"https://api.github.com/users/jbenet/followers","following_url":"https://api.github.com/users/jbenet/following{/other_user}","gists_url":"https://api.github.com/users/jbenet/gists{/gist_id}","starred_url":"https://api.github.com/users/jbenet/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jbenet/subscriptions","organizations_url":"https://api.github.com/users/jbenet/orgs","repos_url":"https://api.github.com/users/jbenet/repos","events_url":"https://api.github.com/users/jbenet/events{/privacy}","received_events_url":"https://api.github.com/users/jbenet/received_events","type":"User","site_admin":false},"repo":{"id":36754903,"name":"multistream","full_name":"jbenet/multistream","owner":{"login":"jbenet","id":138401,"avatar_url":"https://avatars.githubusercontent.com/u/138401?v=3","gravatar_id":"","url":"https://api.github.com/users/jbenet","html_url":"https://github.com/jbenet","followers_url":"https://api.github.com/users/jbenet/followers","following_url":"https://api.github.com/users/jbenet/following{/other_user}","gists_url":"https://api.github.com/users/jbenet/gists{/gist_id}","starred_url":"https://api.github.com/users/jbenet/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jbenet/subscriptions","organizations_url":"https://api.github.com/users/jbenet/orgs","repos_url":"https://api.github.com/users/jbenet/repos","events_url":"https://api.github.com/users/jbenet/events{/privacy}","received_events_url":"https://api.github.com/users/jbenet/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/jbenet/multistream","description":"self-describing protocol streams","fork":false,"url":"https://api.github.com/repos/jbenet/multistream","forks_url":"https://api.github.com/repos/jbenet/multistream/forks","keys_url":"https://api.github.com/repos/jbenet/multistream/keys{/key_id}","collaborators_url":"https://api.github.com/repos/jbenet/multistream/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/jbenet/multistream/teams","hooks_url":"https://api.github.com/repos/jbenet/multistream/hooks","issue_events_url":"https://api.github.com/repos/jbenet/multistream/issues/events{/number}","events_url":"https://api.github.com/repos/jbenet/multistream/events","assignees_url":"https://api.github.com/repos/jbenet/multistream/assignees{/user}","branches_url":"https://api.github.com/repos/jbenet/multistream/branches{/branch}","tags_url":"https://api.github.com/repos/jbenet/multistream/tags","blobs_url":"https://api.github.com/repos/jbenet/multistream/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/jbenet/multistream/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/jbenet/multistream/git/refs{/sha}","trees_url":"https://api.github.com/repos/jbenet/multistream/git/trees{/sha}","statuses_url":"https://api.github.com/repos/jbenet/multistream/statuses/{sha}","languages_url":"https://api.github.com/repos/jbenet/multistream/languages","stargazers_url":"https://api.github.com/repos/jbenet/multistream/stargazers","contributors_url":"https://api.github.com/repos/jbenet/multistream/contributors","subscribers_url":"https://api.github.com/repos/jbenet/multistream/subscribers","subscription_url":"https://api.github.com/repos/jbenet/multistream/subscription","commits_url":"https://api.github.com/repos/jbenet/multistream/commits{/sha}","git_commits_url":"https://api.github.com/repos/jbenet/multistream/git/commits{/sha}","comments_url":"https://api.github.com/repos/jbenet/multistream/comments{/number}","issue_comment_url":"https://api.github.com/repos/jbenet/multistream/issues/comments{/number}","contents_url":"https://api.github.com/repos/jbenet/multistream/contents/{+path}","compare_url":"https://api.github.com/repos/jbenet/multistream/compare/{base}...{head}","merges_url":"https://api.github.com/repos/jbenet/multistream/merges","archive_url":"https://api.github.com/repos/jbenet/multistream/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/jbenet/multistream/downloads","issues_url":"https://api.github.com/repos/jbenet/multistream/issues{/number}","pulls_url":"https://api.github.com/repos/jbenet/multistream/pulls{/number}","milestones_url":"https://api.github.com/repos/jbenet/multistream/milestones{/number}","notifications_url":"https://api.github.com/repos/jbenet/multistream/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/jbenet/multistream/labels{/name}","releases_url":"https://api.github.com/repos/jbenet/multistream/releases{/id}","deployments_url":"https://api.github.com/repos/jbenet/multistream/deployments","created_at":"2015-06-02T18:57:54Z","updated_at":"2016-04-18T15:42:09Z","pushed_at":"2016-04-12T07:59:45Z","git_url":"git://github.com/jbenet/multistream.git","ssh_url":"git@github.com:jbenet/multistream.git","clone_url":"https://github.com/jbenet/multistream.git","svn_url":"https://github.com/jbenet/multistream","homepage":"","size":14,"stargazers_count":15,"watchers_count":15,"language":null,"has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":3,"mirror_url":null,"open_issues_count":3,"forks":3,"open_issues":3,"watchers":15,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/jbenet/multistream/pulls/7"},"html":{"href":"https://github.com/jbenet/multistream/pull/7"},"issue":{"href":"https://api.github.com/repos/jbenet/multistream/issues/7"},"comments":{"href":"https://api.github.com/repos/jbenet/multistream/issues/7/comments"},"review_comments":{"href":"https://api.github.com/repos/jbenet/multistream/pulls/7/comments"},"review_comment":{"href":"https://api.github.com/repos/jbenet/multistream/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/jbenet/multistream/pulls/7/commits"},"statuses":{"href":"https://api.github.com/repos/jbenet/multistream/statuses/225ec351eef267d2ae5f5a153ace34aec4c6416f"}}}}