{"action":"created","comment":{"url":"https://api.github.com/repos/tikv/head-first-tikv/pulls/comments/215598873","pull_request_review_id":152838527,"id":215598873,"node_id":"MDI0OlB1bGxSZXF1ZXN0UmV2aWV3Q29tbWVudDIxNTU5ODg3Mw==","diff_hunk":"@@ -0,0 +1,37 @@\n+# Introduction\n+\n+[TiKV](https://github.com/tikv/tikv) is a distributed, transactional key-value database. It has been widely used in the world, especially in many critical production environments. It has also been adopted by CNCF as a [sandbox project](https://www.cncf.io/blog/2018/08/28/cncf-to-host-tikv-in-the-sandbox/).\n+\n+TiKV provides a fully [ACID](https://en.wikipedia.org/wiki/ACID_(computer_science)) support, automatic horizontal scalability, global data consistency, Geo-replication, and many other useful features, it can be used as a building block to build other high-level services. E.g, we have already used TiKV to support [TiDB](https://github.com/pingcap/tidb) - a next-generation [HTAP](https://en.wikipedia.org/wiki/Hybrid_transactional/analytical_processing_(HTAP)) database. \n+\n+In this book, we will give you an introduction of TiKV. You can know why we build it, what problems do we meet, which technical do we choose, etc. We hope through this book, you can have a deep understanding of TiKV, learn enough knowledge of distributed programming, or even start to build your own distributed system. :-)\n+\n+## History \n+\n+In the middle of 2015, we wanted to build a database which solved the scaling problem of MySQL. At that time, the most used way was to build a proxy on top of the MySQL servers, we didn't think proxy was an elegant way (Definitely, we still don't think either now). \n+\n+As far as we knew, the proxy had following problems:\n+\n++ The proxy can't satisfy a fully ACID compliance easily, especially when the transaction crosses different MySQL servers.\n++ The users still need to care the data distribution and design their sharding key carefully to avoid inefficient query.\n++ The high availability and data consistency of MySQL can't be guaranteed easily based on the traditional Master-Salve replication. \n+\n+Although building a proxy based on MySQL directly might be easy at the beginning, we still decided to choose another way, a more hardcore way - to build a distributed, MySQL compatible database from scratch.  \n+\n+Fortunately, Google met the same problem and had already published some papers to describe how they built [Spanner](http://static.googleusercontent.com/media/research.google.com/en//archive/spanner-osdi2012.pdf) and [F1](https://storage.googleapis.com/pub-tools-public-publication-data/pdf/41344.pdf) to solve it. Spanner is a globally distributed, externally consistent database and F1 is a distributed SQL database based on Spanner. Refer to Spanner and F1, we knew we could do the same thing. So we started to build TiDB - a stateless MySQL layer like F1. After we released TiDB, we knew we need a Spanner-like database, then we began to develop TiKV. \n+\n+## Architecture\n+\n+Following is the architecture of TiKV: \n+\n+![Architecture](./architecture.png)\n+\n+According to the picture, there are three TiKV instances in the cluster, each instance uses one [RocksDB](https://github.com/facebook/rocksdb) to save data. Beyond the RocksDB, we use [Raft](https://raft.github.io/) consensus algorithm to replica the data. Mostly, we should use at lease three replicas to keep data safety and consistence, and these replicas form a Raft group. \n+\n+We use the traditional [MVCC](https://en.wikipedia.org/wiki/Multiversion_concurrency_control) mechanism and build a distributed transaction layer above the Raft layer, we also provide a coprocessor framework to let the users push down their computing logic to the storage layer and let TiKV calculate directly and return the result. \n+\n+All the network communications are through [gRPC](https://grpc.io/) to let the contributor develop their own clients easily.  \n+\n+The whole cluster is managed and scheduled by a center service - [placement driver(PD)](https://github.com/pingcap/pd). ","path":"overview/introduction.md","position":35,"original_position":35,"commit_id":"f506efeb4a1396bd0180c019b1b87bbf9605f198","original_commit_id":"f506efeb4a1396bd0180c019b1b87bbf9605f198","user":{"login":"queenypingcap","id":19528243,"node_id":"MDQ6VXNlcjE5NTI4MjQz","avatar_url":"https://avatars2.githubusercontent.com/u/19528243?v=4","gravatar_id":"","url":"https://api.github.com/users/queenypingcap","html_url":"https://github.com/queenypingcap","followers_url":"https://api.github.com/users/queenypingcap/followers","following_url":"https://api.github.com/users/queenypingcap/following{/other_user}","gists_url":"https://api.github.com/users/queenypingcap/gists{/gist_id}","starred_url":"https://api.github.com/users/queenypingcap/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/queenypingcap/subscriptions","organizations_url":"https://api.github.com/users/queenypingcap/orgs","repos_url":"https://api.github.com/users/queenypingcap/repos","events_url":"https://api.github.com/users/queenypingcap/events{/privacy}","received_events_url":"https://api.github.com/users/queenypingcap/received_events","type":"User","site_admin":false},"body":"1. a center service -> a central service\r\n2. placement driver(PD) -> Placement Driver (PD)\r\n","created_at":"2018-09-06T12:01:42Z","updated_at":"2018-09-06T12:01:56Z","html_url":"https://github.com/tikv/head-first-tikv/pull/2#discussion_r215598873","pull_request_url":"https://api.github.com/repos/tikv/head-first-tikv/pulls/2","author_association":"NONE","_links":{"self":{"href":"https://api.github.com/repos/tikv/head-first-tikv/pulls/comments/215598873"},"html":{"href":"https://github.com/tikv/head-first-tikv/pull/2#discussion_r215598873"},"pull_request":{"href":"https://api.github.com/repos/tikv/head-first-tikv/pulls/2"}}},"pull_request":{"url":"https://api.github.com/repos/tikv/head-first-tikv/pulls/2","id":212666130,"node_id":"MDExOlB1bGxSZXF1ZXN0MjEyNjY2MTMw","html_url":"https://github.com/tikv/head-first-tikv/pull/2","diff_url":"https://github.com/tikv/head-first-tikv/pull/2.diff","patch_url":"https://github.com/tikv/head-first-tikv/pull/2.patch","issue_url":"https://api.github.com/repos/tikv/head-first-tikv/issues/2","number":2,"state":"open","locked":false,"title":"add readme and overview","user":{"login":"siddontang","id":1080370,"node_id":"MDQ6VXNlcjEwODAzNzA=","avatar_url":"https://avatars2.githubusercontent.com/u/1080370?v=4","gravatar_id":"","url":"https://api.github.com/users/siddontang","html_url":"https://github.com/siddontang","followers_url":"https://api.github.com/users/siddontang/followers","following_url":"https://api.github.com/users/siddontang/following{/other_user}","gists_url":"https://api.github.com/users/siddontang/gists{/gist_id}","starred_url":"https://api.github.com/users/siddontang/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/siddontang/subscriptions","organizations_url":"https://api.github.com/users/siddontang/orgs","repos_url":"https://api.github.com/users/siddontang/repos","events_url":"https://api.github.com/users/siddontang/events{/privacy}","received_events_url":"https://api.github.com/users/siddontang/received_events","type":"User","site_admin":false},"body":"Signed-off-by: siddontang <siddontang@gmail.com>\r\n\r\nPTAL @Hoverbear @tikv/pingcap-i18n ","created_at":"2018-09-03T07:48:15Z","updated_at":"2018-09-06T12:01:56Z","closed_at":null,"merged_at":null,"merge_commit_sha":"c61363229514aa93e7c1616f9e31d30dbf542cc9","assignee":null,"assignees":[],"requested_reviewers":[],"requested_teams":[],"labels":[],"milestone":null,"commits_url":"https://api.github.com/repos/tikv/head-first-tikv/pulls/2/commits","review_comments_url":"https://api.github.com/repos/tikv/head-first-tikv/pulls/2/comments","review_comment_url":"https://api.github.com/repos/tikv/head-first-tikv/pulls/comments{/number}","comments_url":"https://api.github.com/repos/tikv/head-first-tikv/issues/2/comments","statuses_url":"https://api.github.com/repos/tikv/head-first-tikv/statuses/f506efeb4a1396bd0180c019b1b87bbf9605f198","head":{"label":"tikv:siddontang/overview","ref":"siddontang/overview","sha":"f506efeb4a1396bd0180c019b1b87bbf9605f198","user":{"login":"tikv","id":41004122,"node_id":"MDEyOk9yZ2FuaXphdGlvbjQxMDA0MTIy","avatar_url":"https://avatars2.githubusercontent.com/u/41004122?v=4","gravatar_id":"","url":"https://api.github.com/users/tikv","html_url":"https://github.com/tikv","followers_url":"https://api.github.com/users/tikv/followers","following_url":"https://api.github.com/users/tikv/following{/other_user}","gists_url":"https://api.github.com/users/tikv/gists{/gist_id}","starred_url":"https://api.github.com/users/tikv/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tikv/subscriptions","organizations_url":"https://api.github.com/users/tikv/orgs","repos_url":"https://api.github.com/users/tikv/repos","events_url":"https://api.github.com/users/tikv/events{/privacy}","received_events_url":"https://api.github.com/users/tikv/received_events","type":"Organization","site_admin":false},"repo":{"id":146400837,"node_id":"MDEwOlJlcG9zaXRvcnkxNDY0MDA4Mzc=","name":"head-first-tikv","full_name":"tikv/head-first-tikv","owner":{"login":"tikv","id":41004122,"node_id":"MDEyOk9yZ2FuaXphdGlvbjQxMDA0MTIy","avatar_url":"https://avatars2.githubusercontent.com/u/41004122?v=4","gravatar_id":"","url":"https://api.github.com/users/tikv","html_url":"https://github.com/tikv","followers_url":"https://api.github.com/users/tikv/followers","following_url":"https://api.github.com/users/tikv/following{/other_user}","gists_url":"https://api.github.com/users/tikv/gists{/gist_id}","starred_url":"https://api.github.com/users/tikv/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tikv/subscriptions","organizations_url":"https://api.github.com/users/tikv/orgs","repos_url":"https://api.github.com/users/tikv/repos","events_url":"https://api.github.com/users/tikv/events{/privacy}","received_events_url":"https://api.github.com/users/tikv/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/tikv/head-first-tikv","description":"How do we build a distributed, transactional key-value database - TiKV?","fork":false,"url":"https://api.github.com/repos/tikv/head-first-tikv","forks_url":"https://api.github.com/repos/tikv/head-first-tikv/forks","keys_url":"https://api.github.com/repos/tikv/head-first-tikv/keys{/key_id}","collaborators_url":"https://api.github.com/repos/tikv/head-first-tikv/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/tikv/head-first-tikv/teams","hooks_url":"https://api.github.com/repos/tikv/head-first-tikv/hooks","issue_events_url":"https://api.github.com/repos/tikv/head-first-tikv/issues/events{/number}","events_url":"https://api.github.com/repos/tikv/head-first-tikv/events","assignees_url":"https://api.github.com/repos/tikv/head-first-tikv/assignees{/user}","branches_url":"https://api.github.com/repos/tikv/head-first-tikv/branches{/branch}","tags_url":"https://api.github.com/repos/tikv/head-first-tikv/tags","blobs_url":"https://api.github.com/repos/tikv/head-first-tikv/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/tikv/head-first-tikv/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/tikv/head-first-tikv/git/refs{/sha}","trees_url":"https://api.github.com/repos/tikv/head-first-tikv/git/trees{/sha}","statuses_url":"https://api.github.com/repos/tikv/head-first-tikv/statuses/{sha}","languages_url":"https://api.github.com/repos/tikv/head-first-tikv/languages","stargazers_url":"https://api.github.com/repos/tikv/head-first-tikv/stargazers","contributors_url":"https://api.github.com/repos/tikv/head-first-tikv/contributors","subscribers_url":"https://api.github.com/repos/tikv/head-first-tikv/subscribers","subscription_url":"https://api.github.com/repos/tikv/head-first-tikv/subscription","commits_url":"https://api.github.com/repos/tikv/head-first-tikv/commits{/sha}","git_commits_url":"https://api.github.com/repos/tikv/head-first-tikv/git/commits{/sha}","comments_url":"https://api.github.com/repos/tikv/head-first-tikv/comments{/number}","issue_comment_url":"https://api.github.com/repos/tikv/head-first-tikv/issues/comments{/number}","contents_url":"https://api.github.com/repos/tikv/head-first-tikv/contents/{+path}","compare_url":"https://api.github.com/repos/tikv/head-first-tikv/compare/{base}...{head}","merges_url":"https://api.github.com/repos/tikv/head-first-tikv/merges","archive_url":"https://api.github.com/repos/tikv/head-first-tikv/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/tikv/head-first-tikv/downloads","issues_url":"https://api.github.com/repos/tikv/head-first-tikv/issues{/number}","pulls_url":"https://api.github.com/repos/tikv/head-first-tikv/pulls{/number}","milestones_url":"https://api.github.com/repos/tikv/head-first-tikv/milestones{/number}","notifications_url":"https://api.github.com/repos/tikv/head-first-tikv/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/tikv/head-first-tikv/labels{/name}","releases_url":"https://api.github.com/repos/tikv/head-first-tikv/releases{/id}","deployments_url":"https://api.github.com/repos/tikv/head-first-tikv/deployments","created_at":"2018-08-28T06:18:56Z","updated_at":"2018-09-06T09:48:42Z","pushed_at":"2018-09-03T07:48:15Z","git_url":"git://github.com/tikv/head-first-tikv.git","ssh_url":"git@github.com:tikv/head-first-tikv.git","clone_url":"https://github.com/tikv/head-first-tikv.git","svn_url":"https://github.com/tikv/head-first-tikv","homepage":null,"size":48,"stargazers_count":20,"watchers_count":20,"language":null,"has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":1,"mirror_url":null,"archived":false,"open_issues_count":2,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0","node_id":"MDc6TGljZW5zZTI="},"forks":1,"open_issues":2,"watchers":20,"default_branch":"master"}},"base":{"label":"tikv:master","ref":"master","sha":"50a78292b3fc97c3356771a327058067397f2607","user":{"login":"tikv","id":41004122,"node_id":"MDEyOk9yZ2FuaXphdGlvbjQxMDA0MTIy","avatar_url":"https://avatars2.githubusercontent.com/u/41004122?v=4","gravatar_id":"","url":"https://api.github.com/users/tikv","html_url":"https://github.com/tikv","followers_url":"https://api.github.com/users/tikv/followers","following_url":"https://api.github.com/users/tikv/following{/other_user}","gists_url":"https://api.github.com/users/tikv/gists{/gist_id}","starred_url":"https://api.github.com/users/tikv/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tikv/subscriptions","organizations_url":"https://api.github.com/users/tikv/orgs","repos_url":"https://api.github.com/users/tikv/repos","events_url":"https://api.github.com/users/tikv/events{/privacy}","received_events_url":"https://api.github.com/users/tikv/received_events","type":"Organization","site_admin":false},"repo":{"id":146400837,"node_id":"MDEwOlJlcG9zaXRvcnkxNDY0MDA4Mzc=","name":"head-first-tikv","full_name":"tikv/head-first-tikv","owner":{"login":"tikv","id":41004122,"node_id":"MDEyOk9yZ2FuaXphdGlvbjQxMDA0MTIy","avatar_url":"https://avatars2.githubusercontent.com/u/41004122?v=4","gravatar_id":"","url":"https://api.github.com/users/tikv","html_url":"https://github.com/tikv","followers_url":"https://api.github.com/users/tikv/followers","following_url":"https://api.github.com/users/tikv/following{/other_user}","gists_url":"https://api.github.com/users/tikv/gists{/gist_id}","starred_url":"https://api.github.com/users/tikv/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tikv/subscriptions","organizations_url":"https://api.github.com/users/tikv/orgs","repos_url":"https://api.github.com/users/tikv/repos","events_url":"https://api.github.com/users/tikv/events{/privacy}","received_events_url":"https://api.github.com/users/tikv/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/tikv/head-first-tikv","description":"How do we build a distributed, transactional key-value database - TiKV?","fork":false,"url":"https://api.github.com/repos/tikv/head-first-tikv","forks_url":"https://api.github.com/repos/tikv/head-first-tikv/forks","keys_url":"https://api.github.com/repos/tikv/head-first-tikv/keys{/key_id}","collaborators_url":"https://api.github.com/repos/tikv/head-first-tikv/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/tikv/head-first-tikv/teams","hooks_url":"https://api.github.com/repos/tikv/head-first-tikv/hooks","issue_events_url":"https://api.github.com/repos/tikv/head-first-tikv/issues/events{/number}","events_url":"https://api.github.com/repos/tikv/head-first-tikv/events","assignees_url":"https://api.github.com/repos/tikv/head-first-tikv/assignees{/user}","branches_url":"https://api.github.com/repos/tikv/head-first-tikv/branches{/branch}","tags_url":"https://api.github.com/repos/tikv/head-first-tikv/tags","blobs_url":"https://api.github.com/repos/tikv/head-first-tikv/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/tikv/head-first-tikv/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/tikv/head-first-tikv/git/refs{/sha}","trees_url":"https://api.github.com/repos/tikv/head-first-tikv/git/trees{/sha}","statuses_url":"https://api.github.com/repos/tikv/head-first-tikv/statuses/{sha}","languages_url":"https://api.github.com/repos/tikv/head-first-tikv/languages","stargazers_url":"https://api.github.com/repos/tikv/head-first-tikv/stargazers","contributors_url":"https://api.github.com/repos/tikv/head-first-tikv/contributors","subscribers_url":"https://api.github.com/repos/tikv/head-first-tikv/subscribers","subscription_url":"https://api.github.com/repos/tikv/head-first-tikv/subscription","commits_url":"https://api.github.com/repos/tikv/head-first-tikv/commits{/sha}","git_commits_url":"https://api.github.com/repos/tikv/head-first-tikv/git/commits{/sha}","comments_url":"https://api.github.com/repos/tikv/head-first-tikv/comments{/number}","issue_comment_url":"https://api.github.com/repos/tikv/head-first-tikv/issues/comments{/number}","contents_url":"https://api.github.com/repos/tikv/head-first-tikv/contents/{+path}","compare_url":"https://api.github.com/repos/tikv/head-first-tikv/compare/{base}...{head}","merges_url":"https://api.github.com/repos/tikv/head-first-tikv/merges","archive_url":"https://api.github.com/repos/tikv/head-first-tikv/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/tikv/head-first-tikv/downloads","issues_url":"https://api.github.com/repos/tikv/head-first-tikv/issues{/number}","pulls_url":"https://api.github.com/repos/tikv/head-first-tikv/pulls{/number}","milestones_url":"https://api.github.com/repos/tikv/head-first-tikv/milestones{/number}","notifications_url":"https://api.github.com/repos/tikv/head-first-tikv/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/tikv/head-first-tikv/labels{/name}","releases_url":"https://api.github.com/repos/tikv/head-first-tikv/releases{/id}","deployments_url":"https://api.github.com/repos/tikv/head-first-tikv/deployments","created_at":"2018-08-28T06:18:56Z","updated_at":"2018-09-06T09:48:42Z","pushed_at":"2018-09-03T07:48:15Z","git_url":"git://github.com/tikv/head-first-tikv.git","ssh_url":"git@github.com:tikv/head-first-tikv.git","clone_url":"https://github.com/tikv/head-first-tikv.git","svn_url":"https://github.com/tikv/head-first-tikv","homepage":null,"size":48,"stargazers_count":20,"watchers_count":20,"language":null,"has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":1,"mirror_url":null,"archived":false,"open_issues_count":2,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0","node_id":"MDc6TGljZW5zZTI="},"forks":1,"open_issues":2,"watchers":20,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/tikv/head-first-tikv/pulls/2"},"html":{"href":"https://github.com/tikv/head-first-tikv/pull/2"},"issue":{"href":"https://api.github.com/repos/tikv/head-first-tikv/issues/2"},"comments":{"href":"https://api.github.com/repos/tikv/head-first-tikv/issues/2/comments"},"review_comments":{"href":"https://api.github.com/repos/tikv/head-first-tikv/pulls/2/comments"},"review_comment":{"href":"https://api.github.com/repos/tikv/head-first-tikv/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/tikv/head-first-tikv/pulls/2/commits"},"statuses":{"href":"https://api.github.com/repos/tikv/head-first-tikv/statuses/f506efeb4a1396bd0180c019b1b87bbf9605f198"}},"author_association":"CONTRIBUTOR"}}
{"actor":{"display_login":"queenypingcap"}}