{"action":"created","issue":{"url":"https://api.github.com/repos/baidu/tera/issues/782","repository_url":"https://api.github.com/repos/baidu/tera","labels_url":"https://api.github.com/repos/baidu/tera/issues/782/labels{/name}","comments_url":"https://api.github.com/repos/baidu/tera/issues/782/comments","events_url":"https://api.github.com/repos/baidu/tera/issues/782/events","html_url":"https://github.com/baidu/tera/issues/782","id":141199507,"number":782,"title":"内存compact表格存在读写一致性漏洞","user":{"login":"xupeilin","id":11419707,"avatar_url":"https://avatars.githubusercontent.com/u/11419707?v=3","gravatar_id":"","url":"https://api.github.com/users/xupeilin","html_url":"https://github.com/xupeilin","followers_url":"https://api.github.com/users/xupeilin/followers","following_url":"https://api.github.com/users/xupeilin/following{/other_user}","gists_url":"https://api.github.com/users/xupeilin/gists{/gist_id}","starred_url":"https://api.github.com/users/xupeilin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/xupeilin/subscriptions","organizations_url":"https://api.github.com/users/xupeilin/orgs","repos_url":"https://api.github.com/users/xupeilin/repos","events_url":"https://api.github.com/users/xupeilin/events{/privacy}","received_events_url":"https://api.github.com/users/xupeilin/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2016-03-16T07:52:34Z","updated_at":"2016-03-29T09:23:20Z","closed_at":null,"body":"现象：\r\n  * 很小概率读到旧数据，高频更新+读（每秒千次以上）的列才会出现\r\n  * 相同测试case在非内存compact表上不复现\r\n\r\n原因：\r\n  * tera保证读写一致，含义为：\r\n   * 单行更新原子性：\r\n    * 列更新的过程中，整行commit完成之前，新数据不可见，旧数据可读\r\n    * 实现为：更新过程中持有一个临时快照（ms级），保证新版本数据生效前旧数据不被删除\r\n   * 数据读取原子性：\r\n    * 读取过程中发生的更新，对本次读取不生效，返回更新前数据\r\n    * 实现为：读取过程中持有一个临时快照（ms级），保护旧数据，并以快照为准进行读取，忽略新数据\r\n  \r\n  * 为了提升读写性能，Tera加入内存compact功能\r\n   * 内存由一个单独的skiplist变为复杂的level结构\r\n   * 上述快照保护实现有遗漏，导致出现新版本未生效、旧版本被删除等情况，读取失败\r\n\r\n影响：\r\n  * 可能发生读取失败，读到非常旧的数据或读不到数据\r\n   * 更新和读的时序有交错的时候，可能触发\r\n   * 内存compact和读的时序有交错的时候，可能触发\r\n   * 更新和读越频繁，时序交错的概率越大，出错概率越大\r\n\r\n解决方法：\r\n  * 读取和更新时，内存compact加上快照保护"},"comment":{"url":"https://api.github.com/repos/baidu/tera/issues/comments/202797384","html_url":"https://github.com/baidu/tera/issues/782#issuecomment-202797384","issue_url":"https://api.github.com/repos/baidu/tera/issues/782","id":202797384,"user":{"login":"xupeilin","id":11419707,"avatar_url":"https://avatars.githubusercontent.com/u/11419707?v=3","gravatar_id":"","url":"https://api.github.com/users/xupeilin","html_url":"https://github.com/xupeilin","followers_url":"https://api.github.com/users/xupeilin/followers","following_url":"https://api.github.com/users/xupeilin/following{/other_user}","gists_url":"https://api.github.com/users/xupeilin/gists{/gist_id}","starred_url":"https://api.github.com/users/xupeilin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/xupeilin/subscriptions","organizations_url":"https://api.github.com/users/xupeilin/orgs","repos_url":"https://api.github.com/users/xupeilin/repos","events_url":"https://api.github.com/users/xupeilin/events{/privacy}","received_events_url":"https://api.github.com/users/xupeilin/received_events","type":"User","site_admin":false},"created_at":"2016-03-29T09:23:20Z","updated_at":"2016-03-29T09:23:20Z","body":"#801 "}}