{"action":"created","issue":{"url":"https://api.github.com/repos/google/stenographer/issues/149","repository_url":"https://api.github.com/repos/google/stenographer","labels_url":"https://api.github.com/repos/google/stenographer/issues/149/labels{/name}","comments_url":"https://api.github.com/repos/google/stenographer/issues/149/comments","events_url":"https://api.github.com/repos/google/stenographer/issues/149/events","html_url":"https://github.com/google/stenographer/issues/149","id":191912534,"number":149,"title":"Slow start up time for `stenographer`","user":{"login":"yew011","id":2474386,"avatar_url":"https://avatars.githubusercontent.com/u/2474386?v=3","gravatar_id":"","url":"https://api.github.com/users/yew011","html_url":"https://github.com/yew011","followers_url":"https://api.github.com/users/yew011/followers","following_url":"https://api.github.com/users/yew011/following{/other_user}","gists_url":"https://api.github.com/users/yew011/gists{/gist_id}","starred_url":"https://api.github.com/users/yew011/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/yew011/subscriptions","organizations_url":"https://api.github.com/users/yew011/orgs","repos_url":"https://api.github.com/users/yew011/repos","events_url":"https://api.github.com/users/yew011/events{/privacy}","received_events_url":"https://api.github.com/users/yew011/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":7,"created_at":"2016-11-28T03:32:34Z","updated_at":"2016-11-29T16:50:36Z","closed_at":null,"body":"When `stenographer` starts up, it tries to recover all the existing\r\npacket/index files.  While doing so, it will call the `NewIndexFile ()`\r\nfunction in which it tries few small reads of `leveldb` for initialization\r\nand version confirmation...  And this recovery is done serially for\r\neach thread (here)[https://github.com/google/stenographer/blob/master/env/env.go#L258-L262]\r\nwhile preventing any query from `stenoread`.\r\n\r\nOn a system will tens of thousand files, this process could take a\r\nnon-negligible amount of time,\r\n\r\nSo, @gconnell, would like to ask if you know about the problem and\r\nif you do any workaround to speed it up,\r\n"},"comment":{"url":"https://api.github.com/repos/google/stenographer/issues/comments/263627388","html_url":"https://github.com/google/stenographer/issues/149#issuecomment-263627388","issue_url":"https://api.github.com/repos/google/stenographer/issues/149","id":263627388,"user":{"login":"gconnell","id":1211830,"avatar_url":"https://avatars.githubusercontent.com/u/1211830?v=3","gravatar_id":"","url":"https://api.github.com/users/gconnell","html_url":"https://github.com/gconnell","followers_url":"https://api.github.com/users/gconnell/followers","following_url":"https://api.github.com/users/gconnell/following{/other_user}","gists_url":"https://api.github.com/users/gconnell/gists{/gist_id}","starred_url":"https://api.github.com/users/gconnell/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gconnell/subscriptions","organizations_url":"https://api.github.com/users/gconnell/orgs","repos_url":"https://api.github.com/users/gconnell/repos","events_url":"https://api.github.com/users/gconnell/events{/privacy}","received_events_url":"https://api.github.com/users/gconnell/received_events","type":"User","site_admin":false},"created_at":"2016-11-29T16:50:36Z","updated_at":"2016-11-29T16:50:36Z","body":"Point 1 is easily fixed, I've committed it to the branch.\r\nM'm not sure why 2 is required?  The write to `t.files` in `syncFilesWithDisk` and `trackNewFile` is protected with a `t.mu.Lock()`, and Lookup is protected with a `t.mu.RLock()`.  Also, I'm not seeing anywhere that `t.files[x]` is set to nil.\r\n\r\nRe: HTTP response, I think the best bet is still to block incoming requests until the first sync is complete, then best-effort any future syncs.  I've committed that as well to syncfiles."}}
{"actor":{"display_login":"gconnell"}}