{"action":"created","issue":{"url":"https://api.github.com/repos/mbdavid/LiteDB/issues/327","repository_url":"https://api.github.com/repos/mbdavid/LiteDB","labels_url":"https://api.github.com/repos/mbdavid/LiteDB/issues/327/labels{/name}","comments_url":"https://api.github.com/repos/mbdavid/LiteDB/issues/327/comments","events_url":"https://api.github.com/repos/mbdavid/LiteDB/issues/327/events","html_url":"https://github.com/mbdavid/LiteDB/issues/327","id":184633389,"number":327,"title":"Is 4096 a good choice for page size?","user":{"login":"nerai","id":1230531,"avatar_url":"https://avatars.githubusercontent.com/u/1230531?v=3","gravatar_id":"","url":"https://api.github.com/users/nerai","html_url":"https://github.com/nerai","followers_url":"https://api.github.com/users/nerai/followers","following_url":"https://api.github.com/users/nerai/following{/other_user}","gists_url":"https://api.github.com/users/nerai/gists{/gist_id}","starred_url":"https://api.github.com/users/nerai/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nerai/subscriptions","organizations_url":"https://api.github.com/users/nerai/orgs","repos_url":"https://api.github.com/users/nerai/repos","events_url":"https://api.github.com/users/nerai/events{/privacy}","received_events_url":"https://api.github.com/users/nerai/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2016-10-22T14:28:32Z","updated_at":"2016-10-22T19:55:28Z","closed_at":null,"body":"Currently, the page size is fixed at 4096 bytes, which is the Windows size for disk access - a good fit.\r\n\r\nHowever, considering that traditional hard disks are far faster at sequential than random access, it might be faster if the page size were larger.\r\n\r\nObviously, this would lead to more wasted time when reading only a single element from a page. However, this waste is probably very small, because the disk drive does not really care if you read 1 or 2 (or 5) pages - it's only a microsecond or so more, since the head is already at the right place of the disk.\r\n\r\nAt the same time, the larger page would contain a lot more co-located information, which is greatly advantageous for e.g. full scans. In short, we get a good chance for a significant performance improvement for almost zero cost.\r\n\r\nThis makes it look like an increase page size  (e.g. 8kb or 16kb or even more) would be beneficial.\r\n\r\nOn the other hand, I see these problems with an increased page size:\r\n\r\n- When increasing volume too much, disk access does become slower, so we should not increase the page size to something ridiculous like 10 MB.\r\n- Fragmentation within a page may be higher (not sure, did not check if this can be the case)\r\n- Parsing the contents of a single page happens linearly, so parsing becomes slower (should not be a bottleneck, though, as disk access is magnitudes slower)\r\n- The operating system may have worse caching behavior due to more, unnecessary pages read from disk. For instance, with 16kb pages, the OS can probably only cache 25% as many pages, which is bad in case of repeated reads of single elements in many different pages\r\n"},"comment":{"url":"https://api.github.com/repos/mbdavid/LiteDB/issues/comments/255550221","html_url":"https://github.com/mbdavid/LiteDB/issues/327#issuecomment-255550221","issue_url":"https://api.github.com/repos/mbdavid/LiteDB/issues/327","id":255550221,"user":{"login":"mbdavid","id":1137197,"avatar_url":"https://avatars.githubusercontent.com/u/1137197?v=3","gravatar_id":"","url":"https://api.github.com/users/mbdavid","html_url":"https://github.com/mbdavid","followers_url":"https://api.github.com/users/mbdavid/followers","following_url":"https://api.github.com/users/mbdavid/following{/other_user}","gists_url":"https://api.github.com/users/mbdavid/gists{/gist_id}","starred_url":"https://api.github.com/users/mbdavid/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mbdavid/subscriptions","organizations_url":"https://api.github.com/users/mbdavid/orgs","repos_url":"https://api.github.com/users/mbdavid/repos","events_url":"https://api.github.com/users/mbdavid/events{/privacy}","received_events_url":"https://api.github.com/users/mbdavid/received_events","type":"User","site_admin":false},"created_at":"2016-10-22T19:55:28Z","updated_at":"2016-10-22T19:55:28Z","body":"Hi @nerai, when I decide to 4096 I search about all databases use. Many databases you can define page size, which is really nice do advanced users (really advanced - because it's hard to know if your choice will be best). Not only page size, but PCT Free and PCT Used is fine tune for databases. LiteDB works with 50% of PCT Free/Used.\r\n\r\nLiteDB works with whole page operation for read and for write. So, change a single document, saves all affected pages (could be only 1 DataPage and N PageIndexes - if index keys changed). \r\n\r\nA long time ago I made some performance tests using 4k, 8k and 16k with no expressive diference.\r\n\r\nThe next version works with a single instance in memory and this will be really nice to performance. Mosted pages will be in memory (cache) and write in disk small pages could be faster than write bigger pages.\r\n\r\nDo you have idea how to do good test performance to compare?"}}
{"actor":{"display_login":"mbdavid"}}