{"action":"created","issue":{"url":"https://api.github.com/repos/nlohmann/json/issues/202","labels_url":"https://api.github.com/repos/nlohmann/json/issues/202/labels{/name}","comments_url":"https://api.github.com/repos/nlohmann/json/issues/202/comments","events_url":"https://api.github.com/repos/nlohmann/json/issues/202/events","html_url":"https://github.com/nlohmann/json/issues/202","id":129940920,"number":202,"title":"Performance in miloyip/nativejson-benchmark","user":{"login":"twelsby","id":16697318,"avatar_url":"https://avatars.githubusercontent.com/u/16697318?v=3","gravatar_id":"","url":"https://api.github.com/users/twelsby","html_url":"https://github.com/twelsby","followers_url":"https://api.github.com/users/twelsby/followers","following_url":"https://api.github.com/users/twelsby/following{/other_user}","gists_url":"https://api.github.com/users/twelsby/gists{/gist_id}","starred_url":"https://api.github.com/users/twelsby/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/twelsby/subscriptions","organizations_url":"https://api.github.com/users/twelsby/orgs","repos_url":"https://api.github.com/users/twelsby/repos","events_url":"https://api.github.com/users/twelsby/events{/privacy}","received_events_url":"https://api.github.com/users/twelsby/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":9,"created_at":"2016-01-30T05:04:21Z","updated_at":"2016-01-30T19:42:00Z","closed_at":null,"body":"Here is the results of testing 1.1.0, 2.0.0 and my proposed changes to 2.0.0 under miloyip/nativejson-benchmark [Results.zip](https://github.com/nlohmann/json/files/110681/Results.zip).\r\n\r\nHere is my (possibly naive) interpretation:\r\n\r\n**Parse**\r\nOur lexer looks to be pretty good and parsing of strings is about as good as it could be.  Some further improvements could be made in number parsing by incorporating custom floating point parsing into the proposed `get_integer()` in lieu of `strtof()`/`strtod()`/`strtold()` but this would greatly increase code complexity and the improvements would likely be marginal at best.\r\n\r\nLooking at the other libraries results (especially the ones with fast parsing) it looks like allocations are what is killing us.  The other thing that might be killing us is the use of map.  This is not ideal for parsing a large unordered file because the tree will be continually re-balanced.  The benefit of using a map is lookup, but that isn't really tested in this benchmark as the timed access is all sequential.  \r\n\r\nNote that I'm not saying we should be moving away from map, because our library is really designed around using the json object to store data, not merely to parse/dump json files.  We probably just have to accept that our parse is always going to be slower.\r\n\r\n**Stringify**\r\nThe proposed change to `snprintf()` for floating point stringifying highlights just how bad `stringstream`s are for this.  There may be further improvements by reducing our reliance on `stringstream`s elsewhere but this would come at the expense of c++ style.  I did try using `snprintf()` for integers, and even a very efficient custom function I've used before on microcontrollers, but didn't achieve any significant improvement so perhaps the issue is only manifest for floating point numbers.\r\n\r\nThe other thing here is that use of map is probably not as fast for sequential dumping as other container types.  I think this is just a price we have to pay.\r\n\r\n**Time**\r\nThis test simply iterates through the document while collecting statistics about number of strings, numbers, objects, arrays etc.  Any difference between library versions is an anomaly as there are no significant differences (only the bitfield overhead for the proposed changes).  Because this test is sequential we are at a disadvantage using maps.\r\n\r\nThe bottom line seems to be that a choice must be made - do we optimize for rapid parsing/dumping/sequential access at the expense of lookup, or do we optimize for lookup and accept that parsing/dumping/sequential access will always be a little slower (but not that much).\r\n\r\n@nlohmann, if you are able to run similar tests under Linux/OSX that would help to make sure this isn't just a VS2015 issue."},"comment":{"url":"https://api.github.com/repos/nlohmann/json/issues/comments/177282220","html_url":"https://github.com/nlohmann/json/issues/202#issuecomment-177282220","issue_url":"https://api.github.com/repos/nlohmann/json/issues/202","id":177282220,"user":{"login":"gregmarr","id":8569738,"avatar_url":"https://avatars.githubusercontent.com/u/8569738?v=3","gravatar_id":"","url":"https://api.github.com/users/gregmarr","html_url":"https://github.com/gregmarr","followers_url":"https://api.github.com/users/gregmarr/followers","following_url":"https://api.github.com/users/gregmarr/following{/other_user}","gists_url":"https://api.github.com/users/gregmarr/gists{/gist_id}","starred_url":"https://api.github.com/users/gregmarr/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gregmarr/subscriptions","organizations_url":"https://api.github.com/users/gregmarr/orgs","repos_url":"https://api.github.com/users/gregmarr/repos","events_url":"https://api.github.com/users/gregmarr/events{/privacy}","received_events_url":"https://api.github.com/users/gregmarr/received_events","type":"User","site_admin":false},"created_at":"2016-01-30T19:42:00Z","updated_at":"2016-01-30T19:42:00Z","body":"Could be lots of hash collisions. I would expect better performance on lookups, but not much change elsewhere.  The flat containers from boost might only help in the memory usage and output, insertion may be worse."}}