{"action":"opened","issue":{"url":"https://api.github.com/repos/jruby/jruby/issues/3921","repository_url":"https://api.github.com/repos/jruby/jruby","labels_url":"https://api.github.com/repos/jruby/jruby/issues/3921/labels{/name}","comments_url":"https://api.github.com/repos/jruby/jruby/issues/3921/comments","events_url":"https://api.github.com/repos/jruby/jruby/issues/3921/events","html_url":"https://github.com/jruby/jruby/issues/3921","id":156632193,"number":3921,"title":"Performance regression with `json/pure` from 1.7 to 9k","user":{"login":"cprice404","id":1341071,"avatar_url":"https://avatars.githubusercontent.com/u/1341071?v=3","gravatar_id":"","url":"https://api.github.com/users/cprice404","html_url":"https://github.com/cprice404","followers_url":"https://api.github.com/users/cprice404/followers","following_url":"https://api.github.com/users/cprice404/following{/other_user}","gists_url":"https://api.github.com/users/cprice404/gists{/gist_id}","starred_url":"https://api.github.com/users/cprice404/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cprice404/subscriptions","organizations_url":"https://api.github.com/users/cprice404/orgs","repos_url":"https://api.github.com/users/cprice404/repos","events_url":"https://api.github.com/users/cprice404/events{/privacy}","received_events_url":"https://api.github.com/users/cprice404/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":0,"created_at":"2016-05-24T23:31:17Z","updated_at":"2016-05-24T23:31:17Z","closed_at":null,"body":"<!--\r\nThis is a simple template for filing JRuby isuses.\r\n\r\nPlease help us help you by providing the information below.\r\n\r\nText inside XML comment tags will not be shown in your report.\r\n-->\r\n\r\n### Environment\r\n\r\n<!--\r\nProvide at least:\r\n* JRuby version (`jruby -v`) and command line (flags, JRUBY_OPTS, etc)\r\n* Operating system and platform (e.g. `uname -a`)\r\n\r\nOther relevant info you may wish to add:\r\n* Installed or activated gems\r\n* Application/framework version (e.g. Rails, Sinatra)\r\n* Environment variables\r\n-->\r\n\r\nJRuby 1.7.20.1, JRuby 9.1.0.0, JRuby 9.1.1.0; run from Java via `ScriptingContainer`, with various values tested for the `setCompileMode` method.  Standalone script execution with no frameworks.  Benchmarked JSON gems `json/pure`, `json/ext`, and `jrjackson`.\r\n\r\n### Expected Behavior\r\n\r\n<!--\r\nDescribe your expectation of how JRuby should behave.\r\nProvide an executable Ruby script or a link to an example repository.\r\n-->\r\n\r\nExpected performance for 9.1.x to be on par with performance for 1.7.x.\r\n\r\n### Actual Behavior\r\n\r\n<!--\r\nDescribe or show the actual behavior.\r\nProvide text or screen capture showing the behavior.\r\n-->\r\n\r\nFor `json/pure`, JRuby 9k performance is *markedly* slower (75%) when `compileMode` is set to `OFF`.\r\n\r\nI have set up a github repo with a simple reproducer here:\r\nhttps://github.com/cprice404/jruby9k-benchmarks/tree/19a2719e4edddc17e3f1fabbf311b9d80c6eba80/json-comparison\r\n\r\nI also graphed some of the results in a google spreadsheet here:\r\nhttps://docs.google.com/spreadsheets/d/1M64QXK_RSFd05_Te9PamTv7ALi7XlAkhWeyRoAoIFZU/edit#gid=866554277\r\n\r\nHere are some relevant numbers from the benchmark output (the rest of it is available in the linked reproducer repro if you are interested):\r\n\r\n```\r\nJRUBY VERSION: 1.7.20.1\r\nJSON VERSION: json/pure\r\nCOMPILE MODE: off\r\n              user     system      total        real\r\nwarmup (100 runs): 79.419000   0.000000  79.419000 ( 79.418000)\r\nmiddle (1000 runs):754.537000   0.000000 754.537000 (754.537000)\r\ntail (100 runs): 74.968000   0.000000  74.968000 ( 74.968000)\r\n\r\nJRUBY VERSION: 9.1.1.0\r\nJSON VERSION: json/pure\r\nCOMPILE MODE: off\r\n              user     system      total        real\r\nwarmup (100 runs):137.360000   1.240000 138.600000 (127.982415)\r\nmiddle (1000 runs):1352.780000   1.620000 1354.400000 (1327.873641)\r\ntail (100 runs):149.260000   0.350000 149.610000 (147.532336)\r\n```\r\n\r\nThe other JSON libraries besides `json/pure` are much, much faster, and are in the ballpark of JRuby 1.7 performance.  Also, when using `json/pure` with any other `compileMode` besides `OFF`, the delta between JRuby 1.7 and JRuby 9k drops significantly.\r\n\r\nUnfortunately, our application codebase currently vendors a fork of `json/pure` that will make it very difficult for us to switch to a different JSON backend.\r\n\r\nAlso, we haven't had great luck in our experiments with using `compileMode` `JIT`, most likely because it seems to be very expensive during the warmup of a `ScriptingContainer`, and we destroy and create new `ScriptingContainer`s with some frequency during our application execution.\r\n\r\nI recognize that many users will be able to avoid using `json/pure`, and thus the priority of solving this may be diminished; however, my hope is that perhaps the reproducer highlights some kind of memory usage or performance issue that would be relevant to other JRuby applications, and thus, that looking into optimizations for this scenario might yield improvements in other areas as well.  Or, at least, that it would be interesting to try to characterize why the performance changes so drastically between JRuby 1.7 and 9k."}}
{"actor":{"display_login":"cprice404"}}