{"action":"created","issue":{"url":"https://api.github.com/repos/elastic/elasticsearch/issues/18060","repository_url":"https://api.github.com/repos/elastic/elasticsearch","labels_url":"https://api.github.com/repos/elastic/elasticsearch/issues/18060/labels{/name}","comments_url":"https://api.github.com/repos/elastic/elasticsearch/issues/18060/comments","events_url":"https://api.github.com/repos/elastic/elasticsearch/issues/18060/events","html_url":"https://github.com/elastic/elasticsearch/pull/18060","id":151807909,"number":18060,"title":"Avoid sliced locked contention in internal engine","user":{"login":"jasontedor","id":4744941,"avatar_url":"https://avatars.githubusercontent.com/u/4744941?v=3","gravatar_id":"","url":"https://api.github.com/users/jasontedor","html_url":"https://github.com/jasontedor","followers_url":"https://api.github.com/users/jasontedor/followers","following_url":"https://api.github.com/users/jasontedor/following{/other_user}","gists_url":"https://api.github.com/users/jasontedor/gists{/gist_id}","starred_url":"https://api.github.com/users/jasontedor/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jasontedor/subscriptions","organizations_url":"https://api.github.com/users/jasontedor/orgs","repos_url":"https://api.github.com/users/jasontedor/repos","events_url":"https://api.github.com/users/jasontedor/events{/privacy}","received_events_url":"https://api.github.com/users/jasontedor/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/elastic/elasticsearch/labels/:Core","name":":Core","color":"009800"},{"url":"https://api.github.com/repos/elastic/elasticsearch/labels/enhancement","name":"enhancement","color":"4a4ea8"},{"url":"https://api.github.com/repos/elastic/elasticsearch/labels/review","name":"review","color":"0052cc"},{"url":"https://api.github.com/repos/elastic/elasticsearch/labels/v5.0.0","name":"v5.0.0","color":"dddddd"}],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":8,"created_at":"2016-04-29T07:22:28Z","updated_at":"2016-04-30T16:12:37Z","closed_at":"2016-04-29T12:05:23Z","pull_request":{"url":"https://api.github.com/repos/elastic/elasticsearch/pulls/18060","html_url":"https://github.com/elastic/elasticsearch/pull/18060","diff_url":"https://github.com/elastic/elasticsearch/pull/18060.diff","patch_url":"https://github.com/elastic/elasticsearch/pull/18060.patch"},"body":"Today we use a sliced lock strategy for acquiring locks to prevent\r\nconcurrent updates to the same document. The number of sliced locks is\r\ncomputed as a linear function of the number of logical\r\nprocessors. Unfortunately, the probability of a collision against a\r\nsliced lock is prone to the birthday problem and grows faster than\r\nexpected. In fact, the mathematics works out such that for a fixed\r\ntarget probability of collision, the number of lock slices should grow\r\nlike the square of the number of logical processors. This is\r\nless-than-ideal, and we can do better anyway. This commit introduces a\r\nstrategy for avoiding lock contention within the internal\r\nengine. Ideally, we would only have lock contention if there were\r\nconcurrent updates to the same document. We can get close to this ideal\r\nworld by associating a lock with the ID of each document. This\r\nassociation can be held in a concurrent hash map. Now, the JDK\r\nConcurrentHashMap also uses a sliced lock internally, but it has several\r\nstrategies for avoiding taking the locks and these locks are only held\r\nfor a very short period of time. This implementation associates a\r\nreference count with the lock that is associated with a document ID and\r\nautomatically removes the document ID from the concurrent hash map when\r\nthe reference count reaches zero.\r\n\r\nCloses #18053"},"comment":{"url":"https://api.github.com/repos/elastic/elasticsearch/issues/comments/215975894","html_url":"https://github.com/elastic/elasticsearch/pull/18060#issuecomment-215975894","issue_url":"https://api.github.com/repos/elastic/elasticsearch/issues/18060","id":215975894,"user":{"login":"jasontedor","id":4744941,"avatar_url":"https://avatars.githubusercontent.com/u/4744941?v=3","gravatar_id":"","url":"https://api.github.com/users/jasontedor","html_url":"https://github.com/jasontedor","followers_url":"https://api.github.com/users/jasontedor/followers","following_url":"https://api.github.com/users/jasontedor/following{/other_user}","gists_url":"https://api.github.com/users/jasontedor/gists{/gist_id}","starred_url":"https://api.github.com/users/jasontedor/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jasontedor/subscriptions","organizations_url":"https://api.github.com/users/jasontedor/orgs","repos_url":"https://api.github.com/users/jasontedor/repos","events_url":"https://api.github.com/users/jasontedor/events{/privacy}","received_events_url":"https://api.github.com/users/jasontedor/received_events","type":"User","site_admin":false},"created_at":"2016-04-30T16:12:37Z","updated_at":"2016-04-30T16:12:37Z","body":"Some of the nightly [benchmarks](https://benchmarks.elastic.co/index.html) showed a healthy bump: \r\n\r\n<img width=\"525\" alt=\"screen shot 2016-04-30 at 12 12 06 pm\" src=\"https://cloud.githubusercontent.com/assets/4744941/14937161/c9bea7da-0ecc-11e6-8fd3-02c0c22645eb.png\">\r\n"}}