{"action":"created","issue":{"url":"https://api.github.com/repos/elastic/elasticsearch/issues/18776","repository_url":"https://api.github.com/repos/elastic/elasticsearch","labels_url":"https://api.github.com/repos/elastic/elasticsearch/issues/18776/labels{/name}","comments_url":"https://api.github.com/repos/elastic/elasticsearch/issues/18776/comments","events_url":"https://api.github.com/repos/elastic/elasticsearch/issues/18776/events","html_url":"https://github.com/elastic/elasticsearch/issues/18776","id":159097178,"number":18776,"title":"Creating Index painfully slow on cluster with large indices ","user":{"login":"puuzll","id":6858674,"avatar_url":"https://avatars.githubusercontent.com/u/6858674?v=3","gravatar_id":"","url":"https://api.github.com/users/puuzll","html_url":"https://github.com/puuzll","followers_url":"https://api.github.com/users/puuzll/followers","following_url":"https://api.github.com/users/puuzll/following{/other_user}","gists_url":"https://api.github.com/users/puuzll/gists{/gist_id}","starred_url":"https://api.github.com/users/puuzll/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/puuzll/subscriptions","organizations_url":"https://api.github.com/users/puuzll/orgs","repos_url":"https://api.github.com/users/puuzll/repos","events_url":"https://api.github.com/users/puuzll/events{/privacy}","received_events_url":"https://api.github.com/users/puuzll/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2016-06-08T07:29:40Z","updated_at":"2016-10-08T01:33:15Z","closed_at":"2016-06-08T20:31:33Z","body":"<!--\r\nGitHub is reserved for bug reports and feature requests. The best place\r\nto ask a general question is at the Elastic Discourse forums at\r\nhttps://discuss.elastic.co. If you are in fact posting a bug report or\r\na feature request, please include one and only one of the below blocks\r\nin your new issue.\r\n-->\r\n\r\nI have a cluster with 2 nodes and approximate 10000 index.Each index has one replication. Creating index with some preset mapping on this cluster is painfully slow( about 1 minute to create a index with 1 replication). There is sufficient  memory , java heap , cpu and disk when creating index. I use hot_threads api and find that 95% period of time is spended on running the following code on master node:\r\n```\r\n\tat com.google.common.collect.Iterators$3.hasNext(Iterators.java:164)\r\n\tat org.elasticsearch.indices.cluster.IndicesClusterStateService.applyDeletedShards(IndicesClusterStateService.java:256)\r\n\tat org.elasticsearch.indices.cluster.IndicesClusterStateService.clusterChanged(IndicesClusterStateService.java:167)\r\n\t- locked <0x00000000f96ea8b0> (a java.lang.Object)\r\n\tat org.elasticsearch.cluster.service.InternalClusterService.runTasksForExecutor(InternalClusterService.java:610)\r\n\tat org.elasticsearch.cluster.service.InternalClusterService$UpdateTask.run(InternalClusterService.java:772)\r\n\tat org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:231)\r\n\tat org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:194)\r\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)\r\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)\r\n\tat java.lang.Thread.run(Thread.java:745)\r\n```\r\nis this a bug? Can I avoid this by setting any configuration? \r\n**Elasticsearch version**:\r\n2.3.3\r\n**JVM version**:\r\n1.8\r\n**OS version**:\r\ndebian7\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/elastic/elasticsearch/issues/comments/252394235","html_url":"https://github.com/elastic/elasticsearch/issues/18776#issuecomment-252394235","issue_url":"https://api.github.com/repos/elastic/elasticsearch/issues/18776","id":252394235,"user":{"login":"jilen","id":546573,"avatar_url":"https://avatars.githubusercontent.com/u/546573?v=3","gravatar_id":"","url":"https://api.github.com/users/jilen","html_url":"https://github.com/jilen","followers_url":"https://api.github.com/users/jilen/followers","following_url":"https://api.github.com/users/jilen/following{/other_user}","gists_url":"https://api.github.com/users/jilen/gists{/gist_id}","starred_url":"https://api.github.com/users/jilen/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jilen/subscriptions","organizations_url":"https://api.github.com/users/jilen/orgs","repos_url":"https://api.github.com/users/jilen/repos","events_url":"https://api.github.com/users/jilen/events{/privacy}","received_events_url":"https://api.github.com/users/jilen/received_events","type":"User","site_admin":false},"created_at":"2016-10-08T01:33:15Z","updated_at":"2016-10-08T01:33:15Z","body":"@jasontedor   I suffered from this. Is there any way to improve the index creation speed ?"}}
{"actor":{"display_login":"jilen"}}